82 Commits

Author SHA1 Message Date
catbref
840f52ff90 Added API call GET /blocks/timestamp/{timestamp} and increase timeout on POST /admin/forcesync from 5s to 30s 2019-08-06 09:47:38 +01:00
catbref
e47b4dceb2 StringBuilder, and other, optimizations for repository-related classes.
Add optional "excludeZero" to API call GET /assets/balances

Added tests to call most API calls to check no exceptions are thrown.
2019-08-06 09:46:31 +01:00
catbref
fc82bcaf49 Tidying up peer info.
lastHeight/blockSig/blockTimestamp, etc. moved from PeerData/repository
to Peer as it's transient so no need to store in repository.

Repository now keeps track of when/who added peer, e.g. API/INIT/another peer.

API calls DELETE /peers and DELETE /peers/known also disconnect peer
as well as deleting from repository.

Connection timestamp now reported by API call GET /peers

Some repository-updating code removed from Network/Controller as no
longer needed.

Removed obsolete Controller.hasShorterBlockchain predicate.
2019-08-02 14:56:19 +01:00
catbref
8b135eb447 Improve startup & shutdown. Improve API call POST /admin/forcesync.
Previous version was prone to throwing exceptions during shutdown
sequence, especially in Peer/Network-related threads.

Shutdown now tries to wait for Peer/Network threads to cleanly exit.

API call POST /admin/forcesync now tries to grab blockchain lock,
with timeout, and then repeatedly sync with requested peer until
either fully synced or something unexpected happens.
2019-08-02 14:54:22 +01:00
catbref
877c126ba3 Don't allow incorrect/null ARBITRARY dataType when building transactions via
API's POST /arbitrary call.
2019-08-02 14:14:56 +01:00
catbref
4cdb5e6b95 Auto-Update improvements
NOTE: Downloaded update JARs are now expected to have been XORed with 0x5A!

This is to help prevent Windows Firewall from blocking update downloads
based on deep packet inspection.

Download read timeout reduced from 5s to 3s.

Download locations reordered so github entries are at the top as they have
better CDNs.

ApplyUpdate now assumes null response from GET /admin/stop means node
is not running.

ApplyUpdate now checks replacement JAR actually exists before attempting
to overwrite previous version.

ApplyUpdate now tries to use Windows EXE launcher in preference to raw
java command line. (This should improve Windows installer behaviour
in detecting running process and possibly firewall implications too).
2019-08-02 14:13:04 +01:00
catbref
4f25fffbe9 Change API transaction subsmission from single-shot to 500ms timeout for obtaining blockchain lock 2019-08-02 13:04:21 +01:00
catbref
02e8bdb034 Add support for fetching updates using a combination of hostname and IP address.
IP address used to create socket, hostname used for SNI, HTTPS, etc.

Added hostname+IP auto-update locations to Settings.
2019-08-02 12:53:49 +01:00
catbref
c2e8392f05 Synchronization improvements (again!)
Bumped version

Controller no longer uses block height to determine whether to sync
but now uses peer's latest block's timestamp and signature.

Also BlockGenerator checks whether it's generating in isolation using
the same peer info (latest block timestamp and signature).

Added API call POST /admin/forcesync peer-address to help get wayward
nodes back on track.

Unified code around, and calling, Transaction.importAsUnconfirmed().

Tidied code around somelock.tryLock() to be more readable.

Controller (post-sync) now broadcasts new chaintip info if
our latest block's signature has changed, not simply the height.

Network.broadcast() only sends out via outbound peer if node has
more than one connection from the same peer. So Controller would
only update one of the peer records with chaintip info.
Controller now updates all connected peers with the ID when it
receives a HEIGHT or HEIGHT_V2 message.

Added node1 thru node7.mcfamily.io to default peers in Network.

Network ignores first "listen port" entry when receiving peers
list from an outbound-connection peer as it already knows
by virtue of having connected to it!

More network message debug logging (hopefully never to be seen).

[some old code left in, but commented out, for a while]
2019-07-23 17:50:07 +01:00
catbref
d910cce807 Improve API peer info output 2019-07-23 17:38:56 +01:00
catbref
b7f53afe68 Improve API output for GET /peers 2019-07-23 16:18:01 +01:00
catbref
3a3cc5a81b Improve API output of GET /blocks/forgers 2019-07-23 16:17:42 +01:00
catbref
dcb8d67236 Don't allow new transactions to be submitted if not up to date 2019-07-23 15:59:01 +01:00
catbref
a2cc4983a0 Add "involved" optional query param to API call GET /address/proxying, for convenience 2019-07-23 15:26:26 +01:00
catbref
c9968b3dd2 Interim commit of *TransactionData classes for safety 2019-07-23 14:42:02 +01:00
catbref
a4f2cf50b0 Updated node management UI (was previously "bundled ui")
Fix root path redirects for node management UI and API documentation servlets.
2019-07-23 12:44:10 +01:00
catbref
b2f235f7a5 Changes to API regarding proxy forging calls 2019-07-23 11:24:25 +01:00
catbref
8f7c954f5a Proxy private keys are now SHA256(shared secret only) instead of SHA256(shared secret + public keys).
HTML/JS in src/test/resources/proxy-key-example.html updated accordingly.

Add handshake status to output of API call GET /peers

Add/correct @ApiErrors annotations on some API calls.

Add API call POST /admin/orphan (target height as body)
to force blockchain orphaning for when node is wildly out of sync.
Added support for above to BlockChain class.

BlockGenerator now requires a minimum number of peers
before it will generate any new blocks.
See "minBlockchainPeers" in settings.

Controller now requires a minimum number of peers
before it will consider synchronizing.
See "minBlockchainPeers" in settings.

Old "minPeers" entry in settings.json no longer valid!

Networking now allows both an outbound and inbound connection
to a peer although will use the outbound connection in preference.

Networking checks peer ID of inbound connections to detect,
and resolve, peer ID clashes/theft.
2019-07-23 11:09:25 +01:00
catbref
40d6190265 API tidy up
GET /addresses/proxykey/{privkey}/{pubkey} now POST /addresses/proxykey

GET /utils/fromBase58 now GET /utils/frombase58
GET /utils/fromBase64 now GET /utils/frombase64

GET /utils/toBase58/{hex} now GET /utils/tobase58/{hex}
GET /utils/toBase64/{hex} now GET /utils/tobase64/{hex}

GET /utils/privateKey/{entropy} now POST /utils/privatekey
GET /utils/publicKey/{privateKey} now POST /utils/publickey
2019-07-23 10:42:23 +01:00
catbref
0259702df2 Fix generating X25519 shared secret.
X25519 shared secrets now match those generated by libsodium.

New tests show that shared secrets are the same using either set
of private+public key combinations.

Changed proxy private key generation from using simple SHA256
of shared secret to using SHA256(shared secret + both public keys).

Added a temporary "BouncyCastle25519" shim class to provide missing
key conversion from Ed25519 to X25519.
2019-07-23 10:29:18 +01:00
catbref
748dddcc32 Add checks to API call POST /admin/forgingaccounts.
Now only accepts private keys for accounts with minting rights
or derives to known proxy forging public key.
2019-07-23 10:15:17 +01:00
catbref
ae6b41a893 Improve API call showing block forger summaries to include proxy info 2019-07-23 10:11:21 +01:00
catbref
bf8a12d422 Delete old, basic block-explorer 2019-07-23 10:07:55 +01:00
catbref
63a36073ec Work on auto-update + repository path + Settings bugfixes
AutoUpdate needs separate logfiles for its process as
log4j2's rolling appender doesn't support locking.
So we create AU_LOGGER in AutoUpdate and set it up
using static { } block.

Added support for starting/querying/stopping auto-update
Windows service.

Changed Controller's public connectionUrl field to
getRepositoryUrl() public static method so it
can incorporate repositoryPath from Settings.

Controller has added support for auto-update on startup.

Fix bugs with Settings not using userPath properly.

Removed obsolete standalone binary classes:
* blockgenerator
* txhex
* v1feeder
2019-07-18 15:26:41 +01:00
catbref
e4482c5ade Initial auto-update support, API improvements, arbitrary tx improvements
Removed all @Produces from API resources as response content type is
sorted by Swagger.

Added API /admin/info for generic node info.

Added API /arbitrary/ endpoints.

Moved arbitrary data storage from ArbitraryTransaction to ArbitraryRepository.

V4 arbitrary transaction signature is based on data's hash.

Original commit was d02f282, and commit message was:
Initial auto-update support, network MAGIC change, arbitrary tx improvements
2019-07-18 14:53:19 +01:00
catbref
747f5e41cf Bump Jetty to 9.4.17.v20190418 due to CVE-2019-10247
+ remove older version of package-info-maven-plugin
2019-04-30 09:00:30 +01:00
catbref
a316b8a810 Potential HSQLDB deadlock fix
After opening repository connection with RepositoryManager.getRepostory(),
any 'read' from repository (e.g. SELECT) starts the transaction
even though HSQLDB documentation states there are no shared/read locks
in MVCC concurrency model.

The work-around for this is to 'reset' HSQLDB's in-transaction flag
by performing a ROLLBACK (Repository.discardChanges) immediately
after acquiring the blockchain lock (which is used to ringfence
changes that might collide like these).

Also adding an extra check to prevent payments to nonexistent AT
addresses as it touches Transaction.
2019-04-29 15:18:44 +01:00
catbref
b4d0f9ab68 Allow multiple txType in API GET /transactions/search 2019-04-26 09:23:44 +01:00
catbref
0296c4bbb1 HSQLDB debugging & optimization for Transaction.isValidUnconfirmed 2019-04-25 18:29:01 +01:00
catbref
51fd029e22 Access to log entries from API & checks for some transaction-based API calls. 2019-04-24 15:02:37 +01:00
catbref
126e651f27 Change sync consensus to favour lower-value block sigs + other changes
API /addresses/{address} now returns lastReference taking unconfirmed into account.

Added DELETE /peers/known to remove all known peers from repository.

Added blockchain locking around Transaction methods like isValidUnconfirmed
as they (temporarily) update account lastReference.

Ditto getInvalidTransactions, etc.
2019-04-24 12:46:50 +01:00
catbref
d33ffee3ba Work on granting forging rights
Move hard-coded forging tiers to blockchain config.
Tests for granting forging rights.

Added API call to list top block forgers.

Fixed typo with Reward[s]ByHeight class name.
2019-04-19 09:55:04 +01:00
catbref
93230e9704 Add API call to list blocks with given generator. +more tests +pad genesis public key 2019-04-17 18:11:16 +01:00
catbref
d1c547f24a Refactor to use BouncyCastle Ed25519/X25519, and more...
Remove old whispersystems, etc. *25519 and use new v1.61 BouncyCastle.

Fix proxy forging private key derivation from X25519 shared secret.
Also include Javascript test version for comparison.

Fix block rewards for proxy forging.

Add extra useful info to API call GET /admin/forgingaccounts.
Fix API response to POST/DELETE /admin/forgingaccounts when
passed invalid private keys.

Added block rewards and account flags to testchain config.

Tests to cover changes above.
2019-04-17 12:32:03 +01:00
catbref
8e74884536 Fixes after rebase from master 2019-04-12 14:41:59 +01:00
catbref
2f6ef32f35 Replace settings-based generator private keys with DB table and API calls 2019-04-12 13:01:26 +01:00
catbref
c9035edd2c Interim proxy minting commit
Added /addresses/proxying to find proxy forging mappings.

Added /addresses/proxykey/{genprivkey}/{recipientpubkey} to calculate proxy private key.

New Block.regenerate factory method to create new Blocks
 but without having to reprocess ATs, etc.

Added support for proxied generator in Block.calcGeneratorsTarget

BlockGenerator now generates and checks new blocks for various generators,
 including proxy generators.

BlockGenerator now uses generator private keys supplied by Settings.
Corresponding changes to Settings to load base58-encoded private keys.

+ minor stuff
2019-04-12 12:57:21 +01:00
catbref
3c06d358b7 interim commit with proxy forging repository/transaction support
no block validity/generator support yet
2019-04-12 12:53:44 +01:00
catbref
cfbf5c12bf Supply extra information fields to various asset-related API calls.
e.g. supply "assetName" in JSON for TRANSFER_ASSET transactions

Also supply have/want asset names, amount asset ID/name, price-pair
and creator address in asset orders.

Show CREATE_ASSET_ORDER amount ID/name & price-pair in correct
format depending on whether transaction was placed before/after
'new' asset pricing took effect. (Orders are always in 'new' form).

Change API call /assets/transfers/{assetid}/{address} to
/assets/transfers/{assetid} with optional "address" query param.
2019-04-10 13:09:30 +01:00
catbref
a5e963911d New asset pricing scheme (take 2)
Orders are back to having "amount" and "price".
(No more "unitPrice" or "wantAmount").

Order "amount" is expressed in terms of asset with highest
assetID.
"price" is expressed in (lowest-assetID)/(highest-assetID).

Given an order with two assets, e.g. QORA (0) and GOLD (31),
"amount" is in GOLD (31), "price" is in QORA/GOLD (0/31).

Order's "fulfilled" is in the same asset as "amount".

Yet more tests and debugging.

For simplicity's sake, the change to HSQLDB repository is
assumed to take place when 'new' pricing switch also
occurs.

Don't forget to change "newAssetPricingTimestamp" in
blockchain config JSON file.
2019-04-10 07:18:50 +01:00
catbref
1b45ee85e7 Fix handling of CORS preflight OPTIONS requests.
Typical log entry would be:

2019-04-05 09:47:08 ERROR WadlGeneratorJAXBGrammarGenerator:401 - Failed to generate the schema for the JAX-B elements
com.sun.xml.internal.bind.v2.runtime.IllegalAnnotationsException: 4 counts of IllegalAnnotationExceptions
        at com.sun.xml.internal.bind.v2.runtime.IllegalAnnotationsException$Builder.check(IllegalAnnotationsException.java:91) ~[?:1.8.0_191]
2019-04-05 10:56:38 +01:00
catbref
7026c35e73 For API "GET /assets" add optional query param "includeData" 2019-04-04 12:33:12 +01:00
catbref
97142fdde8 Return HTTP 400 code with useful message when a ParamException occurs from API call. 2019-04-04 12:30:28 +01:00
catbref
315ebff61d Add optional API request logging via "apiLoggingEnabled" entry in Settings 2019-04-04 11:33:13 +01:00
catbref
26e3adb92b Completing work on new asset trading changes
Changed API call GET /assets to NOT return asset "data" fields
as they can be huge. If need be, call GET /assets/info to fetch
a specific asset's data field.

Improve asset trade amount granularity, especially for indivisible
assets, under "new" pricing scheme only.

Added corresponding tests for granularity adjustments.

Fix/unify asset order logging text under "old" and "new"
pricing schemes.

Change asset order related API data models so that old "price" is
now "unitPrice" and add new "return" as in amount of want-asset
to receive if have-asset "amount" was fully matched.
(Affects OrderData, CreateAssetOrderTransactionData)

Some changes to the HSQLDB tables.

Don't forget to add "newAssetPricingTimestamp" to your blockchain config's
"featureTriggers" map.
2019-04-03 18:00:20 +01:00
catbref
60e562566e Interim commit on new asset trading schema
Better order matching, especially in situations
where inexact fractional representations (e.g. 1/12)
or rounding issues might occur. Also better matching
with indivisible assets.

Essentially change ordering from have-amount & price
to have-amount and want-return, leaving unit price
to be calculated internally to a finer degree (in
some cases to 48 decimal points).

Corresponding unit tests to cover both legacy and new
scenarios. Support for tests to switch between
blockchain configs.

"New" pricing schema is its own 'feature trigger'
independent from general qorav2 switch.

Safety checks added during trading process.

HSQLDB schema changes (will probably need
careful conflict resolution on merge).

Still to do:

API changes
etc.
2019-04-02 21:10:16 +01:00
catbref
c69f84afab Added ApiExceptionManager to try to trace IllegalAnnotationExceptions 2019-03-26 14:34:45 +00:00
catbref
c5e5020369 In API call /assets/balances, change "ordering" to be optional but with default "ASSET_BALANCE_ACCOUNT" 2019-03-25 15:21:10 +00:00
catbref
a4d28d52cc Add variety of results ordering options to API GET /assets/balances 2019-03-25 12:24:22 +00:00
catbref
0fd8ee3547 Add API GET /assets/transfers/{assetid}/{address} 2019-03-25 11:55:03 +00:00