Commit Graph

212 Commits

Author SHA1 Message Date
catbref
921953d304 Clean up inbound peer entries from repository & fix sending peer lists 2019-07-23 15:36:14 +01:00
catbref
912ec22417 Minor warning fix 2019-07-23 15:26:52 +01:00
catbref
a2cc4983a0 Add "involved" optional query param to API call GET /address/proxying, for convenience 2019-07-23 15:26:26 +01:00
catbref
ab9fd681f3 Fix synchronizer so incoming block's transactions have approval status set 2019-07-23 15:13:59 +01:00
catbref
4667b768df Added tests for each group-approval outcome + fixes 2019-07-23 15:13:28 +01:00
catbref
c9f226cf88 group approval tests and fixes 2019-07-23 15:12:35 +01:00
catbref
8af761c1c3 Updated orphaning code of Block and Transaction subclasses 2019-07-23 15:10:43 +01:00
catbref
4b3f877dc0 Interim commit - refactored transaction transformers and fixed unit test compiler errors 2019-07-23 15:06:28 +01:00
catbref
06ba004238 Interim commit - refactored HSQLDBTransactionRepository, and subclasses. Also added approval_height to transactions, renaming height to block_height. 2019-07-23 14:50:56 +01:00
catbref
431e15c7ae interim commit after refactoring TransactionData, and subclasses 2019-07-23 14:49:41 +01:00
catbref
da1bd82c19 Minor integration progress
Remove fetching unconfirmed from Synchronizer

Add extra validity/reference/processable checks to
Transaction.isValidUnconfirmed

Update TransactionUtils to use Transaction.importAsUnconfirmed
for unit tests.
2019-07-23 14:48:49 +01:00
catbref
9bb673ba82 Interim commit of *Transaction refactoring
Transaction subclass isValid methods have last reference checks removed.
(Now handled by Transaction.hasValidReference)

Some checks in subclass' isValid moved to isProcessable, typically
for transaction types that require group-approval.

Removed fee extraction and last-reference update code from
subclass' process() method to Transaction.processReferencesAndFees.

Other changes to transaction/block processing.
2019-07-23 14:46:44 +01:00
catbref
c9968b3dd2 Interim commit of *TransactionData classes for safety 2019-07-23 14:42:02 +01:00
catbref
041773cf41 Synchronizer asks for approval-pending transaction from peer if needed 2019-07-23 12:49:38 +01:00
catbref
3f2453e404 ApiError zh locale (but untranslated) 2019-07-23 12:47:52 +01:00
catbref
414f9679c8 Synchronizer logging improvements 2019-07-23 12:47:20 +01:00
catbref
cd5f9a1e6c Convert ClassLoader.getSystemResource* calls to class.getResource* variant for OSGi, etc. safety 2019-07-23 12:46:27 +01:00
catbref
94fceeb34a Move default blockchain.json into resources 2019-07-23 12:46:05 +01:00
catbref
a4f2cf50b0 Updated node management UI (was previously "bundled ui")
Fix root path redirects for node management UI and API documentation servlets.
2019-07-23 12:44:10 +01:00
catbref
f4022dd243 initial work on adding bundled node-management UI 2019-07-23 12:37:42 +01:00
catbref
99024ee2ef Synchronization improvements
Don't attempt to sync, or generate blocks, if we think we're not up to date.

Notify Controller of newly generated block AFTER releasing blockchain lock.

Loads of changes to synchronizer.

Added missing GET_PEERS handling to Controller.onNetworkMessage.

More detailed peer information (last block sig, last generator sig, last block timestamp, ...)
New HEIGHT_V2 network message to help support above.

More, and improved, logging.

Fix for HSQLDB serialization failure caused by trying to save the same new transaction
received by more than one peer/thread simultaneously.
2019-07-23 12:22:17 +01:00
catbref
23bcba1650 HSQLDBRepository.toOffsetDateTime now returns null if passed null 2019-07-23 12:10:47 +01:00
catbref
58afb53be9 Rename getReferenceGeneratorSignature to getGeneratorSignatureFromReference 2019-07-23 11:46:16 +01:00
catbref
0cb54abb79 Fix bug in checking transaction group-approval threshold 2019-07-23 11:30:07 +01:00
catbref
8b949dc1a1 Added simplistic blockchain validation on startup 2019-07-23 11:29:35 +01:00
catbref
f167d0c831 Add JAXB discriminator to GROUP_INVITE transactions. 2019-07-23 11:25:36 +01:00
catbref
b2f235f7a5 Changes to API regarding proxy forging calls 2019-07-23 11:24:25 +01:00
catbref
f3c588d90f Fix sometimes erroneous Ed25519 public key to X25519 public key conversion.
Added mass (x1000) testing of key conversion and shared secret calculations.

Fix incorrect proxy private key test that has expected result from previous
algorithm.

Added another test HTML/JS file.
2019-07-23 11:22:23 +01:00
catbref
1b4369e5e8 Default settings changes
Use IPv6 wildcard address "::" for P2P bind address as null
seems to make listen socket bind to 127.0.0.1

Increase target minimum outbound connections from 3 to 10.

Increate maximum connection limit from 10 to 30.
2019-07-23 11:17:31 +01:00
catbref
c917191df2 Send network broadcasts over all handshake-completed peers.
Previously we preferred the outbound connection but actually
this leaves our info about inbound connection peers out of date.

Synchronizer still prefers outbound.
2019-07-23 11:17:06 +01:00
catbref
1046f72e7b Slightly better logging for synchronizer 2019-07-23 11:16:02 +01:00
catbref
8f7c954f5a Proxy private keys are now SHA256(shared secret only) instead of SHA256(shared secret + public keys).
HTML/JS in src/test/resources/proxy-key-example.html updated accordingly.

Add handshake status to output of API call GET /peers

Add/correct @ApiErrors annotations on some API calls.

Add API call POST /admin/orphan (target height as body)
to force blockchain orphaning for when node is wildly out of sync.
Added support for above to BlockChain class.

BlockGenerator now requires a minimum number of peers
before it will generate any new blocks.
See "minBlockchainPeers" in settings.

Controller now requires a minimum number of peers
before it will consider synchronizing.
See "minBlockchainPeers" in settings.

Old "minPeers" entry in settings.json no longer valid!

Networking now allows both an outbound and inbound connection
to a peer although will use the outbound connection in preference.

Networking checks peer ID of inbound connections to detect,
and resolve, peer ID clashes/theft.
2019-07-23 11:09:25 +01:00
catbref
40d6190265 API tidy up
GET /addresses/proxykey/{privkey}/{pubkey} now POST /addresses/proxykey

GET /utils/fromBase58 now GET /utils/frombase58
GET /utils/fromBase64 now GET /utils/frombase64

GET /utils/toBase58/{hex} now GET /utils/tobase58/{hex}
GET /utils/toBase64/{hex} now GET /utils/tobase64/{hex}

GET /utils/privateKey/{entropy} now POST /utils/privatekey
GET /utils/publicKey/{privateKey} now POST /utils/publickey
2019-07-23 10:42:23 +01:00
catbref
a3d4cf2900 MAJOR: Don't delete transactions when orphaning - make them unconfirmed again
Lots of edits to Transaction subclasses to change/remove 'delete'.

Corresponding extra changes to help reset some transaction fields to pre-process
state during orphaning.

Changed Block, GenesisBlock & Synchronizer to save transactions where appropriate.

Added enhanced GET_SIGNATURES_V2 network message to reduce the number of
block signatures sent over network.

Peers are now version 2 if they send a new-style build version string,
instead of using first digit from build version.
2019-07-23 10:37:37 +01:00
catbref
07e8e01865 Add check to PROXY_FORGING so that proxy public key always maps to same forger-recipient combo 2019-07-23 10:30:17 +01:00
catbref
d0b45cb696 Proxy forging share to recipient can now be zero 2019-07-23 10:29:53 +01:00
catbref
0259702df2 Fix generating X25519 shared secret.
X25519 shared secrets now match those generated by libsodium.

New tests show that shared secrets are the same using either set
of private+public key combinations.

Changed proxy private key generation from using simple SHA256
of shared secret to using SHA256(shared secret + both public keys).

Added a temporary "BouncyCastle25519" shim class to provide missing
key conversion from Ed25519 to X25519.
2019-07-23 10:29:18 +01:00
catbref
748dddcc32 Add checks to API call POST /admin/forgingaccounts.
Now only accepts private keys for accounts with minting rights
or derives to known proxy forging public key.
2019-07-23 10:15:17 +01:00
catbref
7409c024f6 Include proxy-forged blocks in BlockRepository.countForgedBlocks.
As used during validity check for ENABLE_FORGING transaction.
2019-07-23 10:11:51 +01:00
catbref
ae6b41a893 Improve API call showing block forger summaries to include proxy info 2019-07-23 10:11:21 +01:00
catbref
92e60e2531 Add method to prune old peers from database 2019-07-23 10:10:21 +01:00
catbref
bf8a12d422 Delete old, basic block-explorer 2019-07-23 10:07:55 +01:00
catbref
14686e1e0d Corrected wrong Synchronization enum result 2019-07-23 10:06:16 +01:00
catbref
6a8a0f25c1 Synchronization improvements
Added a finer-grained result (enum) to synchronization to replace
stark boolean result. This allows Controller to decide whether
peer can be retried in the next round (e.g. network issue) or
cooled off for an hour (e.g. peer way too far behind)

Fixed bug with wrong UPDATE_GROUP serialized transaction length.

Added logging to BlockMessage when deserialization to help catch
future bugs.
2019-07-22 18:35:30 +01:00
catbref
1f81784bd6 Improved synchronization logging 2019-07-22 18:27:48 +01:00
catbref
c05fe58676 Add static method to extract generatorSignature from block reference 2019-07-22 18:24:45 +01:00
catbref
ce2dc9b8ad Reduce synchronization attempts from every 10s to every 60s 2019-07-22 18:20:29 +01:00
catbref
1b58d12005 Fix peer data bug where lastConnected was swapped with lastAttempted. 2019-07-22 18:20:04 +01:00
catbref
ad9b42dc5d Install initial peers when creating repository 2019-07-22 18:15:07 +01:00
catbref
6feca1f4d6 Synchronizer improvements & minor fix
Restrict synchronizer to only consider peers at most 60 blocks behind.

Synchronizer will only process a batch of 200 blocks at a time.

Improve debugging message in Peer.
2019-07-22 18:08:06 +01:00
catbref
ecdf8212ea Fix PROXY_FORGING transaction length issue 2019-07-22 18:03:17 +01:00
catbref
af22922025 AutoUpdate: added comment about mandatory use of OpenJDK JRE, not Java SE 2019-07-22 18:01:56 +01:00
catbref
c0ae99161b Auto-Update changes
Checking interval increased from 5 seconds to 5 minutes.
(Reduces repeated fails).

System tray icon should show message just prior to applying downloaded update.
2019-07-22 18:01:29 +01:00
catbref
6be9bf9c3c Add mnemonic phrase support to VanityGen 2019-07-22 18:00:47 +01:00
catbref
7a318c9fc7 Proxy forging improvements + account flags fixes
Proxy forging recipient no longer needs a public key on the blockchain
at the point PROXY_FORGING transaction is submitted.

Proxy forging recipient is given a last-reference, if they don't have one,
when they receive their first block rewards.

Split block fees in proxy forging scenario, using same share proportion.

100% proxy sharing is now allowed.

Fixed account flags processing for accounts in genesis block.
2019-07-22 18:00:12 +01:00
catbref
3ffcf50d7c Added TransactionParticipants index 2019-07-22 17:54:16 +01:00
catbref
29428e9450 Add extra entries to blockchain.json, remove HttpsTest, add Vanity Gen
+ minor whitespace / comment / JAXB mods

Was commit e14381d:
working on genesis block
2019-07-22 17:05:10 +01:00
catbref
57be191814 Splash startup & sys tray 2019-07-18 18:46:18 +01:00
catbref
e5b3166df4 Improve synchronization 2019-07-18 18:40:23 +01:00
catbref
4744232a1c Don't try to sync with a peer that only has genesis block
Was 85bc70b:
new testchain timestamp + don't try to sync with a peer that only has genesis block
2019-07-18 18:36:19 +01:00
catbref
675596ed6e auto update debugging 2019-07-18 18:30:30 +01:00
catbref
52ac881db0 auto update checks download against hash in tx + checks against build timestamp 2019-07-18 18:29:58 +01:00
catbref
a49e3f7a4e auto update 2019-07-18 18:28:56 +01:00
catbref
20aa49a1f1 Another attempt at auto-update 2019-07-18 15:50:05 +01:00
catbref
63a36073ec Work on auto-update + repository path + Settings bugfixes
AutoUpdate needs separate logfiles for its process as
log4j2's rolling appender doesn't support locking.
So we create AU_LOGGER in AutoUpdate and set it up
using static { } block.

Added support for starting/querying/stopping auto-update
Windows service.

Changed Controller's public connectionUrl field to
getRepositoryUrl() public static method so it
can incorporate repositoryPath from Settings.

Controller has added support for auto-update on startup.

Fix bugs with Settings not using userPath properly.

Removed obsolete standalone binary classes:
* blockgenerator
* txhex
* v1feeder
2019-07-18 15:26:41 +01:00
catbref
e4482c5ade Initial auto-update support, API improvements, arbitrary tx improvements
Removed all @Produces from API resources as response content type is
sorted by Swagger.

Added API /admin/info for generic node info.

Added API /arbitrary/ endpoints.

Moved arbitrary data storage from ArbitraryTransaction to ArbitraryRepository.

V4 arbitrary transaction signature is based on data's hash.

Original commit was d02f282, and commit message was:
Initial auto-update support, network MAGIC change, arbitrary tx improvements
2019-07-18 14:53:19 +01:00
catbref
06e6802d97 More work on synchronization
Various fixes to synchronization

Added missing code for processing incoming block summaries in Network.

Fixed block summaries serialization and removed references to BlockData.

Fixed bug in transaction transformation where base transaction length
didn't include reference or fee lengths.

Original commit was ebbab7b
2019-07-18 13:59:49 +01:00
catbref
57b982d2fb Block summaries (repository/data/message/synchronizer) + BlockGenerator
Also refactored some tests.

Original commit was 06fe8fc, with commit message:
Initial implementation of random block generator, etc.
2019-07-18 12:15:11 +01:00
catbref
747f5e41cf Bump Jetty to 9.4.17.v20190418 due to CVE-2019-10247
+ remove older version of package-info-maven-plugin
2019-04-30 09:00:30 +01:00
catbref
882d910631 Remove unneeded HSQLDB debugging and tidy some comments 2019-04-30 08:33:33 +01:00
catbref
a316b8a810 Potential HSQLDB deadlock fix
After opening repository connection with RepositoryManager.getRepostory(),
any 'read' from repository (e.g. SELECT) starts the transaction
even though HSQLDB documentation states there are no shared/read locks
in MVCC concurrency model.

The work-around for this is to 'reset' HSQLDB's in-transaction flag
by performing a ROLLBACK (Repository.discardChanges) immediately
after acquiring the blockchain lock (which is used to ringfence
changes that might collide like these).

Also adding an extra check to prevent payments to nonexistent AT
addresses as it touches Transaction.
2019-04-29 15:18:44 +01:00
catbref
b21ef18533 More HSQLDB deadlock debugging 2019-04-29 12:54:34 +01:00
catbref
db77901026 grab latest block directly instead of via height 2019-04-26 15:01:57 +01:00
catbref
b4d0f9ab68 Allow multiple txType in API GET /transactions/search 2019-04-26 09:23:44 +01:00
catbref
b0b74eb48d Fix JAXB annotations for transactions with creatorPublicKey 2019-04-26 09:04:58 +01:00
catbref
d4ce797241 More HSQLDB debugging 2019-04-26 08:31:10 +01:00
catbref
b7687bf326 More HSQLDB debugging 2019-04-26 08:07:57 +01:00
catbref
0296c4bbb1 HSQLDB debugging & optimization for Transaction.isValidUnconfirmed 2019-04-25 18:29:01 +01:00
catbref
51fd029e22 Access to log entries from API & checks for some transaction-based API calls. 2019-04-24 15:02:37 +01:00
catbref
126e651f27 Change sync consensus to favour lower-value block sigs + other changes
API /addresses/{address} now returns lastReference taking unconfirmed into account.

Added DELETE /peers/known to remove all known peers from repository.

Added blockchain locking around Transaction methods like isValidUnconfirmed
as they (temporarily) update account lastReference.

Ditto getInvalidTransactions, etc.
2019-04-24 12:46:50 +01:00
catbref
d33ffee3ba Work on granting forging rights
Move hard-coded forging tiers to blockchain config.
Tests for granting forging rights.

Added API call to list top block forgers.

Fixed typo with Reward[s]ByHeight class name.
2019-04-19 09:55:04 +01:00
catbref
93230e9704 Add API call to list blocks with given generator. +more tests +pad genesis public key 2019-04-17 18:11:16 +01:00
catbref
d1c547f24a Refactor to use BouncyCastle Ed25519/X25519, and more...
Remove old whispersystems, etc. *25519 and use new v1.61 BouncyCastle.

Fix proxy forging private key derivation from X25519 shared secret.
Also include Javascript test version for comparison.

Fix block rewards for proxy forging.

Add extra useful info to API call GET /admin/forgingaccounts.
Fix API response to POST/DELETE /admin/forgingaccounts when
passed invalid private keys.

Added block rewards and account flags to testchain config.

Tests to cover changes above.
2019-04-17 12:32:03 +01:00
catbref
8e74884536 Fixes after rebase from master 2019-04-12 14:41:59 +01:00
catbref
2f6ef32f35 Replace settings-based generator private keys with DB table and API calls 2019-04-12 13:01:26 +01:00
catbref
c9035edd2c Interim proxy minting commit
Added /addresses/proxying to find proxy forging mappings.

Added /addresses/proxykey/{genprivkey}/{recipientpubkey} to calculate proxy private key.

New Block.regenerate factory method to create new Blocks
 but without having to reprocess ATs, etc.

Added support for proxied generator in Block.calcGeneratorsTarget

BlockGenerator now generates and checks new blocks for various generators,
 including proxy generators.

BlockGenerator now uses generator private keys supplied by Settings.
Corresponding changes to Settings to load base58-encoded private keys.

+ minor stuff
2019-04-12 12:57:21 +01:00
catbref
9b859f3efd Interim minting commit with block rewards (untested)
+ minting long-term simulator
+ Maven pom.xml changes for Eclipse IDE
2019-04-12 12:56:12 +01:00
catbref
3c06d358b7 interim commit with proxy forging repository/transaction support
no block validity/generator support yet
2019-04-12 12:53:44 +01:00
catbref
2dc1720af8 Initial support for account flags + tx (genesis account use only atm) 2019-04-12 12:50:10 +01:00
catbref
85acc4d9df Fix incorrect refunds when cancelling asset orders
Also improved asset tests in that they don't use QORA any more
which takes fees/block rewards out of the picture.

More test assets are issued in the genesis block to
accomplish this:

1m TEST issued to Alice
1m OTHER issued to Bob
1m GOLD issued to Alice
2019-04-12 10:38:25 +01:00
catbref
2f51ced5c0 Fix incorrectly applied price improvement refund.
Also fix broken tests which contributed to this bug slipping by.
2019-04-12 08:44:13 +01:00
catbref
c23f55e6a6 Asset trading: refund saving due to price improvement back to initiator
Added test case to cover the above.

Also improve test harness to properly check balances after orphan back to genesis.
2019-04-11 17:47:12 +01:00
catbref
16dab6972c Fix some asset orders incorrectly matching worse prices. 2019-04-10 15:25:16 +01:00
catbref
cfbf5c12bf Supply extra information fields to various asset-related API calls.
e.g. supply "assetName" in JSON for TRANSFER_ASSET transactions

Also supply have/want asset names, amount asset ID/name, price-pair
and creator address in asset orders.

Show CREATE_ASSET_ORDER amount ID/name & price-pair in correct
format depending on whether transaction was placed before/after
'new' asset pricing took effect. (Orders are always in 'new' form).

Change API call /assets/transfers/{assetid}/{address} to
/assets/transfers/{assetid} with optional "address" query param.
2019-04-10 13:09:30 +01:00
catbref
a5e963911d New asset pricing scheme (take 2)
Orders are back to having "amount" and "price".
(No more "unitPrice" or "wantAmount").

Order "amount" is expressed in terms of asset with highest
assetID.
"price" is expressed in (lowest-assetID)/(highest-assetID).

Given an order with two assets, e.g. QORA (0) and GOLD (31),
"amount" is in GOLD (31), "price" is in QORA/GOLD (0/31).

Order's "fulfilled" is in the same asset as "amount".

Yet more tests and debugging.

For simplicity's sake, the change to HSQLDB repository is
assumed to take place when 'new' pricing switch also
occurs.

Don't forget to change "newAssetPricingTimestamp" in
blockchain config JSON file.
2019-04-10 07:18:50 +01:00
catbref
1b45ee85e7 Fix handling of CORS preflight OPTIONS requests.
Typical log entry would be:

2019-04-05 09:47:08 ERROR WadlGeneratorJAXBGrammarGenerator:401 - Failed to generate the schema for the JAX-B elements
com.sun.xml.internal.bind.v2.runtime.IllegalAnnotationsException: 4 counts of IllegalAnnotationExceptions
        at com.sun.xml.internal.bind.v2.runtime.IllegalAnnotationsException$Builder.check(IllegalAnnotationsException.java:91) ~[?:1.8.0_191]
2019-04-05 10:56:38 +01:00
catbref
7026c35e73 For API "GET /assets" add optional query param "includeData" 2019-04-04 12:33:12 +01:00
catbref
97142fdde8 Return HTTP 400 code with useful message when a ParamException occurs from API call. 2019-04-04 12:30:28 +01:00
catbref
315ebff61d Add optional API request logging via "apiLoggingEnabled" entry in Settings 2019-04-04 11:33:13 +01:00
catbref
ed94c3c5b3 Fix Crypto.isValidAddress to return false when passed empty string 2019-04-04 10:41:04 +01:00