Commit Graph

18 Commits

Author SHA1 Message Date
catbref
a49e3f7a4e auto update 2019-07-18 18:28:56 +01:00
catbref
20aa49a1f1 Another attempt at auto-update 2019-07-18 15:50:05 +01:00
catbref
63a36073ec Work on auto-update + repository path + Settings bugfixes
AutoUpdate needs separate logfiles for its process as
log4j2's rolling appender doesn't support locking.
So we create AU_LOGGER in AutoUpdate and set it up
using static { } block.

Added support for starting/querying/stopping auto-update
Windows service.

Changed Controller's public connectionUrl field to
getRepositoryUrl() public static method so it
can incorporate repositoryPath from Settings.

Controller has added support for auto-update on startup.

Fix bugs with Settings not using userPath properly.

Removed obsolete standalone binary classes:
* blockgenerator
* txhex
* v1feeder
2019-07-18 15:26:41 +01:00
catbref
06e6802d97 More work on synchronization
Various fixes to synchronization

Added missing code for processing incoming block summaries in Network.

Fixed block summaries serialization and removed references to BlockData.

Fixed bug in transaction transformation where base transaction length
didn't include reference or fee lengths.

Original commit was ebbab7b
2019-07-18 13:59:49 +01:00
catbref
57b982d2fb Block summaries (repository/data/message/synchronizer) + BlockGenerator
Also refactored some tests.

Original commit was 06fe8fc, with commit message:
Initial implementation of random block generator, etc.
2019-07-18 12:15:11 +01:00
catbref
126e651f27 Change sync consensus to favour lower-value block sigs + other changes
API /addresses/{address} now returns lastReference taking unconfirmed into account.

Added DELETE /peers/known to remove all known peers from repository.

Added blockchain locking around Transaction methods like isValidUnconfirmed
as they (temporarily) update account lastReference.

Ditto getInvalidTransactions, etc.
2019-04-24 12:46:50 +01:00
catbref
8e74884536 Fixes after rebase from master 2019-04-12 14:41:59 +01:00
catbref
c9035edd2c Interim proxy minting commit
Added /addresses/proxying to find proxy forging mappings.

Added /addresses/proxykey/{genprivkey}/{recipientpubkey} to calculate proxy private key.

New Block.regenerate factory method to create new Blocks
 but without having to reprocess ATs, etc.

Added support for proxied generator in Block.calcGeneratorsTarget

BlockGenerator now generates and checks new blocks for various generators,
 including proxy generators.

BlockGenerator now uses generator private keys supplied by Settings.
Corresponding changes to Settings to load base58-encoded private keys.

+ minor stuff
2019-04-12 12:57:21 +01:00
catbref
eb038b8f31 Controller debugging, including logging build version/timestamp 2019-03-25 11:22:57 +00:00
catbref
43eec116b5 Reorder handshake to increase compat with v1 nodes + request remote peers 2019-03-06 10:40:39 +00:00
catbref
00656f6724 Interim safety commit due to large number of changes!
log4j2.properties now has debugging entries removed.
log4j2-test.properties (not in repo) takes priority
 so using that in development instead.

Unconfirmed transactions no longer wiped on start-up
 by default - see Settings

Reworking of {Public,Private,Genesis}Accounts as it seemed
 possible to silently lose public key in repository.
The use of AccountData didn't work and so field-specific
 repository calls have been made instead
 (e.g. setLastReference) that try to opportunistically
 store public key too, if available (i.e. caller is
 PublicKeyAccount subclass, or better).

Added API call GET /addresses/{address} to return
 general account info in one go. (Essentially the
 AccountData object as fetched from repository).

Initial work on adding default groupID to accounts,
 along with corresponding SET_GROUP transaction type.
In additional, added blockchain-wide default groupID
 and flag to allow/disallow no-group/groupless
 transactions.

Initial work on group-admin approval of transactions
 tied to a specific group via txGroupId.

More work needed on transaction's "effective txGroupId"!

API call /transactions/pending to list transactions
 pending group-admin approval. However, this needs more
 work (see effective txGroupId above) and potentially
 offloading to HSQLDB repository if possible.

Minor CIYAM AT renames to help static reflection initializers.

Block.orphan() no longer adds orphaned transactions back to
 unconfirmed pile as they are themselves deleted during
 Transaction.orphan(). Maybe the answer is to NOT delete
 them during Transaction.orphan() but to add them to
 unconfirmed pile at that point? Very old transactions
 leftover from major resync would simply expire, whereas
 recently transactions leftover from minor resync could
 still make it into a new block on synced chain fork.

Changes/tidying/improvements to block generator regarding
 removing invalid transactions and dealing with transactions
 pending group approval.

Approval threshold added to groups.

Mass refactoring of transaction-related classes to unify
 constructors, particularly field ordering, to fall in line
 with raw transaction layout.
e.g. constructors now reflect that raw transactions mostly
 start with type, timestamp, txGroupId, publicKey, reference
e.g. JAXB afterUnmarshal methods added where needed and corresponding
 nasty code in Transaction subclass constructors ripped out.
e.g. TransactionTransformer subclasses contain less duplicated code.

Fixed bug with repository save points thanks to swapping to Deque.

Some fixes to do with missing transaction types being passed to JAXB
 TransactionData subclass constructors.

Ripped out obsolete toJSON in TransactionTransformers as this
 is all nicely taken care of by Swagger/OpenAPI (thanks @Kc)
2019-02-18 19:00:37 +00:00
catbref
82e9e1e7dc Improved peer addresses.
Added short sleep() to GET /admin/stop to allow time for HTTP response body to be sent back.

Improved documentation for /peers API resources. Added examples, tidied API output models.
Fixed issue where IPv6 literals with port couldn't be parsed. Now uses RFC5952/RFC3986 style
literal IPv6 addresses with ports, e.g. [::1]:9084

Fixed NPE in Controller.potentiallySynchronize() where peer might not have sent height yet.

Improved Handshake to discard inbound connections if we already have an outbound connection
to a peer with that ID. This prevents us from having two connections to the same peer, one
in each direction.

Network.mergePeers() now runs in a separate thread as acquiring the lock might block.

Network.creationConnection() exits fast based on number of outbound connections, instead of
number of total connections.

Network no longer sends 'local' peer addresses to non-local peers.
e.g. it won't send localhost:9084 to node4.qora.org:9084

Added try-catch to Network.broadcast for when we try to broadcast while shutting down.

Added PeerAddress class to deal with the whole hostname/IPv4/IPv6 address situation.

Reworked PEERS_V2 message type to only send sized-strings instead of separate port,
and potentially IPv6 byte arrays.

Change to HSQLDB database shape.
Corresponding changes to HSQLDBNetworkRepository.
2019-02-12 18:50:49 +00:00
catbref
7a53ac17a6 Synchronization, peer management + fixes
Peers now broadcast height after successful synchronization.

Added support for sending unconfirmed transactions to other peers.
This is done on connect and also after a new unconfirmed transaction is submitted via API.

Fixed synchronizer to handle blocks with transactions correctly.

Fixed network-related PoW to not use class-global SHA256 message digester!
(It was being corrupted by simulataneous access by different threads - whoops)

Surrounded Network.mergePeers with a lock to prevent HSQLDB deadlocks.
Also changed HSQLDB concurrency model to MVCC (only takes effect if DB rebuilt).

Added support for logging other HSQLDB sessions in the event of exception.
(Currently only used by HSQLDBSaver)

Transaction transformer modifications to help deserialize TransactionMessages.
2019-02-11 17:37:52 +00:00
catbref
7f4511cb7b Networking and repository
Some pom.xml changes to reduce maven-shade-plugin conflicting classes warnings.

Repository now supports SAVEPOINT and ROLLBACK TO SAVEPOINT.
HSQLDB concurrency/transaction model changed from LOCKS to MVCC to help with
 transaction deadlocks/rollbacks.

More XXXs and TODOs added to Block.java for investigation/fix/improvements.
Also used new repository SAVEPOINT feature when validating transactions
 instead of rolling back entire transaction. This fixes problem during
 synchronization where the rollback would undo previously synchronized,
 but not yet committed, blocks!

Transactions orphaned by Block.orphan ARE now added to unconfirmed pile,
 unlike before.

Concurrent lock now prevents simultaneous block generation and synchronization,
 including synchronization via multiple peers.

Lots of new networking code: peer lists, block signatures, blocks,
 blockchain synchronization.

PEERS_V2 message now supports hostnames, IPv6 and port numbers.

Fixed bug with block serialization for transport over network.
2019-02-01 14:03:06 +00:00
catbref
0db43451d4 Interim networking code commit in case of dev catastrophe!
DB shape change from v29 to add peer info.
New NetworkRepository to handle above.

Peer handshaking with v2 anti-DoS PoW code.
Handshaking refactored into a state-machine-like enum.
Some peer-related API calls added.

Peers exchange pings, heights, peers.

No actual peer sync yet.

Other changes:

Peer version info taken from Maven build properties/resource file.
AnnotationPostProcessor more resilient when fetching PathItems.
Per-repository session debugging flag that can be toggled at will.
HSQLDBRepository.delete() now returns int so callers can detect
 whether anything was actually deleted.
Some renaming to settings.
2019-01-30 18:24:10 +00:00
catbref
4be58514c0 Unify API calls that return lists + offload pagination to repository
API calls that return lists now take limit, offset and reverse params.

API calls that used to return data & optional list (e.g. blockWithTransactions)
now only return base data. The optional lists can be fetched via
a different API call.

Also: SLF4J now routes logging to log4j2 so start up output cleaned up.
Suppressed extraneous Jersey warning about Providers during start-up injection.
2019-01-24 16:42:55 +00:00
catbref
7998166c0a API, switchable MD160,
Now uses working RIPE-MD160 by default but can be switched to broken MD160 using flag in blockchain config,
e.g. for Qora v1 blockchain.

Replaced API signature/reference examples with descriptive text as they weren't very useful.
Replaced API address examples with ones generated using working MD160.

Added GET /transactions/signature/{signature}/raw that returns raw transaction in base58 encoding.

Added "ignoreValidityChecks" query param to POST /transactions/decode to bypass INVALID_REFERENCE errors
if supplying an old/speculative transaction that can't be added to unconfirmed transaction pile.

Finally fixed creating inital assets in BlockChain.

Controller now inserts BouncyCastle as highest priority Security Provider.

TransactionData & transaction repository now tries to return transaction's block height in data when possible.
2019-01-07 13:12:42 +00:00
catbref
5c6e239d76 initial work towards OSGi
refactored packages so they all start with org.qora

added some attempt at OSGi mega bundle using Maven (doesn't work)
2019-01-04 10:19:33 +00:00