Unified Transaction.importAsUnconfirmed() and Controller.onNetworkTransactionMessage()
to both call Controller.onNewTransaction().
Modified Controller.onNewTransaction() to only send transaction signature to
other peers, instead of full transaction. Peers can request full transaction if they
don't have it.
Controller.onNewTransaction() also calls ChatNotifier, which in turn
notifies websocket handlers about new CHAT transactions.
Added jetty websocket dependency to pom.xml
Changes include:
* Allowing renaming
* Tracking last-updated timestamps
* More stringent Unicode processing
* Way more unit tests
* Max name length reduction to 40 chars
Note: HSQLDB repository table changes
We require AT v1.3.4 now!
Updated AT-related logging.
Added "isInitial" flag to AT state data so that state data created at
deployment is not added to serialized block data.
Updated BTC-QORT AT code and tests to cover various scenarios.
Added missing 'testNtpOffset' to various test versions of 'settings.json'.
Added missing 'ciyamAtSettings' to various test blockchain configs.
Loads of AT-related additions/fixes/etc. to core code, e.g Block
Requires fix in CIYAM AT v1.3.2
New version of Qortal cross-trade AT code.
Change how Qortal addresses are managed in QortalATAPI from using
base58 strings (that are too long) to using hex form (25 bytes)
as they need to fix into 32 byte A/B register.
Generate AT addresses using DeployAtTransaction's signature instead
of convoluted hash of AT data like name, description, etc.
Add startTime as arg to GetTransaction test app.
Add missing fields (name, description, ATType, tags) to DeployAT test app.
Bump CIYAM AT requirement to v1.3
Remove multi-blockchain AT aspect for now (BlockchainAPI).
For PUT_PREVIOUS_BLOCK_HASH_INTO_A we no longer use SHA256 to condense 64-byte block signature into 32 bytes.
Now we put block height into A1 and SHA192 of signature into A2 through A4.
This allows possible future lookup of block data using "block hash", with verification that it is the same block.
Some AT functions use "address in B" but sometimes we populate B with account's public key instead.
So the method "getAccountFromB" is smart and checks for an actual, textual address in B starting with 'Q', otherwise assumes B contains public key.
The Settings field "useBitcoinTestNet" (boolean) now replaced with "bitcoinNet" (String) with possible values MAIN (default), TEST3, REGTEST.
This allows for more varied development/testing scenarios.
Use correct Bitcoin nSequence value 0xFFFFFFFE for P2SH, i.e. enable locktime, disable RBF.
Roll REGTEST checkpoints file generator into main BTC class.
Yet another rewrite of Bitcoin P2SH scripts for BTC-QORT cross-chain trading.
Added associated test classes BuildP2SH, CheckP2SH, DeployAT (unfinished).
Bump bitcoinj to 0.15.5 for fixes.
lockTime is int (seconds since epoch), not long (ms since epoch).
Improve output of Initiate1.
Added (most of) Respond2.
Blockchain configs will need "v2Timestamp" feature trigger renaming to "qortalTimestamp"!
Due to Controller.VERSION_PREFIX changing, peer-to-peer protocol version detection has
been changed. Was previous a substring match, now we test peers's buildTimestamp is at
least Peer.V2_PROTOCOL_TIMESTAMP_THRESHOLD. Changes in Peer.java.
Also added comment and throw() to QortalATAPI.getNextTransactionTimestamp()
as this needs given the change to Qortal's block timestamps from legacy Qora.
Changes to HSQLDB data types, e.g. QoraAddress to QortalAddress, means existing
database will need to be thrown away after this commit!
Seems to produce fat JARs with the same MD5 after repeated runs of:
mvn clean package
Needs testing on other platforms, hence this commit.
Also trimmed some unneeded included JAR libraries from pom.xml,
e.g. netty, async-http-client,
and the *.js.map files from Swagger-UI, which seems to reduce
output JAR size from about 51MB to about 44MB.
Updated pom.xml.
Updated dependencies, including various minor code mods, esp. bitcoin-related.
Starts up and talks to Java 8 nodes!
Dev environment should be at least Eclipse 4.11 with m2e 1.9.1+
Now uses several NTP servers to determine mean offset from
system clock to internet time.
If abs(offset) > 500ms or NTP service not running then
user is 'nagged' via system tray pop-up notification
with instructions on how to fix.
Also improved system tray translations!
Remove old whispersystems, etc. *25519 and use new v1.61 BouncyCastle.
Fix proxy forging private key derivation from X25519 shared secret.
Also include Javascript test version for comparison.
Fix block rewards for proxy forging.
Add extra useful info to API call GET /admin/forgingaccounts.
Fix API response to POST/DELETE /admin/forgingaccounts when
passed invalid private keys.
Added block rewards and account flags to testchain config.
Tests to cover changes above.
GET /assets/trades/recent expanded to allow multiple otherAssetIds
When GET /assets/balances is called with address(es) but no assetIDs
then it will return balances for all assets, including zero balances
for assets the addresses don't own.
GET /addresses/{address} no long fakes a default groupID - in fact
defaultGroupId now removed from blockchain config.
Some Eclipse IDE files now hidden/removed from git repo.
GenesisBlock (v4) now supports various transaction types (issue-asset, etc.)
with generated signatures (like genesis transaction signature) and
missing references inserted.
JUnit reverted back to v4 for Eclipse support (for now).
Some pom.xml changes to reduce maven-shade-plugin conflicting classes warnings.
Repository now supports SAVEPOINT and ROLLBACK TO SAVEPOINT.
HSQLDB concurrency/transaction model changed from LOCKS to MVCC to help with
transaction deadlocks/rollbacks.
More XXXs and TODOs added to Block.java for investigation/fix/improvements.
Also used new repository SAVEPOINT feature when validating transactions
instead of rolling back entire transaction. This fixes problem during
synchronization where the rollback would undo previously synchronized,
but not yet committed, blocks!
Transactions orphaned by Block.orphan ARE now added to unconfirmed pile,
unlike before.
Concurrent lock now prevents simultaneous block generation and synchronization,
including synchronization via multiple peers.
Lots of new networking code: peer lists, block signatures, blocks,
blockchain synchronization.
PEERS_V2 message now supports hostnames, IPv6 and port numbers.
Fixed bug with block serialization for transport over network.
DB shape change from v29 to add peer info.
New NetworkRepository to handle above.
Peer handshaking with v2 anti-DoS PoW code.
Handshaking refactored into a state-machine-like enum.
Some peer-related API calls added.
Peers exchange pings, heights, peers.
No actual peer sync yet.
Other changes:
Peer version info taken from Maven build properties/resource file.
AnnotationPostProcessor more resilient when fetching PathItems.
Per-repository session debugging flag that can be toggled at will.
HSQLDBRepository.delete() now returns int so callers can detect
whether anything was actually deleted.
Some renaming to settings.
API calls that return lists now take limit, offset and reverse params.
API calls that used to return data & optional list (e.g. blockWithTransactions)
now only return base data. The optional lists can be fetched via
a different API call.
Also: SLF4J now routes logging to log4j2 so start up output cleaned up.
Suppressed extraneous Jersey warning about Providers during start-up injection.
Also added test for missing creator public key on API-submitted
transaction creation calls, like /payments/pay or /asset/issue.
(Needs to be an OpenAPI validator added at some point).
Refactored to standard Maven layout:
src/main/java
src/main/resources
src/test/java
etc.
New translation code that uses locale-specific ResourceBundles
to load translations on demand.
Reworked API error/exceptions code to a shorter, simpler
@ApiErrors annotation. Processing of @ApiErrors annotations
produces an example for each possible API error and includes
API error string in HTTP response code, e.g.
400 INVALID_SIGNATURE
Missing API error cases added to each API call.
Translation of openAPI.json removed (for now).
block-explorer.html and BIP39 wordlists now read as resources
instead of direct from disk.
Java compile warnings fixed.
Some runtime warnings remain:
WARNING: A provider api.resource.ApiDefinition registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime.
WARNING: A provider api.resource.AnnotationPostProcessor registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime.
WARN org.reflections.Reflections - given scan urls are empty. set urls in the configuration
XmlJavaTypeAdapter api.Base58TypeAdapter converts byte[] to Base58.
This XmlAdapter is applied at package-level to all packages inside data and api.models.
So no need to annotate every byte[] property!
Added package-info-maven-plugin to pom.xml to do this.
block-explorer.html fixed to show/use base58 again
Some data objects (e.g. TransactionData) have added XmlElements that
convert public keys to addresses, for convenience.
Several API calls updated to return specifically text/plain instead
of ambiguous application/json and/or text/plain. (Typically
API calls that return a single value, e.g. an integer, like /blocks/height).
Rejigged pom.xml, extracting common dependency versions as properties.
Removed extraneous HSQLDB dependency (v2.4.1) as we're using svn r5836 for now.
Removed calls to Security.checkApiCallAllowed() for all API calls EXCEPT /admin/stop.
Throws error if remote IP is not localhost.
Added 'global' OpenAPI parameters to fake /admin/dud endpoint to save copy&pasting.
This will need more tidying in the future, or at least future support from swagger-core.
Code added in AnnotationPostProcessor to insert global parameters in top-level
OpenAPI components section.
/block-explorer.html hidden from API UI
BlocksResource now expects Base64 block signatures instead of Base58.
Endpoints that return block data also accept optional "includeTransactions"
query param which does exactly that.
BlockWithTransactions API model added for above.
Some attempt to get transaction-specific data returned but no luck as yet.
(TransactionData, GenesisTransactionData, PaymentTransactionData touched).
See https://github.com/swagger-api/swagger-core/issues/3046
TransactionsResource now has support for optional query params "limit" and "offset"
so that only a subset of large results can be requested.
UtilsResource added to provide convenient Base64<->Base58 conversions.
/admin/uptime fixed to return uptime from application launch instead of
instantiation of AdminResource class!
Controller improved to detect repository and API startup failures.
HSQLDBRepositoryFactory now detects when it can't open database and throws.
(Before it would simply hang).
Removed extraneous import from qora.account.Account
Switched from maven-assembly-plugin to maven-shade-plugin for
building FATJAR.
When running from FATJAR, class-path is ". .." to help find
log4j2.properties file.
Swagger-UI can now be served direct from inside FATJAR instead
of requiring resources in filesystem.
Default package Start now controller/Controller
block-explorer.html now served via Jetty and modified to use
relative URLs instead of absolute http://localhost:9085/... style
Improved shutdown code in controller
/admin/stop API call disabled for now
Highly permissive settings.json added
Added FATJAR packaging support to pom.xml
Added some "summary" fields to API calls but more need doing.
Corrected path clash from having unnecessary @OpenAPIDefinition annotations.
Added API "tags" to group similar calls (address-based, block-related, etc.)
Fixed addresses/lastreference/{address}
Implemented addresses/lastreference/{address}/unconfirmed
Implemented addresses/assets/{address}
Added /admin/stop and /admin/uptime API calls.
Moved general API info into new src/api/ApiDefinition.java
Added CORS support to ApiService
Added /transactions/address/{address} and /transactions/block/{signature}
Replaced references to test.Common.* to do with repository factory.
This fixes issues with building FATJAR due to references to test classes
that are omitted from FATJAR.
Changes to AccountBalanceData, BlockData and TransactionData
to support JAX-RS rendering to JSON.
Added getUnconfirmedLastReference() to Account.
Added getAllBalances(address) to account repository
- returns all asset balances for that address.
Added getAllSignaturesInvolvingAddress(address) to account repository
but currently only uses TransactionRecipients HSQLDB table.
(And even that wasn't automatically populated).
Included: very basic block explorer to be opened in browser as a file:
block-explorer.html
CHANGED: added dependency to javax.mail for because of strange "java.lang.NoClassDefFoundError: javax/mail/internet/MimeMultipart" exception when serializing data objects in API resources.
NB: we're still using HSQLDB svn r5836
Updated README.md
Added log4j2.properties file for logging!
Imported CIYAM-AT jar into project-local Maven repo
CIYAM-AT related:
ATData
ATStateData
ATTransactionData
DeployATTransactionData
AT
DeployATTransaction
ATRepository
HSQLDBATRepository
HSQLDBDeployATTransactionRepository
ATTests
DeployATTransactionTransformer
Fixed Block so correct block hash and timestamps are generated,
especially when previous/next block versions differ.
Added extra call in BlockTransformer to aid this.
Fixed GenesisTransaction.isValid's incorrect amount test.
Fixed comments in TransferAssetTransaction and incorrect use of BlockChain.getVotingReleaseTimestamp()
instead of BlockChain.getAssetsReleaseTimestamp().
Added new TYPEs to HSQLDBDatabaseUpdates, and set LOB granularity to 1KB for AT use.
Added AT_address column to DeployATTransactions in HSQLDB.
Added ATs, ATStates and ATTransactions tables.
(You will need to discard existing database and rebuild).
Fixed incorrect byte array output in IssueAssetTransactionTransformer,
where Asset "references" were not processed correctly.
Added support for BigDecimal serialization to a byte-array size other than the standard 8.
HSQLDB v2.4.0 had some issue with non-padded, case-insensitive string comparisons.
This is fixed in svn r5836-ish of HSQLDB but yet to be pushed out to new HSQLDB release.
So this commit includes hsqldb-r5836.jar and modified pom.xml/.classpath for now.
No need for duplicate, hidden creatorPublicKey in CancelOrderTransactionData,
CreateOrderTransactionData and CreatePollTransactionData.
Various changes to use more try-with-resources, especially with JDBC objects like
Connection, Statement, PreparedStatement, ResultSet.
Added loads of missing @Override annotations.
Fixed bug in Asset exchange order matching where the matching logic loop
would incorrectly adjust temporary amount fulfilled
by the "want" asset amount (in matchedAmount)
instead of the "have" asset amount (in tradePrice).
Disabled check for duplicate asset name in IssueAssetTransactions for old v1 transactions.
In HSQLDB repository we now use ResultSet.getTimestamp(index, UTC-calendar) to make sure we
only store/fetch UTC timestamps. The UTC-calendar is made using static final TimeZone called
HSQLDBRepository.UTC.
To keep asset IDs in line with v1, Assets.asset_id values are generated on-the-fly in HSQLDB
using a "before insert" trigger on Assets table. Corresponding code
calling HSQLDBRepository.callIdentity() replaced with SELECT statement instead.
Moved most of the HSQLDB connection properties from the connection URL to explicit code in
HSQLDBRepositoryFactory.
Fixed incorrect 'amount' lengths in PaymentTransformer, as used by MultiPayment and Arbitrary
transaction types.
Added support for mangled arbitrary transaction bytes when generating/verifying a v1 transaction signature.
In v1 Arbitrary transactions, bytes-for-signing are lost prior to final payment (but only if there are any payments).
Added corresponding code for multi-payment transactions in the same vein.
Added Apache commons-net as maven dependency for NTP support.
Added SAVEPOINT and ROLLBACK TO SAVEPOINT support to DB class.
Added exists() test to DB class.
Add MessageTransactions, with V1/V3 code in one class instead of very similar code
split across two classes. Update DB schema to add version.
More fleshing out of Assets class.
Fleshing out Block class with parse(), generating balance and signature-related methods.
More javadoc. More tests.
Most SQL tables defined but only payment transactions actually implemented.
Maven support added.
Some code imported from 'old' Qora:
RIPEMD160 renamed as BrokenMD160 and deprecated.
whispersystem's Ed25519 implementation (to be replaced with bouncycastle).
Basic Account/PublicKeyAccount/PrivateKeyAccount code.
Some utils like Base58 and Pair.
To use:
Use maven to fetch dependencies.
Build project.
Fire up an old-gen Qora node.
Run src/test/update.java as a JUnit test to build DB structure.
Run src/test/migrate.java as a JUnit test to migrate old Qora blocks to DB.
You should now be able to run src/test/load.java and src/test/save.java
as JUnit tests demonstrating loading/saving Transactions from/to database.
This commit done while halfway through adding Block support!
