Compare commits

...

58 Commits

Author SHA1 Message Date
catbref
360f6cd4f1 Bump to v1.0.7 2020-04-24 09:44:35 +01:00
catbref
f1e4528581 Fix Transaction.calcRecommendedFee() 2020-04-24 09:40:43 +01:00
catbref
1375372380 Added tests to cover validity checks on group min/max block delay values 2020-04-23 17:06:23 +01:00
catbref
a7d0ad27b1 Legacy QORA block reward fix
If there's no more unrewarded legacy QORA held,
then quickly return from Block.distributeBlockRewardToQoraHolders()
instead of causing divide-by-zero.
2020-04-23 17:00:08 +01:00
catbref
5011a2be22 Added isAdmin field to output of API call GET /groups/member/{address} 2020-04-22 16:32:55 +01:00
catbref
33010f82d8 No need to check AT transactions in Block.areTransactionsValid() 2020-04-16 13:16:30 +01:00
catbref
8dbd8c4e65 Performance improvement when checking block's online accounts signatures.
If the timestamp-pubkey-sig is still 'current' then it'll be in
Controller's list of current online accounts, so we can quickly scan
that list before falling back to the more expensive Ed25519 verify.

Added equals() and hashCode() to OnlineAccountData to support above.
2020-04-16 12:25:01 +01:00
catbref
c2a3c1271c Remove missed, extraneous last-reference check from CreateAssetOrderTransaction.isValid() 2020-04-16 09:24:04 +01:00
catbref
1e9a7ac87d Update SysTray for all synchronizing scenarios.
When synchronizing is forced via API call, the SysTray doesn't update
to reflect this.

We fix this by moving the SysTray updating code from
Controller.potentiallySynchronize() to the inner method
Controller.actuallySynchronize(), which is also the method called
directly by the API.
2020-04-15 17:36:12 +01:00
catbref
e25d24964c Make BlockMinter more aggressive about obtaining blockchain lock.
Previously BlockMinter & Synchronizer would both try opportunistic
locking, with no wait/timeout or fairness.

This could lead to a situation where a majority of nodes are
synchronizing, albeit only the top 1 or 2 blocks, but no node
manages to mint within the 'recent' period, so the chain stalls.

However, if a node is at/near the top of the chain then synchronization
shouldn't take very long so we let BlockMinter wait until to 30s
(approx. half typical block time) to obtain lock.

This makes minting blocks more likely in a BlockMinter/Sync fight
which helps keep the chain going.

Detecting chain stalls, and allowing minting if we have plenty of peers,
also produces blockchain 'islands' so isn't a simple fix at this point.
2020-04-15 17:30:49 +01:00
catbref
d90d84ab06 More descriptive tray mouseover, showing sync percent or connecting status
Added sync percent to API call GET /admin/status

Added SysTray i18n "CONNECTING" key to CheckTranslations test app.
2020-04-03 08:31:41 +01:00
catbref
2ddb1fa23e Bump version to v1.0.6 2020-03-31 08:04:35 +01:00
catbref
82f6e38adb Add PID tracking to stop.sh (thanks IceBurst) 2020-03-30 17:39:36 +01:00
catbref
00ac26cf27 Add script to build release ZIP 2020-03-30 17:39:36 +01:00
catbref
fa1aa1c8b2 Show informative page instead of "Forbidden" when user tries to access API documentation when disabled. 2020-03-30 17:39:36 +01:00
catbref
9156325ffc Reduce minimum networking requires to relax CPU usage somewhat 2020-03-30 17:39:36 +01:00
catbref
70131914b2 Auto-updates: increase checking interval & TCP timeouts
Bumped TCP timeouts for fetching auto-update from 5s (connect) and
3s (read) to 30s (connect) and 10s (read) to allow for nodes with
slower internet connections.

Increased interval between checking for auto-updates from 5 minutes
to 20 minutes to reduce load on update sources and also to reduce
the number of nodes that restart at any one time.

Obviously this new checking interval will only apply after the NEXT
auto-update...
2020-03-30 17:39:36 +01:00
catbref
bd87e6cc1a Increase retry interval and count in ApplyUpdate.
Used when checking that node has shutdown and when replacing old JAR with new update.

ApplyUpdate previously waited 5 seconds between checks/retries, for up to 5 times: 25 seconds.
Now waits 10 seconds, for up to 12 times: 120 seconds.
Hopefully this will give slower nodes enough time to shut down and prevent errors like these on Windows installs:

2020-03-24 12:05:50 INFO  ApplyUpdate:114 - Unable to replace JAR: qortal.jar: The process cannot access the file because it is being used by another process.
2020-03-30 17:39:36 +01:00
catbref
6c8e96daae Turn off repository backups by default.
They can be re-enabled by setting "repositoryBackupInterval" to a
non-zero value in settings.json. Note the value is in milliseconds!
2020-03-30 17:39:36 +01:00
catbref
cfb8f53849 Reduce DB space taken up by Blocks 2020-03-30 17:39:36 +01:00
catbref
7bb2f841ad Rip out historic account balances as they take up too much DB space. 2020-03-30 17:39:36 +01:00
catbref
558263521c Minor fix to auto-update tx publish script 2020-03-30 17:39:36 +01:00
catbref
1db8c06291 Merge pull request #2 from IceBurst/patch-1
Java Checks
2020-03-30 11:32:04 +01:00
catbref
edee08a7b5 Merge pull request #1 from nitrokrypt/master
Fixed Images
2020-03-30 11:25:34 +01:00
catbref
0594bdf1c7 Bump to v1.0.5 in pom.xml. Filter out build/test artifacts from shaded JAR. 2020-03-24 11:17:43 +00:00
catbref
72c299a331 Add SysTray notification for DB backup. More translations.
Added a setting "showBackupNotification", which is false by default,
that shows a tray notification when a repository backup occurs.

Above notification, and the auto-update notification, now refer to
the SysTray i18n translation lookup resources.
2020-03-24 09:26:40 +00:00
catbref
0b42a7ad63 Modify minOutboundPeers, maxPeers and maxNetworkThreadPoolSize default settings
Typical users don't need quite so many connections, so minOutboundPeers and
maxPeers reduced accordingly.

maxNetworkThreadPoolSize increased from 10 to 20.
2020-03-24 09:24:22 +00:00
catbref
51e59f6ab7 Change HSQLDB repository log fsync() interval from 500ms to 5s 2020-03-24 09:23:17 +00:00
catbref
38394de661 Reduce peer response timeout from 5s to 2s
5s is way too long, and even 2s might still be considered excessive.
However, reducing the timeout might also reduce the number of
network engine "spawn failures" due to too many threads tied up
waiting for ping responses from overloaded peers.

Does not affect peer handshaking: that has a separate timeout.
2020-03-24 09:20:50 +00:00
catbref
22f9755f4f Performance optimizations. Accounts/NTP/System.currentTimeMillis, etc.
Added Ed25519 private key to public key function accessible from SQL.
Added Ed25519 public key to Qortal address function accessible from SQL.

Used above functions to store minting account public key in SQL to
reduce the number of unnecessarily repeated Ed25519 conversions.

Used above functions to store reward-share minting's accounts address
to reduce the number of unneccessarily repeated PK-to-address conversions.

Reduced the usage of PublicKeyAccount to simply Account where possible,
to reduce the number of Ed25519 conversions.

Account.canMint(), Account.canRewardShare() and Account.getEffectiveMintingLevel()
now only perform 1 repository fetch instead of potentially 2 or more.

Cleaned up NTP main thread to reduce CPU load.
A fixed offset can be applied to NTP.getTime() responses, for both
scenarios when NTP is running or not. Useful for testing or simulating
distant remote peers.

Controller.onNetworkMessage() and Network.onMessage() have both had their
complexity simplified by extracting per-case code to separate methods.

Network's EPC engine's thread pool size no longer hard-coded, but comes
from Settings.maxNetworkThreadPoolSize, which is still 10 by default,
but can be increased for high-availability nodes.

Network's EPC task-producing code streamlined to reduce CPU load.

Generally reduced calls to System.currentTimeMillis(), especially
where the value would only be used in verbose logging situations,
and especially in high-call-volume methods, like within repository.
2020-03-23 11:14:05 +00:00
catbref
4cb2e113cb Log (and discard) duplicate outbound connections to the same peer 2020-03-23 11:07:08 +00:00
catbref
e0f024ef5c Performance improvements in networking ExecuteProduceConsume engine
Keep track of when EPC engine can't spawn a new thread as this
might indicate thread-pool exhaustion and cause some network
messages to be lost.

If logging level is NOT 'trace' (or 'all') then don't call
System.currentTimeMillis() as we'll never use the value.

Similarly, don't set thread names if not logging at 'trace' either.

Update EPC tests, particularly unified per-second/end-of-test stats
reporting.
2020-03-23 11:00:19 +00:00
catbref
f95cb99cdc Add support for logging PROOF network message calculation time 2020-03-23 10:57:23 +00:00
catbref
1f0170bb4b Increase timeout for transaction submission via API POST /transactions/process from 500ms to 30s 2020-03-23 10:54:35 +00:00
catbref
5eafdf3c80 Bump version to 1.0.4 2020-03-19 14:30:02 +00:00
catbref
d7c26c27e1 Add debugging message to Peer regarding lost PING replies 2020-03-19 14:29:47 +00:00
catbref
2d18dd62eb Translation / API response improvements 2020-03-19 14:21:48 +00:00
catbref
51fd177d79 Add access to network engine stats
Added API call GET /peers/enginestats to allow external monitoring.

Extract all engine stats in one synchronized block, instead of
separate calls, for better consistency.
2020-03-19 11:19:49 +00:00
catbref
c4643538f1 Improve Synchronizer to reduce network load
Synchronizer now bails out early when trying to find common block with
a peer. There's no need to keep searching if common block is too far
behind that a TOO_DIVERGENT result would be returned.

fetchSummariesFromCommonBlock() reworked to return a useful
SynchronizationResult directly instead of caller trying to infer
what happened based on null/empty returned list!
2020-03-19 11:11:35 +00:00
catbref
0edadaf901 Remove warning 2020-03-19 11:11:05 +00:00
catbref
c05533fb71 Fix comment 2020-03-19 11:10:40 +00:00
catbref
db270f559f Improve minting/syncing status reporting
Added API call GET /admin/status which reports whether minting
is possible (have minting keys & up-to-date) and whether node is
currently attempting to sync.

Corresponding change to system tray mouseover text.

Corresponding text added to SysTray transaction resources.
2020-03-19 11:07:56 +00:00
catbref
79f7f68b0c Change when BlockMinter decides it's ok to mint a block
Previously BlockMinter would attempt to mint if there were at least
'minBlockchainPeers' connected peers and none of them had an
up-to-date block and we did. This was maybe useful for minting block 2
but possibly causes minting chain islands where a badly connected
node mints by itself, even though connected to not up-to-date peers.

Now BlockMinter requires 'minBlockchainPeers' up-to-date peers, not
simply just connected. This should let synchronization bring the
node up-to-date but does require the node to have better peers.

Currently, the default for minBlockchainPeers is 10. So a node
requires 10 up-to-date peers before it will consider minting. It
might be possible to reduce this in the future to lessen network load.
2020-03-19 10:58:53 +00:00
catbref
d30d61edab Reworking/speed-ups for block rewards & general account DB manipulation
**NOTE** currently under wider test - maybe not be final version!
2020-03-18 18:04:45 +00:00
catbref
f7e2ee383e More complete testing of block reward distribution to founders 2020-03-18 18:03:56 +00:00
catbref
544fdbfbe9 Add database-level CHECK constraint on account balances 2020-03-18 18:03:13 +00:00
catbref
c3d1ecb7e1 Reduce maximum allowed distance back to common block 2020-03-18 18:02:00 +00:00
catbref
873a9d0cee Add support for testing with multiple online accounts 2020-03-18 18:00:46 +00:00
catbref
95cb5f607b Use HSQLDB v2.5.0 but with fix for INSERT...ON DUPLICATE KEY UPDATE bug 2020-03-18 17:58:16 +00:00
catbref
54d0b721c4 Dynamically allocate/deallocate Peer byteBuffer to reduce memory load at the expense of extra GC 2020-03-16 17:50:49 +00:00
catbref
4a4678b331 Immediately close socketChannels after accepting peers we won't use 2020-03-16 16:07:17 +00:00
catbref
12f9ecaaca Faster Synchronizer shutdown by checking Controller.isStopping() 2020-03-16 16:05:27 +00:00
catbref
1d3ee77fb8 Increase default number of peers required before a node can mint/sync, and other settings 2020-03-15 14:14:58 +00:00
IceBurst
83bce3ce52 Java Checks
Validates Java is available and version is 11 or greater
2020-03-14 10:52:54 -04:00
nitrokrypt
bf288dbfc2 Fixed Images
Previous images had a small hole in the icon (probably a result of a background removal), I just filled it back in with white like it's supposed to be. Also, the previous square icons were streched into a square aspect ratio, these are unstreched.
2020-03-12 12:01:24 -05:00
catbref
64055e280d Shutdown controller, and hence entire node, if networking or API fail to start. 2020-03-11 15:46:59 +00:00
catbref
90e0f9dddc Fix system-dependent path separator usage.
Although HSQLDB is happy being given unix-style path separator '/'
and converting as necessary on other platforms (e.g. Windows),
manipulation of repository pathnames in Java, outside of HSQLDB,
needs to use platform-specific path separators.

Thus, changes made to replace '/' with File.separator where
necessary.

This should fix repository rebuild errors, which then lead to odd
start-up errors like:

2020-03-11 13:55:19 INFO  Controller:270 - Starting repository
2020-03-11 13:55:20 INFO  Controller:287 - Validating blockchain
2020-03-11 13:55:20 INFO  HSQLDBRepository:227 - Rebuilding repository from scratch
2020-03-11 13:55:20 INFO  GenesisBlock:296 - Using genesis block timestamp of 1583870000000
2020-03-11 13:55:21 WARN  HSQLDBRepository:720 - Uncommitted changes (882) after connection close, session [3]
java.lang.NullPointerException
    at org.qortal.transform.block.BlockTransformer.decodeOnlineAccounts(BlockTransformer.java:422)
    at org.qortal.block.Block.getExpandedAccounts(Block.java:546)
    at org.qortal.block.Block.increaseAccountLevels(Block.java:1245)
    at org.qortal.block.Block.increaseAccountLevels(Block.java:1239)
    at org.qortal.block.Block.process(Block.java:1206)
    at org.qortal.block.GenesisBlock.process(GenesisBlock.java:345)
    at org.qortal.block.BlockChain.rebuildBlockchain(BlockChain.java:526)
    at org.qortal.block.BlockChain.validate(BlockChain.java:481)
    at org.qortal.controller.Controller.main(Controller.java:289)

The above happens because the old blockchain still exists when trying to process
the genesis block.
2020-03-11 15:25:04 +00:00
catbref
b0b0e2ac18 Strip JNI options before calling ApplyUpdate
AdvancedInstaller's Java launcher EXE seems to use JNI to launch
the JAR, instead of using the command-line 'java' binary directly.

When AI's launcher does this, it adds options like "abort" and "exit",
along with corresponding hook addresses.

These options are returned by the call to
ManagementFactory.getRuntimeMXBean().getInputArguments() which is
done in AutoUpdate while building the command line for launching
ApplyUpdate.

Because command-line 'java' binary doesn't support these options,
they are now stripped out.
2020-03-11 10:41:39 +00:00
69 changed files with 2306 additions and 1628 deletions

View File

@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<modelVersion>4.0.0</modelVersion>
<groupId>org.hsqldb</groupId>
<artifactId>hsqldb</artifactId>
<version>2.5.0-fixed</version>
<description>POM was created from install:install-file</description>
</project>

View File

@@ -0,0 +1,12 @@
<?xml version="1.0" encoding="UTF-8"?>
<metadata>
<groupId>org.hsqldb</groupId>
<artifactId>hsqldb</artifactId>
<versioning>
<release>2.5.0-fixed</release>
<versions>
<version>2.5.0-fixed</version>
</versions>
<lastUpdated>20200318133132</lastUpdated>
</versioning>
</metadata>

View File

@@ -3,7 +3,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>org.qortal</groupId>
<artifactId>qortal</artifactId>
<version>1.0</version>
<version>1.0.7</version>
<packaging>jar</packaging>
<properties>
<bitcoin.version>0.15.4</bitcoin.version>
@@ -13,7 +13,7 @@
<commons-text.version>1.8</commons-text.version>
<dagger.version>1.2.2</dagger.version>
<guava.version>28.1-jre</guava.version>
<hsqldb.version>2.5.0</hsqldb.version>
<hsqldb.version>2.5.0-fixed</hsqldb.version>
<hsqldb-sqltool.version>2.5.0</hsqldb-sqltool.version>
<jersey.version>2.29.1</jersey.version>
<jetty.version>9.4.22.v20191022</jetty.version>
@@ -257,6 +257,8 @@
<!-- Don't include original swagger-UI as we're including our own
modified version -->
<exclude>org.webjars:swagger-ui</exclude>
<!-- Don't include JUnit as it's for testing only! -->
<exclude>junit:junit</exclude>
</excludes>
</artifactSet>
<filters>
@@ -379,12 +381,14 @@
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>3.0.0</version>
<scope>provided</scope><!-- needed for build, not for runtime -->
</dependency>
<!-- https://mvnrepository.com/artifact/com.github.bohnman/package-info-maven-plugin -->
<dependency>
<groupId>com.github.bohnman</groupId>
<artifactId>package-info-maven-plugin</artifactId>
<version>${package-info-maven-plugin.version}</version>
<scope>provided</scope><!-- needed for build, not for runtime -->
</dependency>
<!-- HSQLDB for repository -->
<dependency>

17
run.sh
View File

@@ -6,6 +6,23 @@ if [ "$USER" = "root" ]; then
exit
fi
# Validate Java is installed and the minimum version is available
MIN_JAVA_VER='11'
if command -v java > /dev/null 2>&1; then
version=$(java -version 2>&1 | awk -F '"' '/version/ {print $2}')
version=$(echo $version | cut -d'.' -f1,2)
if [ `echo "${version}>=${MIN_JAVA_VER}" | bc` -eq 1 ]; then
echo 'Passed Java version check'
else
echo 'Please upgrade your Java to version 11 or greater'
exit 1
fi
else
echo 'Java is not available, please install Java 11 or greater'
exit 1
fi
# No qortal.jar but we have a Maven built one?
# Be helpful and copy across to correct location
if [ ! -e qortal.jar -a -f target/qortal*.jar ]; then

View File

@@ -36,8 +36,8 @@ public class ApplyUpdate {
private static final String NEW_JAR_FILENAME = AutoUpdate.NEW_JAR_FILENAME;
private static final String WINDOWS_EXE_LAUNCHER = "qortal.exe";
private static final long CHECK_INTERVAL = 5 * 1000L; // ms
private static final int MAX_ATTEMPTS = 5;
private static final long CHECK_INTERVAL = 10 * 1000L; // ms
private static final int MAX_ATTEMPTS = 12;
public static void main(String[] args) {
Security.insertProviderAt(new BouncyCastleProvider(), 0);

View File

@@ -53,8 +53,8 @@ public class Account {
// Balance manipulations - assetId is 0 for QORT
public BigDecimal getBalance(long assetId, int height) throws DataException {
AccountBalanceData accountBalanceData = this.repository.getAccountRepository().getBalance(this.address, assetId, height);
public BigDecimal getBalance(long assetId) throws DataException {
AccountBalanceData accountBalanceData = this.repository.getAccountRepository().getBalance(this.address, assetId);
if (accountBalanceData == null)
return BigDecimal.ZERO.setScale(8);
@@ -204,11 +204,15 @@ public class Account {
* @throws DataException
*/
public boolean canMint() throws DataException {
Integer level = this.getLevel();
AccountData accountData = this.repository.getAccountRepository().getAccount(this.address);
if (accountData == null)
return false;
Integer level = accountData.getLevel();
if (level != null && level >= BlockChain.getInstance().getMinAccountLevelToMint())
return true;
if (this.isFounder())
if (Account.isFounder(accountData.getFlags()))
return true;
return false;
@@ -226,11 +230,15 @@ public class Account {
* @throws DataException
*/
public boolean canRewardShare() throws DataException {
Integer level = this.getLevel();
AccountData accountData = this.repository.getAccountRepository().getAccount(this.address);
if (accountData == null)
return false;
Integer level = accountData.getLevel();
if (level != null && level >= BlockChain.getInstance().getMinAccountLevelToRewardShare())
return true;
if (this.isFounder())
if (Account.isFounder(accountData.getFlags()))
return true;
return false;
@@ -264,10 +272,14 @@ public class Account {
* @throws DataException
*/
public int getEffectiveMintingLevel() throws DataException {
if (this.isFounder())
AccountData accountData = this.repository.getAccountRepository().getAccount(this.address);
if (accountData == null)
return 0;
if (Account.isFounder(accountData.getFlags()))
return BlockChain.getInstance().getFounderEffectiveMintingLevel();
Integer level = this.getLevel();
Integer level = accountData.getLevel();
if (level == null)
return 0;
@@ -290,7 +302,7 @@ public class Account {
if (rewardShareData == null)
return 0;
PublicKeyAccount rewardShareMinter = new PublicKeyAccount(repository, rewardShareData.getMinterPublicKey());
Account rewardShareMinter = new Account(repository, rewardShareData.getMinter());
return rewardShareMinter.getEffectiveMintingLevel();
}

View File

@@ -149,8 +149,8 @@ public class ApiRequest {
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("GET");
con.setConnectTimeout(5000);
con.setReadTimeout(3000);
con.setConnectTimeout(30000);
con.setReadTimeout(10000);
ApiRequest.setConnectionSSL(con, ipAddress);
int status = con.getResponseCode();

View File

@@ -108,8 +108,19 @@ public class ApiService {
swaggerUIServlet.setInitParameter("pathInfoOnly", "true");
context.addServlet(swaggerUIServlet, "/api-documentation/*");
rewriteHandler.addRule(new RedirectPatternRule("", "/api-documentation/")); // redirect to Swagger UI start page
rewriteHandler.addRule(new RedirectPatternRule("/api-documentation", "/api-documentation/")); // redirect to Swagger UI start page
rewriteHandler.addRule(new RedirectPatternRule("", "/api-documentation/")); // redirect empty path to API docs
rewriteHandler.addRule(new RedirectPatternRule("/api-documentation", "/api-documentation/")); // redirect to add trailing slash if missing
} else {
// Simple pages that explains that API documentation is disabled
ClassLoader loader = this.getClass().getClassLoader();
ServletHolder swaggerUIServlet = new ServletHolder("api-docs-disabled", DefaultServlet.class);
swaggerUIServlet.setInitParameter("resourceBase", loader.getResource("api-docs-disabled/").toString());
swaggerUIServlet.setInitParameter("dirAllowed", "true");
swaggerUIServlet.setInitParameter("pathInfoOnly", "true");
context.addServlet(swaggerUIServlet, "/api-documentation/*");
rewriteHandler.addRule(new RedirectPatternRule("", "/api-documentation/")); // redirect empty path to API docs
rewriteHandler.addRule(new RedirectPatternRule("/api-documentation", "/api-documentation/")); // redirect to add trailing slash if missing
}
// Start server

View File

@@ -32,10 +32,13 @@ public class BlockMinterSummary {
}
/** Constructs BlockMinterSummary in reward-share context. */
public BlockMinterSummary(byte[] rewardSharePublicKey, int blockCount, byte[] mintingAccountPublicKey, String recipientAccount) {
this(mintingAccountPublicKey, blockCount);
public BlockMinterSummary(byte[] rewardSharePublicKey, int blockCount, byte[] mintingAccountPublicKey, String minterAccount, String recipientAccount) {
this.rewardSharePublicKey = rewardSharePublicKey;
this.blockCount = blockCount;
this.mintingAccountPublicKey = mintingAccountPublicKey;
this.mintingAccount = minterAccount;
this.recipientAccount = recipientAccount;
}

View File

@@ -0,0 +1,18 @@
package org.qortal.api.model;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
@XmlAccessorType(XmlAccessType.FIELD)
public class NodeStatus {
public boolean isMintingPossible;
public boolean isSynchronizing;
// Not always present
public Integer syncPercent;
public NodeStatus() {
}
}

View File

@@ -192,7 +192,7 @@ public class AddressesResource {
@Path("/balance/{address}")
@Operation(
summary = "Returns account balance",
description = "Returns account's balance, optionally of given asset and at given height",
description = "Returns account's QORT balance, or of other specified asset",
responses = {
@ApiResponse(
description = "the balance",
@@ -202,8 +202,7 @@ public class AddressesResource {
)
@ApiErrors({ApiError.INVALID_ADDRESS, ApiError.INVALID_ASSET_ID, ApiError.INVALID_HEIGHT, ApiError.REPOSITORY_ISSUE})
public BigDecimal getBalance(@PathParam("address") String address,
@QueryParam("assetId") Long assetId,
@QueryParam("height") Integer height) {
@QueryParam("assetId") Long assetId) {
if (!Crypto.isValidAddress(address))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
@@ -215,12 +214,7 @@ public class AddressesResource {
else if (!repository.getAssetRepository().assetExists(assetId))
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ASSET_ID);
if (height == null)
height = repository.getBlockRepository().getBlockchainHeight();
else if (height <= 0)
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_HEIGHT);
return account.getBalance(assetId, height);
return account.getBalance(assetId);
} catch (ApiException e) {
throw e;
} catch (DataException e) {

View File

@@ -36,8 +36,8 @@ import javax.ws.rs.core.MediaType;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.core.LoggerContext;
import org.apache.logging.log4j.core.appender.RollingFileAppender;
import org.qortal.account.Account;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.account.PublicKeyAccount;
import org.qortal.api.ApiError;
import org.qortal.api.ApiErrors;
import org.qortal.api.ApiException;
@@ -45,6 +45,7 @@ import org.qortal.api.ApiExceptionFactory;
import org.qortal.api.Security;
import org.qortal.api.model.ActivitySummary;
import org.qortal.api.model.NodeInfo;
import org.qortal.api.model.NodeStatus;
import org.qortal.block.BlockChain;
import org.qortal.controller.Controller;
import org.qortal.controller.Synchronizer.SynchronizationResult;
@@ -120,6 +121,30 @@ public class AdminResource {
return nodeInfo;
}
@GET
@Path("/status")
@Operation(
summary = "Fetch node status",
responses = {
@ApiResponse(
content = @Content(mediaType = MediaType.APPLICATION_JSON, schema = @Schema(implementation = NodeStatus.class))
)
}
)
public NodeStatus status() {
Security.checkApiCallAllowed(request);
NodeStatus nodeStatus = new NodeStatus();
nodeStatus.isMintingPossible = Controller.getInstance().isMintingPossible();
nodeStatus.isSynchronizing = Controller.getInstance().isSynchronizing();
if (nodeStatus.isSynchronizing)
nodeStatus.syncPercent = Controller.getInstance().getSyncPercent();
return nodeStatus;
}
@GET
@Path("/stop")
@Operation(
@@ -218,7 +243,7 @@ public class AdminResource {
// ignore
}
return new MintingAccountData(mintingAccountData.getPrivateKey(), rewardShareData);
return new MintingAccountData(mintingAccountData, rewardShareData);
}).collect(Collectors.toList());
return mintingAccounts;
@@ -262,11 +287,11 @@ public class AdminResource {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
// Qortal: check reward-share's minting account is still allowed to mint
PublicKeyAccount rewardShareMintingAccount = new PublicKeyAccount(repository, rewardShareData.getMinterPublicKey());
Account rewardShareMintingAccount = new Account(repository, rewardShareData.getMinter());
if (!rewardShareMintingAccount.canMint())
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.CANNOT_MINT);
MintingAccountData mintingAccountData = new MintingAccountData(seed);
MintingAccountData mintingAccountData = new MintingAccountData(mintingAccount.getPrivateKey(), mintingAccount.getPublicKey());
repository.getAccountRepository().save(mintingAccountData);
repository.saveChanges();

View File

@@ -31,6 +31,7 @@ import org.qortal.network.PeerAddress;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.utils.ExecuteProduceConsume;
@Path("/peers")
@Tag(name = "Peers")
@@ -108,6 +109,29 @@ public class PeersResource {
return Network.getInstance().getSelfPeers();
}
@GET
@Path("/enginestats")
@Operation(
summary = "Fetch statistics snapshot for networking engine",
responses = {
@ApiResponse(
content = @Content(
mediaType = MediaType.APPLICATION_JSON,
array = @ArraySchema(
schema = @Schema(
implementation = ExecuteProduceConsume.StatsSnapshot.class
)
)
)
)
}
)
public ExecuteProduceConsume.StatsSnapshot getEngineStats() {
Security.checkApiCallAllowed(request);
return Network.getInstance().getStatsSnapshot();
}
@POST
@Operation(
summary = "Add new peer address",

View File

@@ -470,7 +470,7 @@ public class TransactionsResource {
throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_SIGNATURE);
ReentrantLock blockchainLock = Controller.getInstance().getBlockchainLock();
if (!blockchainLock.tryLock(500, TimeUnit.MILLISECONDS))
if (!blockchainLock.tryLock(30, TimeUnit.SECONDS))
throw createTransactionInvalidException(request, ValidationResult.NO_BLOCKCHAIN_LOCK);
try {
@@ -573,7 +573,7 @@ public class TransactionsResource {
public static ApiException createTransactionInvalidException(HttpServletRequest request, ValidationResult result) {
String translatedResult = Translator.INSTANCE.translate("TransactionValidity", request.getLocale().getLanguage(), result.name());
return ApiExceptionFactory.INSTANCE.createException(request, ApiError.TRANSACTION_INVALID, null, translatedResult);
return ApiExceptionFactory.INSTANCE.createException(request, ApiError.TRANSACTION_INVALID, null, translatedResult, result.name());
}
}

View File

@@ -9,7 +9,6 @@ import java.math.RoundingMode;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.function.Function;
@@ -126,31 +125,43 @@ public class Block {
protected BigDecimal ourAtFees; // Generated locally
/** Lazy-instantiated expanded info on block's online accounts. */
class ExpandedAccount {
final RewardShareData rewardShareData;
final boolean isRecipientAlsoMinter;
static class ExpandedAccount {
private static final BigDecimal oneHundred = BigDecimal.valueOf(100L);
final Account mintingAccount;
final AccountData mintingAccountData;
final boolean isMinterFounder;
private final Repository repository;
final Account recipientAccount;
final AccountData recipientAccountData;
final boolean isRecipientFounder;
private final RewardShareData rewardShareData;
private final boolean isRecipientAlsoMinter;
private final Account mintingAccount;
private final AccountData mintingAccountData;
private final boolean isMinterFounder;
private final Account recipientAccount;
private final AccountData recipientAccountData;
private final boolean isRecipientFounder;
ExpandedAccount(Repository repository, int accountIndex) throws DataException {
this.repository = repository;
this.rewardShareData = repository.getAccountRepository().getRewardShareByIndex(accountIndex);
this.mintingAccount = new PublicKeyAccount(repository, this.rewardShareData.getMinterPublicKey());
this.recipientAccount = new Account(repository, this.rewardShareData.getRecipient());
this.mintingAccount = new Account(repository, this.rewardShareData.getMinter());
this.mintingAccountData = repository.getAccountRepository().getAccount(this.mintingAccount.getAddress());
this.isMinterFounder = Account.isFounder(mintingAccountData.getFlags());
this.recipientAccountData = repository.getAccountRepository().getAccount(this.recipientAccount.getAddress());
this.isRecipientFounder = Account.isFounder(recipientAccountData.getFlags());
this.isRecipientAlsoMinter = this.rewardShareData.getRecipient().equals(this.mintingAccount.getAddress());
this.isRecipientAlsoMinter = this.mintingAccountData.getAddress().equals(this.recipientAccountData.getAddress());
if (this.isRecipientAlsoMinter) {
// Self-share: minter is also recipient
this.recipientAccount = this.mintingAccount;
this.recipientAccountData = this.mintingAccountData;
this.isRecipientFounder = this.isMinterFounder;
} else {
// Recipient differs from minter
this.recipientAccount = new Account(repository, this.rewardShareData.getRecipient());
this.recipientAccountData = repository.getAccountRepository().getAccount(this.recipientAccount.getAddress());
this.isRecipientFounder = Account.isFounder(recipientAccountData.getFlags());
}
}
/**
@@ -176,22 +187,26 @@ public class Block {
}
void distribute(BigDecimal accountAmount) throws DataException {
final BigDecimal oneHundred = BigDecimal.valueOf(100L);
if (this.mintingAccount.getAddress().equals(this.recipientAccount.getAddress())) {
if (this.isRecipientAlsoMinter) {
// minter & recipient the same - simpler case
LOGGER.trace(() -> String.format("Minter/recipient account %s share: %s", this.mintingAccount.getAddress(), accountAmount.toPlainString()));
this.mintingAccount.setConfirmedBalance(Asset.QORT, this.mintingAccount.getConfirmedBalance(Asset.QORT).add(accountAmount));
if (accountAmount.signum() != 0)
// this.mintingAccount.setConfirmedBalance(Asset.QORT, this.mintingAccount.getConfirmedBalance(Asset.QORT).add(accountAmount));
this.repository.getAccountRepository().modifyAssetBalance(this.mintingAccount.getAddress(), Asset.QORT, accountAmount);
} else {
// minter & recipient different - extra work needed
BigDecimal recipientAmount = accountAmount.multiply(this.rewardShareData.getSharePercent()).divide(oneHundred, RoundingMode.DOWN);
BigDecimal minterAmount = accountAmount.subtract(recipientAmount);
LOGGER.trace(() -> String.format("Minter account %s share: %s", this.mintingAccount.getAddress(), minterAmount.toPlainString()));
this.mintingAccount.setConfirmedBalance(Asset.QORT, this.mintingAccount.getConfirmedBalance(Asset.QORT).add(minterAmount));
if (minterAmount.signum() != 0)
// this.mintingAccount.setConfirmedBalance(Asset.QORT, this.mintingAccount.getConfirmedBalance(Asset.QORT).add(minterAmount));
this.repository.getAccountRepository().modifyAssetBalance(this.mintingAccount.getAddress(), Asset.QORT, minterAmount);
LOGGER.trace(() -> String.format("Recipient account %s share: %s", this.recipientAccount.getAddress(), recipientAmount.toPlainString()));
this.recipientAccount.setConfirmedBalance(Asset.QORT, this.recipientAccount.getConfirmedBalance(Asset.QORT).add(recipientAmount));
if (recipientAmount.signum() != 0)
// this.recipientAccount.setConfirmedBalance(Asset.QORT, this.recipientAccount.getConfirmedBalance(Asset.QORT).add(recipientAmount));
this.repository.getAccountRepository().modifyAssetBalance(this.recipientAccount.getAddress(), Asset.QORT, recipientAmount);
}
}
}
@@ -901,26 +916,34 @@ public class Block {
expandedAccounts.add(rewardShareData);
}
// Possibly check signatures if block is recent
// If block is past a certain age then we simply assume the signatures were correct
long signatureRequirementThreshold = NTP.getTime() - BlockChain.getInstance().getOnlineAccountSignaturesMinLifetime();
if (this.blockData.getTimestamp() >= signatureRequirementThreshold) {
if (this.blockData.getOnlineAccountsSignatures() == null || this.blockData.getOnlineAccountsSignatures().length == 0)
return ValidationResult.ONLINE_ACCOUNT_SIGNATURES_MISSING;
if (this.blockData.getTimestamp() < signatureRequirementThreshold)
return ValidationResult.OK;
if (this.blockData.getOnlineAccountsSignatures().length != expandedAccounts.size() * Transformer.SIGNATURE_LENGTH)
return ValidationResult.ONLINE_ACCOUNT_SIGNATURES_MALFORMED;
if (this.blockData.getOnlineAccountsSignatures() == null || this.blockData.getOnlineAccountsSignatures().length == 0)
return ValidationResult.ONLINE_ACCOUNT_SIGNATURES_MISSING;
// Check signatures
List<byte[]> onlineAccountsSignatures = BlockTransformer.decodeTimestampSignatures(this.blockData.getOnlineAccountsSignatures());
byte[] message = Longs.toByteArray(this.blockData.getOnlineAccountsTimestamp());
if (this.blockData.getOnlineAccountsSignatures().length != expandedAccounts.size() * Transformer.SIGNATURE_LENGTH)
return ValidationResult.ONLINE_ACCOUNT_SIGNATURES_MALFORMED;
for (int i = 0; i < onlineAccountsSignatures.size(); ++i) {
PublicKeyAccount account = new PublicKeyAccount(null, expandedAccounts.get(i).getRewardSharePublicKey());
byte[] signature = onlineAccountsSignatures.get(i);
// Check signatures
List<byte[]> onlineAccountsSignatures = BlockTransformer.decodeTimestampSignatures(this.blockData.getOnlineAccountsSignatures());
long onlineTimestamp = this.blockData.getOnlineAccountsTimestamp();
byte[] onlineTimestampBytes = Longs.toByteArray(onlineTimestamp);
List<OnlineAccountData> onlineAccounts = Controller.getInstance().getOnlineAccounts();
if (!account.verify(signature, message))
return ValidationResult.ONLINE_ACCOUNT_SIGNATURE_INCORRECT;
}
for (int i = 0; i < onlineAccountsSignatures.size(); ++i) {
byte[] signature = onlineAccountsSignatures.get(i);
byte[] publicKey = expandedAccounts.get(i).getRewardSharePublicKey();
// If signature is still current then no need to perform Ed25519 verify
OnlineAccountData onlineAccountData = new OnlineAccountData(onlineTimestamp, signature, publicKey);
if (onlineAccounts.remove(onlineAccountData)) // remove() is like contains() but also reduces the number to check next time
continue;
if (!PublicKeyAccount.verify(publicKey, signature, onlineTimestampBytes))
return ValidationResult.ONLINE_ACCOUNT_SIGNATURE_INCORRECT;
}
return ValidationResult.OK;
@@ -1003,6 +1026,10 @@ public class Block {
for (Transaction transaction : this.getTransactions()) {
TransactionData transactionData = transaction.getTransactionData();
// Skip AT transactions as they are covered by prior call to Block.areAtsValid()
if (transactionData.getType() == TransactionType.AT)
continue;
// GenesisTransactions are not allowed (GenesisBlock overrides isValid() to allow them)
if (transactionData.getType() == TransactionType.GENESIS || transactionData.getType() == TransactionType.ACCOUNT_FLAGS)
return ValidationResult.GENESIS_TRANSACTIONS_INVALID;
@@ -1256,8 +1283,9 @@ public class Block {
AccountData accountData = getAccountData.apply(expandedAccount);
accountData.setBlocksMinted(accountData.getBlocksMinted() + 1);
repository.getAccountRepository().setMintedBlockCount(accountData);
LOGGER.trace(() -> String.format("Block minter %s up to %d minted block%s", accountData.getAddress(), accountData.getBlocksMinted(), (accountData.getBlocksMinted() != 1 ? "s" : "")));
// repository.getAccountRepository().setMintedBlockCount(accountData); int rowCount = 1; // Until HSQLDB rev 6100 is fixed
int rowCount = repository.getAccountRepository().modifyMintedBlockCount(accountData.getAddress(), +1);
LOGGER.trace(() -> String.format("Block minter %s up to %d minted block%s (rowCount: %d)", accountData.getAddress(), accountData.getBlocksMinted(), (accountData.getBlocksMinted() != 1 ? "s" : ""), rowCount));
}
// We are only interested in accounts that are NOT already highest level
@@ -1425,35 +1453,37 @@ public class Block {
public void orphan() throws DataException {
LOGGER.trace(() -> String.format("Orphaning block %d", this.blockData.getHeight()));
// Return AT fees and delete AT states from repository
orphanAtFeesAndStates();
this.repository.setDebug(false);
try {
// Return AT fees and delete AT states from repository
orphanAtFeesAndStates();
// Orphan, and unlink, transactions from this block
orphanTransactionsFromBlock();
// Orphan, and unlink, transactions from this block
orphanTransactionsFromBlock();
// Undo any group-approval decisions that happen at this block
orphanGroupApprovalTransactions();
// Undo any group-approval decisions that happen at this block
orphanGroupApprovalTransactions();
if (this.blockData.getHeight() > 1) {
// Invalidate expandedAccounts as they may have changed due to orphaning TRANSFER_PRIVS transactions, etc.
this.cachedExpandedAccounts = null;
if (this.blockData.getHeight() > 1) {
// Invalidate expandedAccounts as they may have changed due to orphaning TRANSFER_PRIVS transactions, etc.
this.cachedExpandedAccounts = null;
// Deduct any transaction fees from minter/reward-share account(s)
deductTransactionFees();
// Deduct any transaction fees from minter/reward-share account(s)
deductTransactionFees();
// Block rewards removed after transactions undone
orphanBlockRewards();
// Block rewards removed after transactions undone
orphanBlockRewards();
// Decrease account levels
decreaseAccountLevels();
// Decrease account levels
decreaseAccountLevels();
}
// Delete block from blockchain
this.repository.getBlockRepository().delete(this.blockData);
this.blockData.setHeight(null);
} finally {
this.repository.setDebug(false);
}
// Delete orphaned balances
this.repository.getAccountRepository().deleteBalancesFromHeight(this.blockData.getHeight());
// Delete block from blockchain
this.repository.getBlockRepository().delete(this.blockData);
this.blockData.setHeight(null);
}
protected void orphanTransactionsFromBlock() throws DataException {
@@ -1571,8 +1601,9 @@ public class Block {
AccountData accountData = getAccountData.apply(expandedAccount);
accountData.setBlocksMinted(accountData.getBlocksMinted() - 1);
repository.getAccountRepository().setMintedBlockCount(accountData);
LOGGER.trace(() -> String.format("Block minter %s down to %d minted block%s", accountData.getAddress(), accountData.getBlocksMinted(), (accountData.getBlocksMinted() != 1 ? "s" : "")));
// repository.getAccountRepository().setMintedBlockCount(accountData); int rowCount = 1; // Until HSQLDB rev 6100 is fixed
int rowCount = repository.getAccountRepository().modifyMintedBlockCount(accountData.getAddress(), -1);
LOGGER.trace(() -> String.format("Block minter %s down to %d minted block%s (rowCount: %d)", accountData.getAddress(), accountData.getBlocksMinted(), (accountData.getBlocksMinted() != 1 ? "s" : ""), rowCount));
}
// We are only interested in accounts that are NOT already lowest level
@@ -1602,8 +1633,22 @@ public class Block {
protected void distributeBlockReward(BigDecimal totalAmount) throws DataException {
LOGGER.trace(() -> String.format("Distributing: %s", totalAmount.toPlainString()));
List<ShareByLevel> sharesByLevel = BlockChain.getInstance().getBlockSharesByLevel();
// Distribute according to account level
BigDecimal sharedByLevelAmount = distributeBlockRewardByLevel(totalAmount);
LOGGER.trace(() -> String.format("Shared %s of %s based on account levels", sharedByLevelAmount.toPlainString(), totalAmount.toPlainString()));
// Distribute amongst legacy QORA holders
BigDecimal sharedByQoraHoldersAmount = distributeBlockRewardToQoraHolders(totalAmount);
LOGGER.trace(() -> String.format("Shared %s of %s to legacy QORA holders", sharedByQoraHoldersAmount.toPlainString(), totalAmount.toPlainString()));
// Spread remainder across founder accounts
BigDecimal foundersAmount = totalAmount.subtract(sharedByLevelAmount).subtract(sharedByQoraHoldersAmount);
distributeBlockRewardToFounders(foundersAmount);
}
private BigDecimal distributeBlockRewardByLevel(BigDecimal totalAmount) throws DataException {
List<ExpandedAccount> expandedAccounts = this.getExpandedAccounts();
List<ShareByLevel> sharesByLevel = BlockChain.getInstance().getBlockSharesByLevel();
// Distribute amount across bins
BigDecimal sharedAmount = BigDecimal.ZERO;
@@ -1628,36 +1673,17 @@ public class Block {
}
}
// Distribute share across legacy QORA holders
return sharedAmount;
}
private BigDecimal distributeBlockRewardToQoraHolders(BigDecimal totalAmount) throws DataException {
BigDecimal qoraHoldersAmount = BlockChain.getInstance().getQoraHoldersShare().multiply(totalAmount).setScale(8, RoundingMode.DOWN);
LOGGER.trace(() -> String.format("Legacy QORA holders share of %s: %s", totalAmount.toPlainString(), qoraHoldersAmount.toPlainString()));
List<AccountBalanceData> qoraHolders = this.repository.getAccountRepository().getAssetBalances(Asset.LEGACY_QORA, true);
final boolean isProcessingNotOrphaning = totalAmount.signum() >= 0;
// Filter out qoraHolders who have received max QORT due to holding legacy QORA, (ratio from blockchain config)
BigDecimal qoraPerQortReward = BlockChain.getInstance().getQoraPerQortReward();
Iterator<AccountBalanceData> qoraHoldersIterator = qoraHolders.iterator();
while (qoraHoldersIterator.hasNext()) {
AccountBalanceData qoraHolder = qoraHoldersIterator.next();
Account qoraHolderAccount = new Account(repository, qoraHolder.getAddress());
BigDecimal qortFromQora = qoraHolderAccount.getConfirmedBalance(Asset.QORT_FROM_QORA);
// If we're processing a block, then totalAmount will be positive
if (totalAmount.signum() >= 0) {
BigDecimal maxQortFromQora = qoraHolder.getBalance().divide(qoraPerQortReward, RoundingMode.DOWN);
// Disregard qora holders who have already received maximum qort from holding legacy qora
if (qortFromQora.compareTo(maxQortFromQora) >= 0)
qoraHoldersIterator.remove();
} else {
// We're orphaning a block
// so disregard qora holders who have already had their final qort-from-qora reward (i.e. reward reward block is earlier than this one)
QortFromQoraData qortFromQoraData = this.repository.getAccountRepository().getQortFromQoraInfo(qoraHolder.getAddress());
if (qortFromQoraData != null && qortFromQoraData.getFinalBlockHeight() < this.blockData.getHeight())
qoraHoldersIterator.remove();
}
}
List<AccountBalanceData> qoraHolders = this.repository.getAccountRepository().getEligibleLegacyQoraHolders(isProcessingNotOrphaning ? null : this.blockData.getHeight());
BigDecimal totalQoraHeld = BigDecimal.ZERO;
for (int i = 0; i < qoraHolders.size(); ++i)
@@ -1666,6 +1692,10 @@ public class Block {
BigDecimal finalTotalQoraHeld = totalQoraHeld;
LOGGER.trace(() -> String.format("Total legacy QORA held: %s", finalTotalQoraHeld.toPlainString()));
BigDecimal sharedAmount = BigDecimal.ZERO;
if (totalQoraHeld.signum() <= 0)
return sharedAmount;
for (int h = 0; h < qoraHolders.size(); ++h) {
AccountBalanceData qoraHolder = qoraHolders.get(h);
@@ -1674,12 +1704,16 @@ public class Block {
LOGGER.trace(() -> String.format("QORA holder %s has %s / %s QORA so share: %s",
qoraHolder.getAddress(), qoraHolder.getBalance().toPlainString(), finalTotalQoraHeld, finalHolderReward.toPlainString()));
// Too small to register this time?
if (holderReward.signum() == 0)
continue;
Account qoraHolderAccount = new Account(repository, qoraHolder.getAddress());
BigDecimal newQortFromQoraBalance = qoraHolderAccount.getConfirmedBalance(Asset.QORT_FROM_QORA).add(holderReward);
// If processing, make sure we don't overpay
if (totalAmount.signum() >= 0) {
if (isProcessingNotOrphaning) {
BigDecimal maxQortFromQora = qoraHolder.getBalance().divide(qoraPerQortReward, RoundingMode.DOWN);
if (newQortFromQoraBalance.compareTo(maxQortFromQora) >= 0) {
@@ -1689,7 +1723,7 @@ public class Block {
holderReward = holderReward.subtract(adjustment);
newQortFromQoraBalance = newQortFromQoraBalance.subtract(adjustment);
// This is also qora holders final qort-from-qora block
// This is also the QORA holder's final QORT-from-QORA block
QortFromQoraData qortFromQoraData = new QortFromQoraData(qoraHolder.getAddress(), holderReward, this.blockData.getHeight());
this.repository.getAccountRepository().save(qortFromQoraData);
@@ -1701,9 +1735,10 @@ public class Block {
// Orphaning
QortFromQoraData qortFromQoraData = this.repository.getAccountRepository().getQortFromQoraInfo(qoraHolder.getAddress());
if (qortFromQoraData != null) {
// Note use of negate() here as qortFromQora will be negative during orphaning,
// but final qort-from-qora is stored in repository during processing (and hence positive).
BigDecimal adjustment = holderReward.subtract(qortFromQoraData.getFinalQortFromQora().negate());
// Final QORT-from-QORA amount from repository was stored during processing, and hence positive.
// So we use add() here as qortFromQora is negative during orphaning.
// More efficient than holderReward.subtract(final-qort-from-qora.negate())
BigDecimal adjustment = holderReward.add(qortFromQoraData.getFinalQortFromQora());
holderReward = holderReward.subtract(adjustment);
newQortFromQoraBalance = newQortFromQoraBalance.subtract(adjustment);
@@ -1716,7 +1751,8 @@ public class Block {
}
}
qoraHolderAccount.setConfirmedBalance(Asset.QORT, qoraHolderAccount.getConfirmedBalance(Asset.QORT).add(holderReward));
// qoraHolderAccount.setConfirmedBalance(Asset.QORT, qoraHolderAccount.getConfirmedBalance(Asset.QORT).add(holderReward));
this.repository.getAccountRepository().modifyAssetBalance(qoraHolder.getAddress(), Asset.QORT, holderReward);
if (newQortFromQoraBalance.signum() > 0)
qoraHolderAccount.setConfirmedBalance(Asset.QORT_FROM_QORA, newQortFromQoraBalance);
@@ -1727,27 +1763,39 @@ public class Block {
sharedAmount = sharedAmount.add(holderReward);
}
// Spread remainder across founder accounts
BigDecimal foundersAmount = totalAmount.subtract(sharedAmount);
BigDecimal finalSharedAmount = sharedAmount;
return sharedAmount;
}
private void distributeBlockRewardToFounders(BigDecimal foundersAmount) throws DataException {
// Remaining reward portion is spread across all founders, online or not
List<AccountData> founderAccounts = this.repository.getAccountRepository().getFlaggedAccounts(Account.FOUNDER_FLAG);
BigDecimal foundersCount = BigDecimal.valueOf(founderAccounts.size());
BigDecimal perFounderAmount = foundersAmount.divide(foundersCount, RoundingMode.DOWN);
LOGGER.trace(() -> String.format("Shared %s of %s, remaining %s to %d founder%s, %s each",
finalSharedAmount.toPlainString(), totalAmount.toPlainString(),
LOGGER.trace(() -> String.format("Sharing remaining %s to %d founder%s, %s each",
foundersAmount.toPlainString(), founderAccounts.size(), (founderAccounts.size() != 1 ? "s" : ""),
perFounderAmount.toPlainString()));
List<ExpandedAccount> expandedAccounts = this.getExpandedAccounts();
for (int a = 0; a < founderAccounts.size(); ++a) {
Account founderAccount = new Account(this.repository, founderAccounts.get(a).getAddress());
// If founder is minter in any online reward-shares then founder's amount is spread across these, otherwise founder gets whole amount.
/* Fixed version:
List<ExpandedAccount> founderExpandedAccounts = expandedAccounts.stream().filter(
accountInfo -> accountInfo.isMinterFounder &&
accountInfo.mintingAccountData.getAddress().equals(founderAccount.getAddress())
).collect(Collectors.toList());
*/
// Broken version:
List<ExpandedAccount> founderExpandedAccounts = expandedAccounts.stream().filter(accountInfo -> accountInfo.isMinterFounder).collect(Collectors.toList());
if (founderExpandedAccounts.isEmpty()) {
// Simple case: no founder-as-minter reward-shares online so founder gets whole amount.
Account founderAccount = new Account(this.repository, founderAccounts.get(a).getAddress());
founderAccount.setConfirmedBalance(Asset.QORT, founderAccount.getConfirmedBalance(Asset.QORT).add(perFounderAmount));
// founderAccount.setConfirmedBalance(Asset.QORT, founderAccount.getConfirmedBalance(Asset.QORT).add(perFounderAmount));
this.repository.getAccountRepository().modifyAssetBalance(founderAccount.getAddress(), Asset.QORT, perFounderAmount);
} else {
// Distribute over reward-shares
BigDecimal perFounderRewardShareAmount = perFounderAmount.divide(BigDecimal.valueOf(founderExpandedAccounts.size()), RoundingMode.DOWN);

View File

@@ -5,6 +5,7 @@ import java.util.ArrayList;
import java.util.Arrays;
import java.util.Iterator;
import java.util.List;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.ReentrantLock;
import java.util.stream.Collectors;
@@ -12,7 +13,6 @@ import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.account.Account;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.account.PublicKeyAccount;
import org.qortal.block.Block.ValidationResult;
import org.qortal.controller.Controller;
import org.qortal.data.account.MintingAccountData;
@@ -76,20 +76,15 @@ public class BlockMinter extends Thread {
boolean isMintingPossible = false;
boolean wasMintingPossible = isMintingPossible;
while (running) {
repository.discardChanges(); // Free repository locks, if any
if (isMintingPossible != wasMintingPossible)
Controller.getInstance().onMintingPossibleChange(isMintingPossible);
wasMintingPossible = isMintingPossible;
// Sleep for a while
try {
repository.discardChanges(); // Free repository locks, if any
if (isMintingPossible != wasMintingPossible)
Controller.getInstance().onMintingPossibleChange(isMintingPossible);
wasMintingPossible = isMintingPossible;
Thread.sleep(1000);
} catch (InterruptedException e) {
// We've been interrupted - time to exit
return;
}
Thread.sleep(1000);
isMintingPossible = false;
@@ -123,7 +118,7 @@ public class BlockMinter extends Thread {
continue;
}
PublicKeyAccount mintingAccount = new PublicKeyAccount(repository, rewardShareData.getMinterPublicKey());
Account mintingAccount = new Account(repository, rewardShareData.getMinter());
if (!mintingAccount.canMint()) {
// Minting-account component of reward-share can no longer mint - disregard
madi.remove();
@@ -137,19 +132,18 @@ public class BlockMinter extends Thread {
// Disregard peers that have "misbehaved" recently
peers.removeIf(Controller.hasMisbehaved);
// Don't mint if we don't have enough connected peers as where would the transactions/consensus come from?
if (peers.size() < Settings.getInstance().getMinBlockchainPeers())
continue;
// Disregard peers that don't have a recent block
peers.removeIf(Controller.hasNoRecentBlock);
// If we have any peers with a recent block, but our latest block isn't recent
// then we need to synchronize instead of minting.
// Don't mint if we don't have enough up-to-date peers as where would the transactions/consensus come from?
if (peers.size() < Settings.getInstance().getMinBlockchainPeers())
continue;
// If our latest block isn't recent then we need to synchronize instead of minting.
if (!peers.isEmpty() && lastBlockData.getTimestamp() < minLatestBlockTimestamp)
continue;
// There are no peers with a recent block and/or our latest block is recent
// There are enough peers with a recent block and our latest block is recent
// so go ahead and mint a block if possible.
isMintingPossible = true;
@@ -159,12 +153,12 @@ public class BlockMinter extends Thread {
newBlocks.clear();
}
// Discard accounts we have already built blocks with
mintingAccountsData.removeIf(mintingAccountData -> newBlocks.stream().anyMatch(newBlock -> Arrays.equals(newBlock.getBlockData().getMinterPublicKey(), mintingAccountData.getPublicKey())));
// Do we need to build any potential new blocks?
List<PrivateKeyAccount> mintingAccounts = mintingAccountsData.stream().map(accountData -> new PrivateKeyAccount(repository, accountData.getPrivateKey())).collect(Collectors.toList());
// Discard accounts we have blocks for
mintingAccounts.removeIf(account -> newBlocks.stream().anyMatch(newBlock -> newBlock.getMinter().getAddress().equals(account.getAddress())));
for (PrivateKeyAccount mintingAccount : mintingAccounts) {
// First block does the AT heavy-lifting
if (newBlocks.isEmpty()) {
@@ -183,7 +177,7 @@ public class BlockMinter extends Thread {
// Make sure we're the only thread modifying the blockchain
ReentrantLock blockchainLock = Controller.getInstance().getBlockchainLock();
if (!blockchainLock.tryLock())
if (!blockchainLock.tryLock(30, TimeUnit.SECONDS))
continue;
boolean newBlockMinted = false;
@@ -258,11 +252,10 @@ public class BlockMinter extends Thread {
RewardShareData rewardShareData = repository.getAccountRepository().getRewardShare(newBlock.getBlockData().getMinterPublicKey());
if (rewardShareData != null) {
PublicKeyAccount mintingAccount = new PublicKeyAccount(repository, rewardShareData.getMinterPublicKey());
LOGGER.info(String.format("Minted block %d, sig %.8s by %s on behalf of %s",
newBlock.getBlockData().getHeight(),
Base58.encode(newBlock.getBlockData().getSignature()),
mintingAccount.getAddress(),
rewardShareData.getMinter(),
rewardShareData.getRecipient()));
} else {
LOGGER.info(String.format("Minted block %d, sig %.8s by %s",
@@ -289,6 +282,9 @@ public class BlockMinter extends Thread {
}
} catch (DataException e) {
LOGGER.warn("Repository issue while running block minter", e);
} catch (InterruptedException e) {
// We've been interrupted - time to exit
return;
}
}
@@ -341,17 +337,19 @@ public class BlockMinter extends Thread {
this.interrupt();
}
public static void mintTestingBlock(Repository repository, PrivateKeyAccount mintingAccount) throws DataException {
public static void mintTestingBlock(Repository repository, PrivateKeyAccount... mintingAndOnlineAccounts) throws DataException {
if (!BlockChain.getInstance().isTestChain()) {
LOGGER.warn("Ignoring attempt to mint testing block for non-test chain!");
return;
}
// Ensure mintingAccount is 'online' so blocks can be minted
Controller.getInstance().ensureTestingAccountOnline(mintingAccount);
Controller.getInstance().ensureTestingAccountsOnline(mintingAndOnlineAccounts);
BlockData previousBlockData = repository.getBlockRepository().getLastBlock();
PrivateKeyAccount mintingAccount = mintingAndOnlineAccounts[0];
Block newBlock = Block.mint(repository, previousBlockData, mintingAccount);
// Make sure we're the only thread modifying the blockchain

View File

@@ -21,6 +21,7 @@ import org.qortal.ApplyUpdate;
import org.qortal.api.ApiRequest;
import org.qortal.data.transaction.ArbitraryTransactionData;
import org.qortal.data.transaction.TransactionData;
import org.qortal.globalization.Translator;
import org.qortal.gui.SysTray;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
@@ -40,7 +41,7 @@ public class AutoUpdate extends Thread {
public static final String NEW_JAR_FILENAME = "new-" + JAR_FILENAME;
private static final Logger LOGGER = LogManager.getLogger(AutoUpdate.class);
private static final long CHECK_INTERVAL = 5 * 60 * 1000L; // ms
private static final long CHECK_INTERVAL = 20 * 60 * 1000L; // ms
private static final int DEV_GROUP_ID = 1;
private static final int UPDATE_SERVICE = 1;
@@ -209,7 +210,7 @@ public class AutoUpdate extends Thread {
return false; // failed - try another repo
}
} catch (IOException e) {
LOGGER.warn(String.format("Failed to fetch update from %s", repoUri));
LOGGER.warn(String.format("Failed to fetch update from %s: %s", repoUri, e.getMessage()));
return false; // failed - try another repo
}
@@ -231,6 +232,10 @@ public class AutoUpdate extends Thread {
// JVM arguments
javaCmd.addAll(ManagementFactory.getRuntimeMXBean().getInputArguments());
// Remove JNI options as they won't be supported by command-line 'java'
// These are typically added by the AdvancedInstaller Java launcher EXE
javaCmd.removeAll(Arrays.asList("abort", "exit", "vfprintf"));
// Call ApplyUpdate using new JAR
javaCmd.addAll(Arrays.asList("-cp", NEW_JAR_FILENAME, ApplyUpdate.class.getCanonicalName()));
@@ -241,7 +246,9 @@ public class AutoUpdate extends Thread {
LOGGER.info(String.format("Applying update with: %s", String.join(" ", javaCmd)));
SysTray.getInstance().showMessage("Auto Update", "Applying automatic update and restarting...", MessageType.INFO);
SysTray.getInstance().showMessage(Translator.INSTANCE.translate("SysTray", "AUTO_UPDATE"),
Translator.INSTANCE.translate("SysTray", "APPLYING_UPDATE_AND_RESTARTING"),
MessageType.INFO);
new ProcessBuilder(javaCmd).start();

File diff suppressed because it is too large Load Diff

View File

@@ -3,7 +3,7 @@ package org.qortal.controller;
import java.math.BigInteger;
import java.text.DecimalFormat;
import java.text.NumberFormat;
import java.util.Collections;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.locks.ReentrantLock;
import java.util.stream.Collectors;
@@ -39,13 +39,13 @@ public class Synchronizer {
private static final int INITIAL_BLOCK_STEP = 8;
private static final int MAXIMUM_BLOCK_STEP = 500;
private static final int MAXIMUM_COMMON_DELTA = 1440; // XXX move to Settings?
private static final int MAXIMUM_COMMON_DELTA = 240; // XXX move to Settings?
private static final int SYNC_BATCH_SIZE = 200;
private static Synchronizer instance;
public enum SynchronizationResult {
OK, NOTHING_TO_DO, GENESIS_ONLY, NO_COMMON_BLOCK, TOO_DIVERGENT, NO_REPLY, INFERIOR_CHAIN, INVALID_DATA, NO_BLOCKCHAIN_LOCK, REPOSITORY_ISSUE;
OK, NOTHING_TO_DO, GENESIS_ONLY, NO_COMMON_BLOCK, TOO_DIVERGENT, NO_REPLY, INFERIOR_CHAIN, INVALID_DATA, NO_BLOCKCHAIN_LOCK, REPOSITORY_ISSUE, SHUTTING_DOWN;
}
// Constructors
@@ -93,15 +93,11 @@ public class Synchronizer {
peerHeight, Base58.encode(peersLastBlockSignature), peer.getChainTipData().getLastBlockTimestamp(),
ourInitialHeight, Base58.encode(ourLastBlockSignature), ourLatestBlockData.getTimestamp()));
List<BlockSummaryData> peerBlockSummaries = fetchSummariesFromCommonBlock(repository, peer, ourInitialHeight);
if (peerBlockSummaries == null) {
LOGGER.info(String.format("Error while trying to find common block with peer %s", peer));
return SynchronizationResult.NO_REPLY;
}
if (peerBlockSummaries.isEmpty()) {
LOGGER.info(String.format("Failure to find common block with peer %s", peer));
return SynchronizationResult.NO_COMMON_BLOCK;
}
List<BlockSummaryData> peerBlockSummaries = new ArrayList<>();
SynchronizationResult findCommonBlockResult = fetchSummariesFromCommonBlock(repository, peer, ourInitialHeight, force, peerBlockSummaries);
if (findCommonBlockResult != SynchronizationResult.OK)
// Logging performed by fetchSummariesFromCommonBlock() above
return findCommonBlockResult;
// First summary is common block
final BlockData commonBlockData = repository.getBlockRepository().fromSignature(peerBlockSummaries.get(0).getSignature());
@@ -129,13 +125,6 @@ public class Synchronizer {
return SynchronizationResult.NOTHING_TO_DO;
}
// If common block is too far behind us then we're on massively different forks so give up.
int minCommonHeight = ourInitialHeight - MAXIMUM_COMMON_DELTA;
if (!force && commonBlockHeight < minCommonHeight) {
LOGGER.info(String.format("Blockchain too divergent with peer %s", peer));
return SynchronizationResult.TOO_DIVERGENT;
}
// Unless we're doing a forced sync, we might need to compare blocks after common block
if (!force && ourInitialHeight > commonBlockHeight) {
// If our latest block is very old, we're very behind and should ditch our fork.
@@ -154,6 +143,9 @@ public class Synchronizer {
int peerBlockCount = peerHeight - commonBlockHeight;
while (peerBlockSummaries.size() < peerBlockCount) {
if (Controller.isStopping())
return SynchronizationResult.SHUTTING_DOWN;
int lastSummaryHeight = commonBlockHeight + peerBlockSummaries.size();
byte[] previousSignature;
if (peerBlockSummaries.isEmpty())
@@ -212,6 +204,9 @@ public class Synchronizer {
LOGGER.debug(String.format("Orphaning blocks back to common block height %d, sig %.8s", commonBlockHeight, commonBlockSig58));
while (ourHeight > commonBlockHeight) {
if (Controller.isStopping())
return SynchronizationResult.SHUTTING_DOWN;
BlockData blockData = repository.getBlockRepository().fromHeight(ourHeight);
Block block = new Block(repository, blockData);
block.orphan();
@@ -232,6 +227,9 @@ public class Synchronizer {
List<byte[]> peerBlockSignatures = peerBlockSummaries.stream().map(BlockSummaryData::getSignature).collect(Collectors.toList());
while (ourHeight < peerHeight && ourHeight < maxBatchHeight) {
if (Controller.isStopping())
return SynchronizationResult.SHUTTING_DOWN;
// Do we need more signatures?
if (peerBlockSignatures.isEmpty()) {
int numberRequested = maxBatchHeight - ourHeight;
@@ -320,45 +318,59 @@ public class Synchronizer {
* @throws DataException
* @throws InterruptedException
*/
private List<BlockSummaryData> fetchSummariesFromCommonBlock(Repository repository, Peer peer, int ourHeight) throws DataException, InterruptedException {
private SynchronizationResult fetchSummariesFromCommonBlock(Repository repository, Peer peer, int ourHeight, boolean force, List<BlockSummaryData> blockSummariesFromCommon) throws DataException, InterruptedException {
// Start by asking for a few recent block hashes as this will cover a majority of reorgs
// Failing that, back off exponentially
int step = INITIAL_BLOCK_STEP;
List<BlockSummaryData> blockSummaries = null;
int testHeight = Math.max(ourHeight - step, 1);
BlockData testBlockData = null;
List<BlockSummaryData> blockSummariesBatch = null;
while (testHeight >= 1) {
// Are we shutting down?
if (Controller.isStopping())
return SynchronizationResult.SHUTTING_DOWN;
// Fetch our block signature at this height
testBlockData = repository.getBlockRepository().fromHeight(testHeight);
if (testBlockData == null) {
// Not found? But we've locked the blockchain and height is below blockchain's tip!
LOGGER.error("Failed to get block at height lower than blockchain tip during synchronization?");
return null;
return SynchronizationResult.REPOSITORY_ISSUE;
}
// Ask for block signatures since test block's signature
byte[] testSignature = testBlockData.getSignature();
LOGGER.trace(String.format("Requesting %d summar%s after height %d", step, (step != 1 ? "ies": "y"), testHeight));
blockSummaries = this.getBlockSummaries(peer, testSignature, step);
blockSummariesBatch = this.getBlockSummaries(peer, testSignature, step);
if (blockSummaries == null)
if (blockSummariesBatch == null) {
// No response - give up this time
return null;
LOGGER.info(String.format("Error while trying to find common block with peer %s", peer));
return SynchronizationResult.NO_REPLY;
}
LOGGER.trace(String.format("Received %s summar%s", blockSummaries.size(), (blockSummaries.size() != 1 ? "ies" : "y")));
LOGGER.trace(String.format("Received %s summar%s", blockSummariesBatch.size(), (blockSummariesBatch.size() != 1 ? "ies" : "y")));
// Empty list means remote peer is unaware of test signature OR has no new blocks after test signature
if (!blockSummaries.isEmpty())
if (!blockSummariesBatch.isEmpty())
// We have entries so we have found a common block
break;
// No blocks after genesis block?
// We don't get called for a peer at genesis height so this means NO blocks in common
if (testHeight == 1)
return Collections.emptyList();
if (testHeight == 1) {
LOGGER.info(String.format("Failure to find common block with peer %s", peer));
return SynchronizationResult.NO_COMMON_BLOCK;
}
// If common block is too far behind us then we're on massively different forks so give up.
if (!force && testHeight < ourHeight - MAXIMUM_COMMON_DELTA) {
LOGGER.info(String.format("Blockchain too divergent with peer %s", peer));
return SynchronizationResult.TOO_DIVERGENT;
}
if (peer.getVersion() >= 2) {
step <<= 1;
@@ -373,20 +385,21 @@ public class Synchronizer {
// Prepend test block's summary as first block summary, as summaries returned are *after* test block
BlockSummaryData testBlockSummary = new BlockSummaryData(testBlockData);
blockSummaries.add(0, testBlockSummary);
blockSummariesFromCommon.add(0, testBlockSummary);
blockSummariesFromCommon.addAll(blockSummariesBatch);
// Trim summaries so that first summary is common block.
// Currently we work back from the end until we hit a block we also have.
// TODO: rewrite as modified binary search!
for (int i = blockSummaries.size() - 1; i > 0; --i) {
if (repository.getBlockRepository().exists(blockSummaries.get(i).getSignature())) {
for (int i = blockSummariesFromCommon.size() - 1; i > 0; --i) {
if (repository.getBlockRepository().exists(blockSummariesFromCommon.get(i).getSignature())) {
// Note: index i isn't cleared: List.subList is fromIndex inclusive to toIndex exclusive
blockSummaries.subList(0, i).clear();
blockSummariesFromCommon.subList(0, i).clear();
break;
}
}
return blockSummaries;
return SynchronizationResult.OK;
}
private List<BlockSummaryData> getBlockSummaries(Peer peer, byte[] parentSignature, int numberRequested) throws InterruptedException {

View File

@@ -2,10 +2,8 @@ package org.qortal.data.account;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlTransient;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.crypto.Crypto;
import io.swagger.v3.oas.annotations.media.Schema;
@@ -16,14 +14,17 @@ import io.swagger.v3.oas.annotations.media.Schema.AccessMode;
public class MintingAccountData {
// Properties
// Never actually displayed by API
@Schema(hidden = true)
@XmlTransient
protected byte[] privateKey;
// Not always present - used by API if not null
@XmlTransient
@Schema(hidden = true)
// Read-only by API, we never ask for it as input
@Schema(accessMode = AccessMode.READ_ONLY)
protected byte[] publicKey;
// Not always present - used by API if not null
protected String mintingAccount;
protected String recipientAccount;
protected String address;
@@ -34,17 +35,17 @@ public class MintingAccountData {
protected MintingAccountData() {
}
public MintingAccountData(byte[] privateKey) {
public MintingAccountData(byte[] privateKey, byte[] publicKey) {
this.privateKey = privateKey;
this.publicKey = PrivateKeyAccount.toPublicKey(privateKey);
this.publicKey = publicKey;
}
public MintingAccountData(byte[] privateKey, RewardShareData rewardShareData) {
this(privateKey);
public MintingAccountData(MintingAccountData srcMintingAccountData, RewardShareData rewardShareData) {
this(srcMintingAccountData.privateKey, srcMintingAccountData.publicKey);
if (rewardShareData != null) {
this.recipientAccount = rewardShareData.getRecipient();
this.mintingAccount = Crypto.toAddress(rewardShareData.getMinterPublicKey());
this.mintingAccount = rewardShareData.getMinter();
} else {
this.address = Crypto.toAddress(this.publicKey);
}
@@ -56,8 +57,6 @@ public class MintingAccountData {
return this.privateKey;
}
@XmlElement(name = "publicKey")
@Schema(accessMode = AccessMode.READ_ONLY)
public byte[] getPublicKey() {
return this.publicKey;
}

View File

@@ -5,8 +5,9 @@ import java.math.BigDecimal;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlTransient;
import org.qortal.crypto.Crypto;
import io.swagger.v3.oas.annotations.media.Schema;
// All properties to be converted to JSON via JAXB
@XmlAccessorType(XmlAccessType.FIELD)
@@ -14,6 +15,12 @@ public class RewardShareData {
// Properties
private byte[] minterPublicKey;
// "minter" is called "mintingAccount" instead
@XmlTransient
@Schema(hidden = true)
private String minter;
private String recipient;
private byte[] rewardSharePublicKey;
private BigDecimal sharePercent;
@@ -25,8 +32,9 @@ public class RewardShareData {
}
// Used when fetching from repository
public RewardShareData(byte[] minterPublicKey, String recipient, byte[] rewardSharePublicKey, BigDecimal sharePercent) {
public RewardShareData(byte[] minterPublicKey, String minter, String recipient, byte[] rewardSharePublicKey, BigDecimal sharePercent) {
this.minterPublicKey = minterPublicKey;
this.minter = minter;
this.recipient = recipient;
this.rewardSharePublicKey = rewardSharePublicKey;
this.sharePercent = sharePercent;
@@ -38,6 +46,10 @@ public class RewardShareData {
return this.minterPublicKey;
}
public String getMinter() {
return this.minter;
}
public String getRecipient() {
return this.recipient;
}
@@ -52,7 +64,7 @@ public class RewardShareData {
@XmlElement(name = "mintingAccount")
public String getMintingAccount() {
return Crypto.toAddress(this.minterPublicKey);
return this.minter;
}
}

View File

@@ -35,6 +35,9 @@ public class GroupData {
)
private int creationGroupId;
// We abuse this class for API purposes by adding this unrelated field. Not always present.
private Boolean isAdmin;
// Constructors
// necessary for JAX-RS serialization
@@ -140,4 +143,14 @@ public class GroupData {
return this.creationGroupId;
}
// This is for API call GET /groups/member/{address}
public Boolean isAdmin() {
return this.isAdmin;
}
public void setIsAdmin(boolean isAdmin) {
this.isAdmin = isAdmin;
}
}

View File

@@ -1,5 +1,7 @@
package org.qortal.data.network;
import java.util.Arrays;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlElement;
@@ -44,4 +46,36 @@ public class OnlineAccountData {
return new PublicKeyAccount(null, this.publicKey).getAddress();
}
// Comparison
@Override
public boolean equals(Object other) {
if (other == this)
return true;
if (!(other instanceof OnlineAccountData))
return false;
OnlineAccountData otherOnlineAccountData = (OnlineAccountData) other;
// Very quick comparison
if (otherOnlineAccountData.timestamp != this.timestamp)
return false;
// Signature more likely to be unique than public key
if (!Arrays.equals(otherOnlineAccountData.signature, this.signature))
return false;
if (!Arrays.equals(otherOnlineAccountData.publicKey, this.publicKey))
return false;
return true;
}
@Override
public int hashCode() {
// Pretty lazy implementation
return (int) this.timestamp;
}
}

View File

@@ -62,11 +62,10 @@ public enum Handshake {
// Is this ID already connected inbound or outbound?
Peer otherInboundPeer = Network.getInstance().getInboundPeerWithId(peerId);
Peer otherOutboundPeer = Network.getInstance().getOutboundHandshakedPeerWithId(peerId);
// Extra checks on inbound peers with known IDs, to prevent ID stealing
if (!peer.isOutbound() && otherInboundPeer != null) {
Peer otherOutboundPeer = Network.getInstance().getOutboundHandshakedPeerWithId(peerId);
if (otherOutboundPeer == null) {
// We already have an inbound peer with this ID, but no outgoing peer with which to request verification
LOGGER.trace(String.format("Discarding inbound peer %s with existing ID", peer));
@@ -86,6 +85,11 @@ public enum Handshake {
// Generate verification codes for later
peer.generateVerificationCodes();
}
} else if (peer.isOutbound() && otherOutboundPeer != null) {
// We already have an outbound connection to this peer?
LOGGER.info(String.format("We already have another outbound connection to peer %s - discarding", peer));
// Handshake failure - caller will deal with disconnect
return null;
} else {
// Set peer's ID
peer.setPeerId(peerId);
@@ -231,7 +235,7 @@ public enum Handshake {
private static void sendProof(Peer peer) {
if (peer.isOutbound()) {
// For outbound connections we need to generate real proof
new Proof(peer).start();
new Proof(peer).start(); // Calculate & send in a new thread to free up networking processing
} else {
// For incoming connections we only need to send a fake proof message as confirmation
Message proofMessage = new ProofMessage(peer.getConnectionTimestamp(), 0, 0);

View File

@@ -56,6 +56,7 @@ import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.settings.Settings;
import org.qortal.utils.ExecuteProduceConsume;
import org.qortal.utils.ExecuteProduceConsume.StatsSnapshot;
import org.qortal.utils.NTP;
// For managing peers
@@ -88,9 +89,14 @@ public class Network {
"node4.qortal.org",
"node5.qortal.org",
"node6.qortal.org",
"node7.qortal.org"
"node7.qortal.org",
"node8.qortal.org",
"node9.qortal.org",
"node10.qortal.org"
};
private static final long NETWORK_EPC_KEEPALIVE = 10L; // seconds
public static final int MAX_SIGNATURES_PER_REPLY = 500;
public static final int MAX_BLOCK_SUMMARIES_PER_REPLY = 500;
public static final int PEER_ID_LENGTH = 128;
@@ -138,9 +144,10 @@ public class Network {
mergePeersLock = new ReentrantLock();
// We'll use a cached thread pool, max 10 threads, but with more aggressive 10 second timeout.
ExecutorService networkExecutor = new ThreadPoolExecutor(1, 10,
10L, TimeUnit.SECONDS,
// We'll use a cached thread pool but with more aggressive timeout.
ExecutorService networkExecutor = new ThreadPoolExecutor(1,
Settings.getInstance().getMaxNetworkThreadPoolSize(),
NETWORK_EPC_KEEPALIVE, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>());
networkEPC = new NetworkProcessor(networkExecutor);
}
@@ -196,6 +203,10 @@ public class Network {
return this.maxMessageSize;
}
public StatsSnapshot getStatsSnapshot() {
return this.networkEPC.getStatsSnapshot();
}
// Peer lists
public List<Peer> getConnectedPeers() {
@@ -294,15 +305,17 @@ public class Network {
if (task != null)
return task;
task = maybeProducePeerPingTask();
final Long now = NTP.getTime();
task = maybeProducePeerPingTask(now);
if (task != null)
return task;
task = maybeProduceConnectPeerTask();
task = maybeProduceConnectPeerTask(now);
if (task != null)
return task;
task = maybeProduceBroadcastTask();
task = maybeProduceBroadcastTask(now);
if (task != null)
return task;
@@ -315,6 +328,65 @@ public class Network {
return null;
}
private Task maybeProducePeerMessageTask() {
for (Peer peer : getConnectedPeers()) {
Task peerTask = peer.getMessageTask();
if (peerTask != null)
return peerTask;
}
return null;
}
private Task maybeProducePeerPingTask(Long now) {
// Ask connected peers whether they need a ping
for (Peer peer : getConnectedPeers()) {
Task peerTask = peer.getPingTask(now);
if (peerTask != null)
return peerTask;
}
return null;
}
class PeerConnectTask implements ExecuteProduceConsume.Task {
private final Peer peer;
public PeerConnectTask(Peer peer) {
this.peer = peer;
}
@Override
public void perform() throws InterruptedException {
connectPeer(peer);
}
}
private Task maybeProduceConnectPeerTask(Long now) throws InterruptedException {
if (now == null || now < nextConnectTaskTimestamp)
return null;
if (getOutboundHandshakedPeers().size() >= minOutboundPeers)
return null;
nextConnectTaskTimestamp = now + 1000L;
Peer targetPeer = getConnectablePeer(now);
if (targetPeer == null)
return null;
// Create connection task
return new PeerConnectTask(targetPeer);
}
private Task maybeProduceBroadcastTask(Long now) {
if (now == null || now < nextBroadcastTimestamp)
return null;
nextBroadcastTimestamp = now + BROADCAST_INTERVAL;
return () -> Controller.getInstance().doNetworkBroadcast();
}
class ChannelTask implements ExecuteProduceConsume.Task {
private final SelectionKey selectionKey;
@@ -397,67 +469,6 @@ public class Network {
return new ChannelTask(nextSelectionKey);
}
private Task maybeProducePeerMessageTask() {
for (Peer peer : getConnectedPeers()) {
Task peerTask = peer.getMessageTask();
if (peerTask != null)
return peerTask;
}
return null;
}
private Task maybeProducePeerPingTask() {
// Ask connected peers whether they need a ping
for (Peer peer : getConnectedPeers()) {
Task peerTask = peer.getPingTask();
if (peerTask != null)
return peerTask;
}
return null;
}
class PeerConnectTask implements ExecuteProduceConsume.Task {
private final Peer peer;
public PeerConnectTask(Peer peer) {
this.peer = peer;
}
@Override
public void perform() throws InterruptedException {
connectPeer(peer);
}
}
private Task maybeProduceConnectPeerTask() throws InterruptedException {
if (getOutboundHandshakedPeers().size() >= minOutboundPeers)
return null;
final Long now = NTP.getTime();
if (now == null || now < nextConnectTaskTimestamp)
return null;
nextConnectTaskTimestamp = now + 1000L;
Peer targetPeer = getConnectablePeer();
if (targetPeer == null)
return null;
// Create connection task
return new PeerConnectTask(targetPeer);
}
private Task maybeProduceBroadcastTask() {
final Long now = NTP.getTime();
if (now == null || now < nextBroadcastTimestamp)
return null;
nextBroadcastTimestamp = now + BROADCAST_INTERVAL;
return () -> Controller.getInstance().doNetworkBroadcast();
}
}
private void acceptConnection(ServerSocketChannel serverSocketChannel) throws InterruptedException {
@@ -478,18 +489,20 @@ public class Network {
try {
if (now == null) {
LOGGER.debug(String.format("Connection discarded from peer %s due to lack of NTP sync", socketChannel.getRemoteAddress()));
LOGGER.debug(() -> String.format("Connection discarded from peer %s due to lack of NTP sync", PeerAddress.fromSocket(socketChannel.socket())));
socketChannel.close();
return;
}
synchronized (this.connectedPeers) {
if (connectedPeers.size() >= maxPeers) {
// We have enough peers
LOGGER.debug(String.format("Connection discarded from peer %s", socketChannel.getRemoteAddress()));
LOGGER.debug(() -> String.format("Connection discarded from peer %s", PeerAddress.fromSocket(socketChannel.socket())));
socketChannel.close();
return;
}
LOGGER.debug(String.format("Connection accepted from peer %s", socketChannel.getRemoteAddress()));
LOGGER.debug(() -> String.format("Connection accepted from peer %s", PeerAddress.fromSocket(socketChannel.socket())));
newPeer = new Peer(socketChannel);
this.connectedPeers.add(newPeer);
@@ -578,9 +591,27 @@ public class Network {
}
}
private Peer getConnectablePeer() throws InterruptedException {
final long now = NTP.getTime();
private final Predicate<PeerData> isSelfPeer = peerData -> {
PeerAddress peerAddress = peerData.getAddress();
return this.selfPeers.stream().anyMatch(selfPeer -> selfPeer.equals(peerAddress));
};
private final Predicate<PeerData> isConnectedPeer = peerData -> {
PeerAddress peerAddress = peerData.getAddress();
return this.connectedPeers.stream().anyMatch(peer -> peer.getPeerData().getAddress().equals(peerAddress));
};
private final Predicate<PeerData> isResolvedAsConnectedPeer = peerData -> {
try {
InetSocketAddress resolvedSocketAddress = peerData.getAddress().toSocketAddress();
return this.connectedPeers.stream().anyMatch(peer -> peer.getResolvedAddress().equals(resolvedSocketAddress));
} catch (UnknownHostException e) {
// Can't resolve - no point even trying to connect
return true;
}
};
private Peer getConnectablePeer(final Long now) throws InterruptedException {
// We can't block here so use tryRepository(). We don't NEED to connect a new peer.
try (final Repository repository = RepositoryManager.tryRepository()) {
if (repository == null)
@@ -596,36 +627,17 @@ public class Network {
peerData.getLastAttempted() > lastAttemptedThreshold);
// Don't consider peers that we know loop back to ourself
Predicate<PeerData> isSelfPeer = peerData -> {
PeerAddress peerAddress = peerData.getAddress();
return this.selfPeers.stream().anyMatch(selfPeer -> selfPeer.equals(peerAddress));
};
synchronized (this.selfPeers) {
peers.removeIf(isSelfPeer);
}
// Don't consider already connected peers (simple address match)
Predicate<PeerData> isConnectedPeer = peerData -> {
PeerAddress peerAddress = peerData.getAddress();
return this.connectedPeers.stream().anyMatch(peer -> peer.getPeerData().getAddress().equals(peerAddress));
};
synchronized (this.connectedPeers) {
peers.removeIf(isConnectedPeer);
}
// Don't consider already connected peers (resolved address match)
Predicate<PeerData> isResolvedAsConnectedPeer = peerData -> {
try {
InetSocketAddress resolvedSocketAddress = peerData.getAddress().toSocketAddress();
return this.connectedPeers.stream().anyMatch(peer -> peer.getResolvedAddress().equals(resolvedSocketAddress));
} catch (UnknownHostException e) {
// Can't resolve - no point even trying to connect
return true;
}
};
// XXX This might be too slow if we end up waiting a long time for hostnames to resolve via DNS
synchronized (this.connectedPeers) {
peers.removeIf(isResolvedAsConnectedPeer);
}
@@ -725,126 +737,43 @@ public class Network {
Handshake handshakeStatus = peer.getHandshakeStatus();
if (handshakeStatus != Handshake.COMPLETED) {
try {
// Still handshaking
LOGGER.trace(() -> String.format("Handshake status %s, message %s from peer %s", handshakeStatus.name(), (message != null ? message.getType().name() : "null"), peer));
// v1 nodes are keen on sending PINGs early. Send to back of queue so we'll process right after handshake
if (message != null && message.getType() == MessageType.PING) {
peer.queueMessage(message);
return;
}
// Check message type is as expected
if (handshakeStatus.expectedMessageType != null && message.getType() != handshakeStatus.expectedMessageType) {
LOGGER.debug(String.format("Unexpected %s message from %s, expected %s", message.getType().name(), peer, handshakeStatus.expectedMessageType));
peer.disconnect("unexpected message");
return;
}
Handshake newHandshakeStatus = handshakeStatus.onMessage(peer, message);
if (newHandshakeStatus == null) {
// Handshake failure
LOGGER.debug(String.format("Handshake failure with peer %s message %s", peer, message.getType().name()));
peer.disconnect("handshake failure");
return;
}
if (peer.isOutbound())
// If we made outbound connection then we need to act first
newHandshakeStatus.action(peer);
else
// We have inbound connection so we need to respond in kind with what we just received
handshakeStatus.action(peer);
peer.setHandshakeStatus(newHandshakeStatus);
if (newHandshakeStatus == Handshake.COMPLETED)
this.onHandshakeCompleted(peer);
return;
} finally {
peer.resetHandshakeMessagePending();
}
onHandshakingMessage(peer, message, handshakeStatus);
return;
}
// Should be non-handshaking messages from now on
// Ordered by message type value
switch (message.getType()) {
case PEER_VERIFY:
// Remote peer wants extra verification
possibleVerificationResponse(peer);
case GET_PEERS:
onGetPeersMessage(peer, message);
break;
case VERIFICATION_CODES:
VerificationCodesMessage verificationCodesMessage = (VerificationCodesMessage) message;
case PEERS:
onPeersMessage(peer, message);
break;
// Remote peer is sending the code it wants to receive back via our outbound connection to it
Peer ourUnverifiedPeer = Network.getInstance().getInboundPeerWithId(Network.getInstance().getOurPeerId());
ourUnverifiedPeer.setVerificationCodes(verificationCodesMessage.getVerificationCodeSent(), verificationCodesMessage.getVerificationCodeExpected());
possibleVerificationResponse(ourUnverifiedPeer);
case PING:
onPingMessage(peer, message);
break;
case VERSION:
case PEER_ID:
case PROOF:
LOGGER.debug(String.format("Unexpected handshaking message %s from peer %s", message.getType().name(), peer));
LOGGER.debug(() -> String.format("Unexpected handshaking message %s from peer %s", message.getType().name(), peer));
peer.disconnect("unexpected handshaking message");
return;
case PING:
PingMessage pingMessage = (PingMessage) message;
// Generate 'pong' using same ID
PingMessage pongMessage = new PingMessage();
pongMessage.setId(pingMessage.getId());
if (!peer.sendMessage(pongMessage))
peer.disconnect("failed to send ping reply");
break;
case PEERS:
PeersMessage peersMessage = (PeersMessage) message;
List<PeerAddress> peerAddresses = new ArrayList<>();
// v1 PEERS message doesn't support port numbers so we have to add default port
for (InetAddress peerAddress : peersMessage.getPeerAddresses())
// This is always IPv4 so we don't have to worry about bracketing IPv6.
peerAddresses.add(PeerAddress.fromString(peerAddress.getHostAddress()));
// Also add peer's details
peerAddresses.add(PeerAddress.fromString(peer.getPeerData().getAddress().getHost()));
mergePeers(peer.toString(), peerAddresses);
break;
case PEERS_V2:
PeersV2Message peersV2Message = (PeersV2Message) message;
List<PeerAddress> peerV2Addresses = peersV2Message.getPeerAddresses();
// First entry contains remote peer's listen port but empty address.
int peerPort = peerV2Addresses.get(0).getPort();
peerV2Addresses.remove(0);
// If inbound peer, use listen port and socket address to recreate first entry
if (!peer.isOutbound()) {
PeerAddress sendingPeerAddress = PeerAddress.fromString(peer.getPeerData().getAddress().getHost() + ":" + peerPort);
LOGGER.trace(() -> String.format("PEERS_V2 sending peer's listen address: %s", sendingPeerAddress.toString()));
peerV2Addresses.add(0, sendingPeerAddress);
}
mergePeers(peer.toString(), peerV2Addresses);
onPeersV2Message(peer, message);
break;
case GET_PEERS:
// Send our known peers
if (!peer.sendMessage(buildPeersMessage(peer)))
peer.disconnect("failed to send peers list");
case PEER_VERIFY:
onPeerVerifyMessage(peer, message);
break;
case VERIFICATION_CODES:
onVerificationCodesMessage(peer, message);
break;
default:
@@ -854,6 +783,116 @@ public class Network {
}
}
private void onHandshakingMessage(Peer peer, Message message, Handshake handshakeStatus) {
try {
// Still handshaking
LOGGER.trace(() -> String.format("Handshake status %s, message %s from peer %s", handshakeStatus.name(), (message != null ? message.getType().name() : "null"), peer));
// v1 nodes are keen on sending PINGs early. Send to back of queue so we'll process right after handshake
if (message != null && message.getType() == MessageType.PING) {
peer.queueMessage(message);
return;
}
// Check message type is as expected
if (handshakeStatus.expectedMessageType != null && message.getType() != handshakeStatus.expectedMessageType) {
LOGGER.debug(() -> String.format("Unexpected %s message from %s, expected %s", message.getType().name(), peer, handshakeStatus.expectedMessageType));
peer.disconnect("unexpected message");
return;
}
Handshake newHandshakeStatus = handshakeStatus.onMessage(peer, message);
if (newHandshakeStatus == null) {
// Handshake failure
LOGGER.debug(() -> String.format("Handshake failure with peer %s message %s", peer, message.getType().name()));
peer.disconnect("handshake failure");
return;
}
if (peer.isOutbound())
// If we made outbound connection then we need to act first
newHandshakeStatus.action(peer);
else
// We have inbound connection so we need to respond in kind with what we just received
handshakeStatus.action(peer);
peer.setHandshakeStatus(newHandshakeStatus);
if (newHandshakeStatus == Handshake.COMPLETED)
this.onHandshakeCompleted(peer);
} finally {
peer.resetHandshakeMessagePending();
}
}
private void onGetPeersMessage(Peer peer, Message message) {
// Send our known peers
if (!peer.sendMessage(buildPeersMessage(peer)))
peer.disconnect("failed to send peers list");
}
private void onPeersMessage(Peer peer, Message message) {
PeersMessage peersMessage = (PeersMessage) message;
List<PeerAddress> peerAddresses = new ArrayList<>();
// v1 PEERS message doesn't support port numbers so we have to add default port
for (InetAddress peerAddress : peersMessage.getPeerAddresses())
// This is always IPv4 so we don't have to worry about bracketing IPv6.
peerAddresses.add(PeerAddress.fromString(peerAddress.getHostAddress()));
// Also add peer's details
peerAddresses.add(PeerAddress.fromString(peer.getPeerData().getAddress().getHost()));
mergePeers(peer.toString(), peerAddresses);
}
private void onPingMessage(Peer peer, Message message) {
PingMessage pingMessage = (PingMessage) message;
// Generate 'pong' using same ID
PingMessage pongMessage = new PingMessage();
pongMessage.setId(pingMessage.getId());
if (!peer.sendMessage(pongMessage))
peer.disconnect("failed to send ping reply");
}
private void onPeersV2Message(Peer peer, Message message) {
PeersV2Message peersV2Message = (PeersV2Message) message;
List<PeerAddress> peerV2Addresses = peersV2Message.getPeerAddresses();
// First entry contains remote peer's listen port but empty address.
int peerPort = peerV2Addresses.get(0).getPort();
peerV2Addresses.remove(0);
// If inbound peer, use listen port and socket address to recreate first entry
if (!peer.isOutbound()) {
PeerAddress sendingPeerAddress = PeerAddress.fromString(peer.getPeerData().getAddress().getHost() + ":" + peerPort);
LOGGER.trace(() -> String.format("PEERS_V2 sending peer's listen address: %s", sendingPeerAddress.toString()));
peerV2Addresses.add(0, sendingPeerAddress);
}
mergePeers(peer.toString(), peerV2Addresses);
}
private void onPeerVerifyMessage(Peer peer, Message message) {
// Remote peer wants extra verification
possibleVerificationResponse(peer);
}
private void onVerificationCodesMessage(Peer peer, Message message) {
VerificationCodesMessage verificationCodesMessage = (VerificationCodesMessage) message;
// Remote peer is sending the code it wants to receive back via our outbound connection to it
Peer ourUnverifiedPeer = Network.getInstance().getInboundPeerWithId(Network.getInstance().getOurPeerId());
ourUnverifiedPeer.setVerificationCodes(verificationCodesMessage.getVerificationCodeSent(), verificationCodesMessage.getVerificationCodeExpected());
possibleVerificationResponse(ourUnverifiedPeer);
}
private void possibleVerificationResponse(Peer peer) {
// Can't respond if we don't have the codes (yet?)
if (peer.getVerificationCodeExpected() == null)

View File

@@ -43,7 +43,7 @@ public class Peer {
private static final int CONNECT_TIMEOUT = 1000; // ms
/** Maximum time to wait for a message reply to arrive from peer. (ms) */
private static final int RESPONSE_TIMEOUT = 5000; // ms
private static final int RESPONSE_TIMEOUT = 2000; // ms
/**
* Interval between PING messages to a peer. (ms)
@@ -61,6 +61,8 @@ public class Peer {
private InetSocketAddress resolvedAddress = null;
/** True if remote address is loopback/link-local/site-local, false otherwise. */
private boolean isLocal;
private final Object byteBufferLock = new Object();
private volatile ByteBuffer byteBuffer;
private Map<Integer, BlockingQueue<Message>> replyQueues;
private LinkedBlockingQueue<Message> pendingMessages;
@@ -256,7 +258,7 @@ public class Peer {
this.connectionTimestamp = NTP.getTime();
this.socketChannel.setOption(StandardSocketOptions.TCP_NODELAY, true);
this.socketChannel.configureBlocking(false);
this.byteBuffer = ByteBuffer.allocate(Network.getInstance().getMaxMessageSize());
this.byteBuffer = null; // Defer allocation to when we need it, to save memory. Sorry GC!
this.replyQueues = Collections.synchronizedMap(new HashMap<Integer, BlockingQueue<Message>>());
this.pendingMessages = new LinkedBlockingQueue<>();
}
@@ -292,11 +294,15 @@ public class Peer {
* @throws IOException
*/
/* package */ void readChannel() throws IOException {
synchronized (this.byteBuffer) {
synchronized (this.byteBufferLock) {
while(true) {
if (!this.socketChannel.isOpen() || this.socketChannel.socket().isClosed())
return;
// Do we need to allocate byteBuffer?
if (this.byteBuffer == null)
this.byteBuffer = ByteBuffer.allocate(Network.getInstance().getMaxMessageSize());
final int bytesRead = this.socketChannel.read(this.byteBuffer);
if (bytesRead == -1) {
this.disconnect("EOF");
@@ -318,9 +324,15 @@ public class Peer {
return;
}
if (message == null && bytesRead == 0 && !wasByteBufferFull)
if (message == null && bytesRead == 0 && !wasByteBufferFull) {
// No complete message in buffer, no more bytes to read from socket even though there was room to read bytes
// If byteBuffer is empty then we can deallocate it, to save memory, albeit costing GC
if (this.byteBuffer.remaining() == this.byteBuffer.capacity())
this.byteBuffer = null;
return;
}
if (message == null)
// No complete message in buffer, but maybe more bytes to read from socket
@@ -452,18 +464,16 @@ public class Peer {
}
/* package */ void startPings() {
// Replacing initial null value allows pingCheck() to start sending pings.
// Replacing initial null value allows getPingTask() to start sending pings.
LOGGER.trace(() -> String.format("Enabling pings for peer %s", this));
this.lastPingSent = System.currentTimeMillis();
this.lastPingSent = NTP.getTime();
}
/* package */ ExecuteProduceConsume.Task getPingTask() {
/* package */ ExecuteProduceConsume.Task getPingTask(Long now) {
// Pings not enabled yet?
if (this.lastPingSent == null)
if (now == null || this.lastPingSent == null)
return null;
final long now = System.currentTimeMillis();
// Time to send another ping?
if (now < this.lastPingSent + PING_INTERVAL)
return null; // Not yet
@@ -474,14 +484,14 @@ public class Peer {
return () -> {
PingMessage pingMessage = new PingMessage();
Message message = this.getResponse(pingMessage);
final long after = System.currentTimeMillis();
if (message == null || message.getType() != MessageType.PING) {
LOGGER.debug(() -> String.format("Didn't receive reply from %s for PING ID %d", this, pingMessage.getId()));
this.disconnect("no ping received");
return;
}
this.setLastPing(after - now);
this.setLastPing(NTP.getTime() - now);
};
}

View File

@@ -5,6 +5,8 @@ import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.util.HashSet;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.network.message.ProofMessage;
import com.google.common.primitives.Longs;
@@ -13,6 +15,7 @@ public class Proof extends Thread {
private static final int MIN_PROOF_ZEROS = 2;
private static final HashSet<Long> seenSalts = new HashSet<>();
private static final Logger LOGGER = LogManager.getLogger(Proof.class);
private Peer peer;
@@ -38,6 +41,7 @@ public class Proof extends Thread {
setName("Proof for peer " + this.peer);
// Do proof-of-work calculation to gain acceptance with remote end
final long startTime = LOGGER.isTraceEnabled() ? System.currentTimeMillis() : 0;
// Remote end knows this (approximately)
long timestamp = this.peer.getConnectionTimestamp();
@@ -64,7 +68,7 @@ public class Proof extends Thread {
long nonce;
for (nonce = 0; nonce < Long.MAX_VALUE; ++nonce) {
// Check whether we're shutting down every so often
if ((nonce & 0xff) == 0 && (peer.isStopping() || Thread.currentThread().isInterrupted()))
if ((nonce & 0xff) == 0 && (this.peer.isStopping() || Thread.currentThread().isInterrupted()))
// throw new InterruptedException("Interrupted during peer proof calculation");
return;
@@ -79,6 +83,8 @@ public class Proof extends Thread {
sha256.reset();
}
LOGGER.trace(() -> String.format("Proof for peer %s took %dms", this.peer, System.currentTimeMillis() - startTime));
ProofMessage proofMessage = new ProofMessage(timestamp, salt, nonce);
peer.sendMessage(proofMessage);
}

View File

@@ -1,5 +1,6 @@
package org.qortal.repository;
import java.math.BigDecimal;
import java.util.List;
import org.qortal.data.account.AccountBalanceData;
@@ -82,6 +83,12 @@ public interface AccountRepository {
*/
public void setMintedBlockCount(AccountData accountData) throws DataException;
/** Modifies account's minted block count only.
* <p>
* @return 2 if minted block count updated, 1 if block count set to delta, 0 if address not found.
*/
public int modifyMintedBlockCount(String address, int delta) throws DataException;
/** Delete account from repository. */
public void delete(String address) throws DataException;
@@ -89,12 +96,6 @@ public interface AccountRepository {
public AccountBalanceData getBalance(String address, long assetId) throws DataException;
/** Returns account balance data for address & assetId at (or before) passed block height. */
public AccountBalanceData getBalance(String address, long assetId, int height) throws DataException;
/** Returns per-height historic balance for address & assetId. */
public List<AccountBalanceData> getHistoricBalances(String address, long assetId) throws DataException;
public enum BalanceOrdering {
ASSET_BALANCE_ACCOUNT,
ACCOUNT_ASSET,
@@ -105,13 +106,12 @@ public interface AccountRepository {
public List<AccountBalanceData> getAssetBalances(List<String> addresses, List<Long> assetIds, BalanceOrdering balanceOrdering, Boolean excludeZero, Integer limit, Integer offset, Boolean reverse) throws DataException;
public void modifyAssetBalance(String address, long assetId, BigDecimal deltaBalance) throws DataException;
public void save(AccountBalanceData accountBalanceData) throws DataException;
public void delete(String address, long assetId) throws DataException;
/** Deletes orphaned balances at block height >= <tt>height</tt>. */
public int deleteBalancesFromHeight(int height) throws DataException;
// Reward-shares
public RewardShareData getRewardShare(byte[] mintingAccountPublicKey, String recipientAccount) throws DataException;
@@ -155,6 +155,21 @@ public interface AccountRepository {
// Managing QORT from legacy QORA
/**
* Returns balance data for accounts with legacy QORA asset that are eligible
* for more block reward (block processing) or for block reward removal (block orphaning).
* <p>
* For block processing, accounts that have already received their final QORT reward for owning
* legacy QORA are omitted from the results. <tt>blockHeight</tt> should be <tt>null</tt>.
* <p>
* For block orphaning, accounts that did not receive a QORT reward at <tt>blockHeight</tt>
* are omitted from the results.
*
* @param blockHeight QORT reward must have be present at this height (for orphaning only)
* @throws DataException
*/
public List<AccountBalanceData> getEligibleLegacyQoraHolders(Integer blockHeight) throws DataException;
public QortFromQoraData getQortFromQoraInfo(String address) throws DataException;
public void save(QortFromQoraData qortFromQoraData) throws DataException;

View File

@@ -4,9 +4,9 @@ import java.math.BigDecimal;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import org.qortal.asset.Asset;
import org.qortal.data.account.AccountBalanceData;
import org.qortal.data.account.AccountData;
import org.qortal.data.account.MintingAccountData;
@@ -144,6 +144,10 @@ public class HSQLDBAccountRepository implements AccountRepository {
@Override
public void ensureAccount(AccountData accountData) throws DataException {
/*
* Why do we need to check/set the public_key?
* Is there something that sets an account's balance which also needs to set the public key?
byte[] publicKey = accountData.getPublicKey();
String sql = "SELECT public_key FROM Accounts WHERE account = ?";
@@ -168,6 +172,15 @@ public class HSQLDBAccountRepository implements AccountRepository {
} catch (SQLException e) {
throw new DataException("Unable to ensure minimal account in repository", e);
}
*/
String sql = "INSERT IGNORE INTO Accounts (account) VALUES (?)"; // MySQL syntax
try {
this.repository.checkedExecuteUpdateCount(sql, accountData.getAddress());
} catch (SQLException e) {
throw new DataException("Unable to ensure minimal account in repository", e);
}
}
@Override
@@ -273,6 +286,18 @@ public class HSQLDBAccountRepository implements AccountRepository {
}
}
@Override
public int modifyMintedBlockCount(String address, int delta) throws DataException {
String sql = "INSERT INTO Accounts (account, blocks_minted) VALUES (?, ?) " +
"ON DUPLICATE KEY UPDATE blocks_minted = blocks_minted + ?";
try {
return this.repository.checkedExecuteUpdateCount(sql, address, delta, delta);
} catch (SQLException e) {
throw new DataException("Unable to modify account's minted block count in repository", e);
}
}
@Override
public void delete(String address) throws DataException {
// NOTE: Account balances are deleted automatically by the database thanks to "ON DELETE CASCADE" in AccountBalances' FOREIGN KEY
@@ -302,44 +327,6 @@ public class HSQLDBAccountRepository implements AccountRepository {
}
}
@Override
public AccountBalanceData getBalance(String address, long assetId, int height) throws DataException {
String sql = "SELECT IFNULL(balance, 0) FROM HistoricAccountBalances WHERE account = ? AND asset_id = ? AND height <= ? ORDER BY height DESC LIMIT 1";
try (ResultSet resultSet = this.repository.checkedExecute(sql, address, assetId, height)) {
if (resultSet == null)
return null;
BigDecimal balance = resultSet.getBigDecimal(1).setScale(8);
return new AccountBalanceData(address, assetId, balance);
} catch (SQLException e) {
throw new DataException("Unable to fetch account balance from repository", e);
}
}
@Override
public List<AccountBalanceData> getHistoricBalances(String address, long assetId) throws DataException {
String sql = "SELECT height, balance FROM HistoricAccountBalances WHERE account = ? AND asset_id = ? ORDER BY height DESC";
List<AccountBalanceData> historicBalances = new ArrayList<>();
try (ResultSet resultSet = this.repository.checkedExecute(sql, address, assetId)) {
if (resultSet == null)
return historicBalances;
do {
int height = resultSet.getInt(1);
BigDecimal balance = resultSet.getBigDecimal(2);
historicBalances.add(new AccountBalanceData(address, assetId, balance, height));
} while (resultSet.next());
return historicBalances;
} catch (SQLException e) {
throw new DataException("Unable to fetch historic account balances from repository", e);
}
}
@Override
public List<AccountBalanceData> getAssetBalances(long assetId, Boolean excludeZero) throws DataException {
StringBuilder sql = new StringBuilder(1024);
@@ -471,44 +458,48 @@ public class HSQLDBAccountRepository implements AccountRepository {
}
@Override
public void save(AccountBalanceData accountBalanceData) throws DataException {
// If balance is zero and there are no prior historic balance, then simply delete balances for this assetId (typically during orphaning)
if (accountBalanceData.getBalance().signum() == 0) {
String existsSql = "account = ? AND asset_id = ? AND height < (SELECT height - 1 FROM NextBlockHeight)"; // height prior to current block. no matches (obviously) prior to genesis block
public void modifyAssetBalance(String address, long assetId, BigDecimal deltaBalance) throws DataException {
// If deltaBalance is zero then do nothing
if (deltaBalance.signum() == 0)
return;
boolean hasPriorBalances;
// If deltaBalance is negative then we assume AccountBalances & parent Accounts rows exist
if (deltaBalance.signum() < 0) {
// Perform actual balance change
String sql = "UPDATE AccountBalances set balance = balance + ? WHERE account = ? AND asset_id = ?";
try {
hasPriorBalances = this.repository.exists("HistoricAccountBalances", existsSql, accountBalanceData.getAddress(), accountBalanceData.getAssetId());
this.repository.checkedExecuteUpdateCount(sql, deltaBalance, address, assetId);
} catch (SQLException e) {
throw new DataException("Unable to check for historic account balances in repository", e);
throw new DataException("Unable to reduce account balance in repository", e);
}
} else {
// We have to ensure parent row exists to satisfy foreign key constraint
try {
String sql = "INSERT IGNORE INTO Accounts (account) VALUES (?)"; // MySQL syntax
this.repository.checkedExecuteUpdateCount(sql, address);
} catch (SQLException e) {
throw new DataException("Unable to ensure minimal account in repository", e);
}
if (!hasPriorBalances) {
try {
this.repository.delete("AccountBalances", "account = ? AND asset_id = ?", accountBalanceData.getAddress(), accountBalanceData.getAssetId());
} catch (SQLException e) {
throw new DataException("Unable to delete account balance from repository", e);
}
// I don't think we need to do this as Block.orphan() would do this for us?
try {
this.repository.delete("HistoricAccountBalances", "account = ? AND asset_id = ?", accountBalanceData.getAddress(), accountBalanceData.getAssetId());
} catch (SQLException e) {
throw new DataException("Unable to delete historic account balances from repository", e);
}
return;
// Perform actual balance change
String sql = "INSERT INTO AccountBalances (account, asset_id, balance) VALUES (?, ?, ?) " +
"ON DUPLICATE KEY UPDATE balance = balance + ?";
try {
this.repository.checkedExecuteUpdateCount(sql, address, assetId, deltaBalance, deltaBalance);
} catch (SQLException e) {
throw new DataException("Unable to increase account balance in repository", e);
}
}
}
@Override
public void save(AccountBalanceData accountBalanceData) throws DataException {
HSQLDBSaver saveHelper = new HSQLDBSaver("AccountBalances");
saveHelper.bind("account", accountBalanceData.getAddress()).bind("asset_id", accountBalanceData.getAssetId()).bind("balance",
accountBalanceData.getBalance());
try {
// HistoricAccountBalances auto-updated via trigger
saveHelper.execute(this.repository);
} catch (SQLException e) {
throw new DataException("Unable to save account balance into repository", e);
@@ -522,37 +513,23 @@ public class HSQLDBAccountRepository implements AccountRepository {
} catch (SQLException e) {
throw new DataException("Unable to delete account balance from repository", e);
}
try {
this.repository.delete("HistoricAccountBalances", "account = ? AND asset_id = ?", address, assetId);
} catch (SQLException e) {
throw new DataException("Unable to delete historic account balances from repository", e);
}
}
@Override
public int deleteBalancesFromHeight(int height) throws DataException {
try {
return this.repository.delete("HistoricAccountBalances", "height >= ?", height);
} catch (SQLException e) {
throw new DataException("Unable to delete historic account balances from repository", e);
}
}
// Reward-Share
@Override
public RewardShareData getRewardShare(byte[] minterPublicKey, String recipient) throws DataException {
String sql = "SELECT reward_share_public_key, share_percent FROM RewardShares WHERE minter_public_key = ? AND recipient = ?";
String sql = "SELECT minter, reward_share_public_key, share_percent FROM RewardShares WHERE minter_public_key = ? AND recipient = ?";
try (ResultSet resultSet = this.repository.checkedExecute(sql, minterPublicKey, recipient)) {
if (resultSet == null)
return null;
byte[] rewardSharePublicKey = resultSet.getBytes(1);
BigDecimal sharePercent = resultSet.getBigDecimal(2);
String minter = resultSet.getString(1);
byte[] rewardSharePublicKey = resultSet.getBytes(2);
BigDecimal sharePercent = resultSet.getBigDecimal(3);
return new RewardShareData(minterPublicKey, recipient, rewardSharePublicKey, sharePercent);
return new RewardShareData(minterPublicKey, minter, recipient, rewardSharePublicKey, sharePercent);
} catch (SQLException e) {
throw new DataException("Unable to fetch reward-share info from repository", e);
}
@@ -560,17 +537,18 @@ public class HSQLDBAccountRepository implements AccountRepository {
@Override
public RewardShareData getRewardShare(byte[] rewardSharePublicKey) throws DataException {
String sql = "SELECT minter_public_key, recipient, share_percent FROM RewardShares WHERE reward_share_public_key = ?";
String sql = "SELECT minter_public_key, minter, recipient, share_percent FROM RewardShares WHERE reward_share_public_key = ?";
try (ResultSet resultSet = this.repository.checkedExecute(sql, rewardSharePublicKey)) {
if (resultSet == null)
return null;
byte[] minterPublicKey = resultSet.getBytes(1);
String recipient = resultSet.getString(2);
BigDecimal sharePercent = resultSet.getBigDecimal(3);
String minter = resultSet.getString(2);
String recipient = resultSet.getString(3);
BigDecimal sharePercent = resultSet.getBigDecimal(4);
return new RewardShareData(minterPublicKey, recipient, rewardSharePublicKey, sharePercent);
return new RewardShareData(minterPublicKey, minter, recipient, rewardSharePublicKey, sharePercent);
} catch (SQLException e) {
throw new DataException("Unable to fetch reward-share info from repository", e);
}
@@ -598,7 +576,7 @@ public class HSQLDBAccountRepository implements AccountRepository {
@Override
public List<RewardShareData> getRewardShares() throws DataException {
String sql = "SELECT minter_public_key, recipient, share_percent, reward_share_public_key FROM RewardShares";
String sql = "SELECT minter_public_key, minter, recipient, share_percent, reward_share_public_key FROM RewardShares";
List<RewardShareData> rewardShares = new ArrayList<>();
@@ -608,11 +586,12 @@ public class HSQLDBAccountRepository implements AccountRepository {
do {
byte[] minterPublicKey = resultSet.getBytes(1);
String recipient = resultSet.getString(2);
BigDecimal sharePercent = resultSet.getBigDecimal(3);
byte[] rewardSharePublicKey = resultSet.getBytes(4);
String minter = resultSet.getString(2);
String recipient = resultSet.getString(3);
BigDecimal sharePercent = resultSet.getBigDecimal(4);
byte[] rewardSharePublicKey = resultSet.getBytes(5);
rewardShares.add(new RewardShareData(minterPublicKey, recipient, rewardSharePublicKey, sharePercent));
rewardShares.add(new RewardShareData(minterPublicKey, minter, recipient, rewardSharePublicKey, sharePercent));
} while (resultSet.next());
return rewardShares;
@@ -625,7 +604,7 @@ public class HSQLDBAccountRepository implements AccountRepository {
public List<RewardShareData> findRewardShares(List<String> minters, List<String> recipients, List<String> involvedAddresses,
Integer limit, Integer offset, Boolean reverse) throws DataException {
StringBuilder sql = new StringBuilder(1024);
sql.append("SELECT DISTINCT minter_public_key, recipient, share_percent, reward_share_public_key FROM RewardShares ");
sql.append("SELECT DISTINCT minter_public_key, minter, recipient, share_percent, reward_share_public_key FROM RewardShares ");
List<Object> args = new ArrayList<>();
@@ -695,11 +674,12 @@ public class HSQLDBAccountRepository implements AccountRepository {
do {
byte[] minterPublicKey = resultSet.getBytes(1);
String recipient = resultSet.getString(2);
BigDecimal sharePercent = resultSet.getBigDecimal(3);
byte[] rewardSharePublicKey = resultSet.getBytes(4);
String minter = resultSet.getString(2);
String recipient = resultSet.getString(3);
BigDecimal sharePercent = resultSet.getBigDecimal(4);
byte[] rewardSharePublicKey = resultSet.getBytes(5);
rewardShares.add(new RewardShareData(minterPublicKey, recipient, rewardSharePublicKey, sharePercent));
rewardShares.add(new RewardShareData(minterPublicKey, minter, recipient, rewardSharePublicKey, sharePercent));
} while (resultSet.next());
return rewardShares;
@@ -724,7 +704,7 @@ public class HSQLDBAccountRepository implements AccountRepository {
@Override
public RewardShareData getRewardShareByIndex(int index) throws DataException {
String sql = "SELECT minter_public_key, recipient, share_percent, reward_share_public_key FROM RewardShares "
String sql = "SELECT minter_public_key, minter, recipient, share_percent, reward_share_public_key FROM RewardShares "
+ "ORDER BY reward_share_public_key ASC "
+ "OFFSET ? LIMIT 1";
@@ -733,11 +713,12 @@ public class HSQLDBAccountRepository implements AccountRepository {
return null;
byte[] minterPublicKey = resultSet.getBytes(1);
String recipient = resultSet.getString(2);
BigDecimal sharePercent = resultSet.getBigDecimal(3);
byte[] rewardSharePublicKey = resultSet.getBytes(4);
String minter = resultSet.getString(2);
String recipient = resultSet.getString(3);
BigDecimal sharePercent = resultSet.getBigDecimal(4);
byte[] rewardSharePublicKey = resultSet.getBytes(5);
return new RewardShareData(minterPublicKey, recipient, rewardSharePublicKey, sharePercent);
return new RewardShareData(minterPublicKey, minter, recipient, rewardSharePublicKey, sharePercent);
} catch (SQLException e) {
throw new DataException("Unable to fetch reward-share info from repository", e);
}
@@ -747,8 +728,9 @@ public class HSQLDBAccountRepository implements AccountRepository {
public void save(RewardShareData rewardShareData) throws DataException {
HSQLDBSaver saveHelper = new HSQLDBSaver("RewardShares");
saveHelper.bind("minter_public_key", rewardShareData.getMinterPublicKey()).bind("recipient", rewardShareData.getRecipient())
.bind("reward_share_public_key", rewardShareData.getRewardSharePublicKey()).bind("share_percent", rewardShareData.getSharePercent());
saveHelper.bind("minter_public_key", rewardShareData.getMinterPublicKey()).bind("minter", rewardShareData.getMinter())
.bind("recipient", rewardShareData.getRecipient()).bind("reward_share_public_key", rewardShareData.getRewardSharePublicKey())
.bind("share_percent", rewardShareData.getSharePercent());
try {
saveHelper.execute(this.repository);
@@ -768,17 +750,19 @@ public class HSQLDBAccountRepository implements AccountRepository {
// Minting accounts used by BlockMinter
@Override
public List<MintingAccountData> getMintingAccounts() throws DataException {
List<MintingAccountData> mintingAccounts = new ArrayList<>();
try (ResultSet resultSet = this.repository.checkedExecute("SELECT minter_private_key FROM MintingAccounts")) {
try (ResultSet resultSet = this.repository.checkedExecute("SELECT minter_private_key, minter_public_key FROM MintingAccounts")) {
if (resultSet == null)
return mintingAccounts;
do {
byte[] minterPrivateKey = resultSet.getBytes(1);
byte[] minterPublicKey = resultSet.getBytes(2);
mintingAccounts.add(new MintingAccountData(minterPrivateKey));
mintingAccounts.add(new MintingAccountData(minterPrivateKey, minterPublicKey));
} while (resultSet.next());
return mintingAccounts;
@@ -787,10 +771,12 @@ public class HSQLDBAccountRepository implements AccountRepository {
}
}
@Override
public void save(MintingAccountData mintingAccountData) throws DataException {
HSQLDBSaver saveHelper = new HSQLDBSaver("MintingAccounts");
saveHelper.bind("minter_private_key", mintingAccountData.getPrivateKey());
saveHelper.bind("minter_private_key", mintingAccountData.getPrivateKey())
.bind("minter_public_key", mintingAccountData.getPublicKey());
try {
saveHelper.execute(this.repository);
@@ -799,6 +785,7 @@ public class HSQLDBAccountRepository implements AccountRepository {
}
}
@Override
public int delete(byte[] minterPrivateKey) throws DataException {
try {
return this.repository.delete("MintingAccounts", "minter_private_key = ?", minterPrivateKey);
@@ -809,6 +796,42 @@ public class HSQLDBAccountRepository implements AccountRepository {
// Managing QORT from legacy QORA
@Override
public List<AccountBalanceData> getEligibleLegacyQoraHolders(Integer blockHeight) throws DataException {
StringBuilder sql = new StringBuilder(1024);
sql.append("SELECT account, balance from AccountBalances ");
sql.append("LEFT OUTER JOIN AccountQortFromQoraInfo USING (account) ");
sql.append("WHERE asset_id = ");
sql.append(Asset.LEGACY_QORA); // int is safe to use literally
sql.append(" AND (final_block_height IS NULL");
if (blockHeight != null) {
sql.append(" OR final_block_height >= ");
sql.append(blockHeight);
}
sql.append(")");
List<AccountBalanceData> accountBalances = new ArrayList<>();
try (ResultSet resultSet = this.repository.checkedExecute(sql.toString())) {
if (resultSet == null)
return accountBalances;
do {
String address = resultSet.getString(1);
BigDecimal balance = resultSet.getBigDecimal(2).setScale(8);
accountBalances.add(new AccountBalanceData(address, Asset.LEGACY_QORA, balance));
} while (resultSet.next());
return accountBalances;
} catch (SQLException e) {
throw new DataException("Unable to fetch eligible legacy QORA holders from repository", e);
}
}
@Override
public QortFromQoraData getQortFromQoraInfo(String address) throws DataException {
String sql = "SELECT final_qort_from_qora, final_block_height FROM AccountQortFromQoraInfo WHERE account = ?";
@@ -827,6 +850,7 @@ public class HSQLDBAccountRepository implements AccountRepository {
}
}
@Override
public void save(QortFromQoraData qortFromQoraData) throws DataException {
HSQLDBSaver saveHelper = new HSQLDBSaver("AccountQortFromQoraInfo");
@@ -841,6 +865,7 @@ public class HSQLDBAccountRepository implements AccountRepository {
}
}
@Override
public int deleteQortFromQoraInfo(String address) throws DataException {
try {
return this.repository.delete("AccountQortFromQoraInfo", "account = ?", address);

View File

@@ -201,7 +201,7 @@ public class HSQLDBBlockRepository implements BlockRepository {
String subquerySql = "SELECT minter, COUNT(signature) FROM Blocks GROUP BY minter";
StringBuilder sql = new StringBuilder(1024);
sql.append("SELECT DISTINCT block_minter, n_blocks, minter_public_key, recipient FROM (");
sql.append("SELECT DISTINCT block_minter, n_blocks, minter_public_key, minter, recipient FROM (");
sql.append(subquerySql);
sql.append(") AS Minters (block_minter, n_blocks) LEFT OUTER JOIN RewardShares ON reward_share_public_key = block_minter ");
@@ -239,14 +239,17 @@ public class HSQLDBBlockRepository implements BlockRepository {
do {
byte[] blockMinterPublicKey = resultSet.getBytes(1);
int nBlocks = resultSet.getInt(2);
// May not be present if no reward-share:
byte[] mintingAccountPublicKey = resultSet.getBytes(3);
String recipientAccount = resultSet.getString(4);
String minterAccount = resultSet.getString(4);
String recipientAccount = resultSet.getString(5);
BlockMinterSummary blockMinterSummary;
if (recipientAccount == null)
blockMinterSummary = new BlockMinterSummary(blockMinterPublicKey, nBlocks);
else
blockMinterSummary = new BlockMinterSummary(blockMinterPublicKey, nBlocks, mintingAccountPublicKey, recipientAccount);
blockMinterSummary = new BlockMinterSummary(blockMinterPublicKey, nBlocks, mintingAccountPublicKey, minterAccount, recipientAccount);
summaries.add(blockMinterSummary);
} while (resultSet.next());
@@ -260,13 +263,13 @@ public class HSQLDBBlockRepository implements BlockRepository {
@Override
public List<BlockSummaryData> getBlockSummariesByMinter(byte[] minterPublicKey, Integer limit, Integer offset, Boolean reverse) throws DataException {
StringBuilder sql = new StringBuilder(512);
sql.append("SELECT signature, height, minter, online_accounts_count FROM ");
sql.append("SELECT signature, height, Blocks.minter, online_accounts_count FROM ");
// List of minter account's public key and reward-share public keys with minter's public key
sql.append("(SELECT * FROM (VALUES (CAST(? AS QortalPublicKey))) UNION (SELECT reward_share_public_key FROM RewardShares WHERE minter_public_key = ?)) AS PublicKeys (public_key) ");
// Match Blocks signed with public key from above list
sql.append("JOIN Blocks ON minter = public_key ");
sql.append("JOIN Blocks ON Blocks.minter = public_key ");
sql.append("ORDER BY Blocks.height ");
if (reverse != null && reverse)

View File

@@ -928,6 +928,53 @@ public class HSQLDBDatabaseUpdates {
stmt.execute("CREATE INDEX IF NOT EXISTS HistoricAccountBalancesHeightIndex ON HistoricAccountBalances (height)");
break;
case 66:
// Add CHECK constraint to account balances
stmt.execute("ALTER TABLE AccountBalances ADD CONSTRAINT CheckBalanceNotNegative CHECK (balance >= 0)");
break;
case 67:
// Provide external function to convert private keys to public keys
stmt.execute("CREATE FUNCTION Ed25519_private_to_public_key (IN privateKey VARBINARY(32)) RETURNS VARBINARY(32) LANGUAGE JAVA DETERMINISTIC NO SQL EXTERNAL NAME 'CLASSPATH:org.qortal.repository.hsqldb.HSQLDBRepository.ed25519PrivateToPublicKey'");
// Cache minting account public keys to save us recalculating them
stmt.execute("ALTER TABLE MintingAccounts ADD minter_public_key QortalPublicKey");
stmt.execute("UPDATE MintingAccounts SET minter_public_key = Ed25519_private_to_public_key(minter_private_key)");
stmt.execute("ALTER TABLE MintingAccounts ALTER COLUMN minter_public_key SET NOT NULL");
// Provide external function to convert public keys to addresses
stmt.execute("CREATE FUNCTION Ed25519_public_key_to_address (IN privateKey VARBINARY(32)) RETURNS VARCHAR(36) LANGUAGE JAVA DETERMINISTIC NO SQL EXTERNAL NAME 'CLASSPATH:org.qortal.repository.hsqldb.HSQLDBRepository.ed25519PublicKeyToAddress'");
// Cache reward-share minting account's address
stmt.execute("ALTER TABLE RewardShares ADD minter QortalAddress BEFORE recipient");
stmt.execute("UPDATE RewardShares SET minter = Ed25519_public_key_to_address(minter_public_key)");
stmt.execute("ALTER TABLE RewardShares ALTER COLUMN minter SET NOT NULL");
break;
case 68:
// Slow down log fsync() calls from every 500ms to reduce I/O load
stmt.execute("SET FILES WRITE DELAY 5"); // only fsync() every 5 seconds
break;
case 69:
// Get rid of historic account balances as they simply use up way too much space
stmt.execute("DROP TRIGGER Historic_Account_Balance_Insert_Trigger");
stmt.execute("DROP TRIGGER Historic_Account_Balance_Update_Trigger");
stmt.execute("DROP TABLE HistoricAccountBalances");
// Reclaim space
stmt.execute("CHECKPOINT");
stmt.execute("CHECKPOINT DEFRAG");
break;
case 70:
// Reduce space used for storing online account in Blocks
stmt.execute("ALTER TABLE Blocks ALTER COLUMN online_accounts BLOB(1M)");
stmt.execute("ALTER TABLE Blocks ALTER COLUMN online_accounts_signatures BLOB(1M)");
// Reclaim space
stmt.execute("CHECKPOINT");
stmt.execute("CHECKPOINT DEFRAG");
break;
default:
// nothing to do
return false;

View File

@@ -206,8 +206,10 @@ public class HSQLDBGroupRepository implements GroupRepository {
public List<GroupData> getGroupsWithMember(String member, Integer limit, Integer offset, Boolean reverse) throws DataException {
StringBuilder sql = new StringBuilder(512);
sql.append("SELECT group_id, owner, group_name, description, created, updated, reference, is_open, "
+ "approval_threshold, min_block_delay, max_block_delay, creation_group_id FROM Groups "
+ "JOIN GroupMembers USING (group_id) WHERE address = ? ORDER BY group_name");
+ "approval_threshold, min_block_delay, max_block_delay, creation_group_id, admin FROM Groups "
+ "JOIN GroupMembers USING (group_id) "
+ "LEFT OUTER JOIN GroupAdmins ON GroupAdmins.group_id = GroupMembers.group_id AND GroupAdmins.admin = GroupMembers.address "
+ "WHERE address = ? ORDER BY group_name");
if (reverse != null && reverse)
sql.append(" DESC");
@@ -239,8 +241,13 @@ public class HSQLDBGroupRepository implements GroupRepository {
int maxBlockDelay = resultSet.getInt(11);
int creationGroupId = resultSet.getInt(12);
resultSet.getString(13);
boolean isAdmin = !resultSet.wasNull();
groups.add(new GroupData(groupId, owner, groupName, description, created, updated, isOpen, approvalThreshold, minBlockDelay, maxBlockDelay, reference, creationGroupId));
GroupData groupData = new GroupData(groupId, owner, groupName, description, created, updated, isOpen, approvalThreshold, minBlockDelay, maxBlockDelay, reference, creationGroupId);
groupData.setIsAdmin(isAdmin);
groups.add(groupData);
} while (resultSet.next());
return groups;

View File

@@ -28,6 +28,8 @@ import java.util.regex.Pattern;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.crypto.Crypto;
import org.qortal.repository.ATRepository;
import org.qortal.repository.AccountRepository;
import org.qortal.repository.ArbitraryRepository;
@@ -57,6 +59,8 @@ public class HSQLDBRepository implements Repository {
protected List<String> sqlStatements;
protected long sessionId;
// Constructors
// NB: no visibility modifier so only callable from within same package
/* package */ HSQLDBRepository(Connection connection) throws DataException {
this.connection = connection;
@@ -84,6 +88,8 @@ public class HSQLDBRepository implements Repository {
assertEmptyTransaction("connection creation");
}
// Getters / setters
@Override
public ATRepository getATRepository() {
return new HSQLDBATRepository(this);
@@ -134,6 +140,18 @@ public class HSQLDBRepository implements Repository {
return new HSQLDBVotingRepository(this);
}
@Override
public boolean getDebug() {
return this.debugState;
}
@Override
public void setDebug(boolean debugState) {
this.debugState = debugState;
}
// Transaction COMMIT / ROLLBACK / savepoints
@Override
public void saveChanges() throws DataException {
try {
@@ -203,6 +221,8 @@ public class HSQLDBRepository implements Repository {
}
}
// Close / backup / rebuild / restore
@Override
public void close() throws DataException {
// Already closed? No need to do anything but maybe report double-call
@@ -257,16 +277,6 @@ public class HSQLDBRepository implements Repository {
}
}
@Override
public boolean getDebug() {
return this.debugState;
}
@Override
public void setDebug(boolean debugState) {
this.debugState = debugState;
}
@Override
public void backup(boolean quick) throws DataException {
if (!quick)
@@ -337,8 +347,8 @@ public class HSQLDBRepository implements Repository {
Path oldRepoFilePath = oldRepoPath.getFileName();
// Try to open backup. We need to remove "create=true" and insert "backup" dir before final filename.
String backupUrlTemplate = "jdbc:hsqldb:file:%s/backup/%s;create=false;hsqldb.full_log_replay=true";
return String.format(backupUrlTemplate, oldRepoDirPath.toString(), oldRepoFilePath.toString());
String backupUrlTemplate = "jdbc:hsqldb:file:%s%sbackup%s%s;create=false;hsqldb.full_log_replay=true";
return String.format(backupUrlTemplate, oldRepoDirPath.toString(), File.separator, File.separator, oldRepoFilePath.toString());
}
/* package */ static void attemptRecovery(String connectionUrl) throws DataException {
@@ -361,8 +371,8 @@ public class HSQLDBRepository implements Repository {
.forEach(File::delete);
try (Statement stmt = connection.createStatement()) {
// Now "backup" the backup back to original repository location (the parent)
// NOTE: trailing / is OK because HSQLDB checks for both / and O/S-specific separator
// Now "backup" the backup back to original repository location (the parent).
// NOTE: trailing / is OK because HSQLDB checks for both / and O/S-specific separator.
// textdb.allow_full_path connection property is required to be able to use '..'
stmt.execute("BACKUP DATABASE TO '../' BLOCKING AS FILES");
} catch (SQLException e) {
@@ -386,6 +396,8 @@ public class HSQLDBRepository implements Repository {
}
}
// SQL statements, etc.
/**
* Returns prepared statement using passed SQL, logging query if necessary.
*/
@@ -399,19 +411,6 @@ public class HSQLDBRepository implements Repository {
return this.connection.prepareStatement(sql);
}
/**
* Logs this transaction's SQL statements, if enabled.
*/
public void logStatements() {
if (this.sqlStatements == null)
return;
LOGGER.info(String.format("HSQLDB SQL statements (session %d) leading up to this were:", this.sessionId));
for (String sql : this.sqlStatements)
LOGGER.info(sql);
}
/**
* Execute SQL and return ResultSet with but added checking.
* <p>
@@ -429,15 +428,18 @@ public class HSQLDBRepository implements Repository {
// We can't use try-with-resources here as closing the PreparedStatement on return would also prematurely close the ResultSet.
preparedStatement.closeOnCompletion();
long beforeQuery = System.currentTimeMillis();
long beforeQuery = this.slowQueryThreshold == null ? 0 : System.currentTimeMillis();
ResultSet resultSet = this.checkedExecuteResultSet(preparedStatement, objects);
long queryTime = System.currentTimeMillis() - beforeQuery;
if (this.slowQueryThreshold != null && queryTime > this.slowQueryThreshold) {
LOGGER.info(String.format("HSQLDB query took %d ms: %s", queryTime, sql), new SQLException("slow query"));
if (this.slowQueryThreshold != null) {
long queryTime = System.currentTimeMillis() - beforeQuery;
logStatements();
if (queryTime > this.slowQueryThreshold) {
LOGGER.info(String.format("HSQLDB query took %d ms: %s", queryTime, sql), new SQLException("slow query"));
logStatements();
}
}
return resultSet;
@@ -500,16 +502,19 @@ public class HSQLDBRepository implements Repository {
try (PreparedStatement preparedStatement = this.prepareStatement(sql)) {
prepareExecute(preparedStatement, objects);
long beforeQuery = System.currentTimeMillis();
long beforeQuery = this.slowQueryThreshold == null ? 0 : System.currentTimeMillis();
if (preparedStatement.execute())
throw new SQLException("Database produced results, not row count");
long queryTime = System.currentTimeMillis() - beforeQuery;
if (this.slowQueryThreshold != null && queryTime > this.slowQueryThreshold) {
LOGGER.info(String.format("HSQLDB query took %d ms: %s", queryTime, sql), new SQLException("slow query"));
if (this.slowQueryThreshold != null) {
long queryTime = System.currentTimeMillis() - beforeQuery;
logStatements();
if (queryTime > this.slowQueryThreshold) {
LOGGER.info(String.format("HSQLDB query took %d ms: %s", queryTime, sql), new SQLException("slow query"));
logStatements();
}
}
int rowCount = preparedStatement.getUpdateCount();
@@ -670,6 +675,21 @@ public class HSQLDBRepository implements Repository {
stringBuilder.append(") ");
}
// Debugging
/**
* Logs this transaction's SQL statements, if enabled.
*/
public void logStatements() {
if (this.sqlStatements == null)
return;
LOGGER.info(String.format("HSQLDB SQL statements (session %d) leading up to this were:", this.sessionId));
for (String sql : this.sqlStatements)
LOGGER.info(sql);
}
/** Logs other HSQLDB sessions then re-throws passed exception */
public SQLException examineException(SQLException e) throws SQLException {
LOGGER.error(String.format("HSQLDB error (session %d): %s", this.sessionId, e.getMessage()), e);
@@ -726,6 +746,22 @@ public class HSQLDBRepository implements Repository {
}
}
// Utility methods
public static byte[] ed25519PrivateToPublicKey(byte[] privateKey) {
if (privateKey == null)
return null;
return PrivateKeyAccount.toPublicKey(privateKey);
}
public static String ed25519PublicKeyToAddress(byte[] publicKey) {
if (publicKey == null)
return null;
return Crypto.toAddress(publicKey);
}
/** Converts milliseconds from epoch to OffsetDateTime needed for TIMESTAMP WITH TIME ZONE columns. */
/* package */ static OffsetDateTime toOffsetDateTime(Long timestamp) {
if (timestamp == null)

View File

@@ -71,17 +71,23 @@ public class Settings {
private int maxTransactionTimestampFuture = 24 * 60 * 60 * 1000; // milliseconds
/** Whether we check, fetch and install auto-updates */
private boolean autoUpdateEnabled = true;
/** How long between repository backups (ms), or 0 if disabled. */
private long repositoryBackupInterval = 0; // ms
/** Whether to show a notification when we backup repository. */
private boolean showBackupNotification = false;
// Peer-to-peer related
private boolean isTestNet = false;
/** Port number for inbound peer-to-peer connections. */
private Integer listenPort;
/** Minimum number of peers to allow block minting / synchronization. */
private int minBlockchainPeers = 5;
private int minBlockchainPeers = 8;
/** Target number of outbound connections to peers we should make. */
private int minOutboundPeers = 20;
private int minOutboundPeers = 16;
/** Maximum number of peer connections we allow. */
private int maxPeers = 50;
private int maxPeers = 32;
/** Maximum number of threads for network engine. */
private int maxNetworkThreadPoolSize = 20;
// Which blockchains this node is running
private String blockchainConfig = null; // use default from resources
@@ -113,7 +119,7 @@ public class Settings {
"3.cn.pool.ntp.org"
};
/** Additional offset added to values returned by NTP.getTime() */
private long testNtpOffset = 0;
private Long testNtpOffset = null;
// Constructors
@@ -331,6 +337,10 @@ public class Settings {
return this.maxPeers;
}
public int getMaxNetworkThreadPoolSize() {
return this.maxNetworkThreadPoolSize;
}
public String getBlockchainConfig() {
return this.blockchainConfig;
}
@@ -359,8 +369,16 @@ public class Settings {
return this.ntpServers;
}
public long getTestNtpOffset() {
public Long getTestNtpOffset() {
return this.testNtpOffset;
}
public long getRepositoryBackupInterval() {
return this.repositoryBackupInterval;
}
public boolean getShowBackupNotification() {
return this.showBackupNotification;
}
}

View File

@@ -2,7 +2,6 @@ package org.qortal.transaction;
import java.math.BigDecimal;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import org.qortal.account.Account;
@@ -173,10 +172,6 @@ public class CreateAssetOrderTransaction extends Transaction {
return ValidationResult.NO_BALANCE;
}
// Check reference is correct
if (!Arrays.equals(creator.getLastReference(), createOrderTransactionData.getReference()))
return ValidationResult.INVALID_REFERENCE;
return ValidationResult.OK;
}

View File

@@ -77,6 +77,16 @@ public class CreateGroupTransaction extends Transaction {
if (createGroupTransactionData.getApprovalThreshold() == null)
return ValidationResult.INVALID_GROUP_APPROVAL_THRESHOLD;
// Check min/max block delay values
if (createGroupTransactionData.getMinimumBlockDelay() < 0)
return ValidationResult.INVALID_GROUP_BLOCK_DELAY;
if (createGroupTransactionData.getMaximumBlockDelay() < 1)
return ValidationResult.INVALID_GROUP_BLOCK_DELAY;
if (createGroupTransactionData.getMaximumBlockDelay() < createGroupTransactionData.getMinimumBlockDelay())
return ValidationResult.INVALID_GROUP_BLOCK_DELAY;
// Check group name size bounds
int groupNameLength = Utf8.encodedLength(createGroupTransactionData.getGroupName());
if (groupNameLength < 1 || groupNameLength > Group.MAX_NAME_SIZE)

View File

@@ -195,7 +195,9 @@ public class RewardShareTransaction extends Transaction {
this.repository.getAccountRepository().delete(mintingAccount.getPublicKey(), rewardShareTransactionData.getRecipient());
} else {
// Save reward-share info
rewardShareData = new RewardShareData(mintingAccount.getPublicKey(), rewardShareTransactionData.getRecipient(), rewardShareTransactionData.getRewardSharePublicKey(), rewardShareTransactionData.getSharePercent());
rewardShareData = new RewardShareData(mintingAccount.getPublicKey(), mintingAccount.getAddress(),
rewardShareTransactionData.getRecipient(), rewardShareTransactionData.getRewardSharePublicKey(),
rewardShareTransactionData.getSharePercent());
this.repository.getAccountRepository().save(rewardShareData);
}
}
@@ -217,8 +219,9 @@ public class RewardShareTransaction extends Transaction {
if (rewardShareTransactionData.getPreviousSharePercent() != null) {
// Revert previous sharing arrangement
RewardShareData rewardShareData = new RewardShareData(mintingAccount.getPublicKey(), rewardShareTransactionData.getRecipient(),
rewardShareTransactionData.getRewardSharePublicKey(), rewardShareTransactionData.getPreviousSharePercent());
RewardShareData rewardShareData = new RewardShareData(mintingAccount.getPublicKey(), mintingAccount.getAddress(),
rewardShareTransactionData.getRecipient(), rewardShareTransactionData.getRewardSharePublicKey(),
rewardShareTransactionData.getPreviousSharePercent());
this.repository.getAccountRepository().save(rewardShareData);
} else {

View File

@@ -242,6 +242,7 @@ public abstract class Transaction {
ACCOUNT_CANNOT_REWARD_SHARE(90),
SELF_SHARE_EXISTS(91),
ACCOUNT_ALREADY_EXISTS(92),
INVALID_GROUP_BLOCK_DELAY(93),
NOT_YET_RELEASED(1000);
public final int value;
@@ -345,18 +346,10 @@ public abstract class Transaction {
BigDecimal maxBytePerUnitFee = BlockChain.getInstance().getMaxBytesPerUnitFee();
BigDecimal recommendedFee = BigDecimal.valueOf(dataLength).divide(maxBytePerUnitFee, MathContext.DECIMAL32).setScale(8);
BigDecimal unitFeeCount = BigDecimal.valueOf(dataLength).divide(maxBytePerUnitFee, RoundingMode.UP);
// security margin
recommendedFee = recommendedFee.add(new BigDecimal("0.00000001"));
if (recommendedFee.compareTo(BlockChain.getInstance().getUnitFee()) <= 0) {
recommendedFee = BlockChain.getInstance().getUnitFee();
} else {
recommendedFee = recommendedFee.setScale(0, RoundingMode.CEILING);
}
return recommendedFee.setScale(8);
BigDecimal recommendedFee = BlockChain.getInstance().getUnitFee().multiply(unitFeeCount).setScale(8);
return recommendedFee;
}
/**

View File

@@ -83,6 +83,16 @@ public class UpdateGroupTransaction extends Transaction {
if (updateGroupTransactionData.getNewApprovalThreshold() == null)
return ValidationResult.INVALID_GROUP_APPROVAL_THRESHOLD;
// Check min/max block delay values
if (updateGroupTransactionData.getNewMinimumBlockDelay() < 0)
return ValidationResult.INVALID_GROUP_BLOCK_DELAY;
if (updateGroupTransactionData.getNewMaximumBlockDelay() < 1)
return ValidationResult.INVALID_GROUP_BLOCK_DELAY;
if (updateGroupTransactionData.getNewMaximumBlockDelay() < updateGroupTransactionData.getNewMinimumBlockDelay())
return ValidationResult.INVALID_GROUP_BLOCK_DELAY;
// Check new description size bounds
int newDescriptionLength = Utf8.encodedLength(updateGroupTransactionData.getNewDescription());
if (newDescriptionLength < 1 || newDescriptionLength > Group.MAX_DESCRIPTION_SIZE)

View File

@@ -5,13 +5,30 @@ import java.util.concurrent.Executors;
import java.util.concurrent.RejectedExecutionException;
import java.util.concurrent.TimeUnit;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
public abstract class ExecuteProduceConsume implements Runnable {
@XmlAccessorType(XmlAccessType.FIELD)
public static class StatsSnapshot {
public int activeThreadCount = 0;
public int greatestActiveThreadCount = 0;
public int consumerCount = 0;
public int tasksProduced = 0;
public int tasksConsumed = 0;
public int spawnFailures = 0;
public StatsSnapshot() {
}
}
private final String className;
private final Logger logger;
private final boolean isLoggerTraceEnabled;
private ExecutorService executor;
@@ -24,12 +41,14 @@ public abstract class ExecuteProduceConsume implements Runnable {
private volatile int consumerCount = 0;
private volatile int tasksProduced = 0;
private volatile int tasksConsumed = 0;
private volatile int spawnFailures = 0;
private volatile boolean hasThreadPending = false;
public ExecuteProduceConsume(ExecutorService executor) {
this.className = this.getClass().getSimpleName();
this.logger = LogManager.getLogger(this.getClass());
this.isLoggerTraceEnabled = this.logger.isTraceEnabled();
this.executor = executor;
}
@@ -51,28 +70,19 @@ public abstract class ExecuteProduceConsume implements Runnable {
return this.executor.awaitTermination(timeout, TimeUnit.MILLISECONDS);
}
public int getActiveThreadCount() {
synchronized (this) {
return this.activeThreadCount;
}
}
public StatsSnapshot getStatsSnapshot() {
StatsSnapshot snapshot = new StatsSnapshot();
public int getGreatestActiveThreadCount() {
synchronized (this) {
return this.greatestActiveThreadCount;
snapshot.activeThreadCount = this.activeThreadCount;
snapshot.greatestActiveThreadCount = this.greatestActiveThreadCount;
snapshot.consumerCount = this.consumerCount;
snapshot.tasksProduced = this.tasksProduced;
snapshot.tasksConsumed = this.tasksConsumed;
snapshot.spawnFailures = this.spawnFailures;
}
}
public int getTasksProduced() {
synchronized (this) {
return this.tasksProduced;
}
}
public int getTasksConsumed() {
synchronized (this) {
return this.tasksConsumed;
}
return snapshot;
}
/**
@@ -91,7 +101,8 @@ public abstract class ExecuteProduceConsume implements Runnable {
@Override
public void run() {
Thread.currentThread().setName(this.className + "-" + Thread.currentThread().getId());
if (this.isLoggerTraceEnabled)
Thread.currentThread().setName(this.className + "-" + Thread.currentThread().getId());
boolean wasThreadPending;
synchronized (this) {
@@ -126,10 +137,9 @@ public abstract class ExecuteProduceConsume implements Runnable {
this.logger.trace(() -> String.format("[%d] producing, activeThreadCount: %d, consumerCount: %d, canBlock is %b...",
Thread.currentThread().getId(), this.activeThreadCount, this.consumerCount, lambdaCanIdle));
final long now = System.currentTimeMillis();
final long beforeProduce = isLoggerTraceEnabled ? System.currentTimeMillis() : 0;
task = produceTask(canBlock);
final long delay = System.currentTimeMillis() - now;
this.logger.trace(() -> String.format("[%d] producing took %dms", Thread.currentThread().getId(), delay));
this.logger.trace(() -> String.format("[%d] producing took %dms", Thread.currentThread().getId(), System.currentTimeMillis() - beforeProduce));
}
if (task == null)
@@ -167,6 +177,7 @@ public abstract class ExecuteProduceConsume implements Runnable {
try {
this.executor.execute(this); // Same object, different thread
} catch (RejectedExecutionException e) {
++this.spawnFailures;
this.hasThreadPending = false;
this.logger.trace(() -> String.format("[%d] failed to spawn another thread", Thread.currentThread().getId()));
}
@@ -193,7 +204,8 @@ public abstract class ExecuteProduceConsume implements Runnable {
} catch (InterruptedException e) {
// We're in shutdown situation so exit
} finally {
Thread.currentThread().setName(this.className + "-dormant");
if (this.isLoggerTraceEnabled)
Thread.currentThread().setName(this.className);
}
}

View File

@@ -11,6 +11,7 @@ import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorCompletionService;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.stream.Collectors;
import org.apache.commons.net.ntp.NTPUDPClient;
import org.apache.commons.net.ntp.NtpV3Packet;
@@ -18,7 +19,6 @@ import org.apache.commons.net.ntp.TimeInfo;
import org.apache.logging.log4j.Level;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.qortal.settings.Settings;
public class NTP implements Runnable {
@@ -53,15 +53,10 @@ public class NTP implements Runnable {
this.remote = remote;
}
public boolean doPoll(NTPUDPClient client) {
public boolean doPoll(NTPUDPClient client, final long now) {
Thread.currentThread().setName(String.format("NTP: %s", this.remote));
try {
final long now = System.currentTimeMillis();
if (now < this.nextPoll)
return false;
boolean isUpdated = false;
try {
TimeInfo timeInfo = client.getTime(InetAddress.getByName(remote));
@@ -110,26 +105,26 @@ public class NTP implements Runnable {
}
private final NTPUDPClient client;
private List<NTPServer> ntpServers = new ArrayList<>();
private final List<NTPServer> ntpServers = new ArrayList<>();
private final ExecutorService serverExecutor;
private NTP() {
private NTP(String[] serverNames) {
client = new NTPUDPClient();
client.setDefaultTimeout(2000);
for (String serverName : Settings.getInstance().getNtpServers())
for (String serverName : serverNames)
ntpServers.add(new NTPServer(serverName));
serverExecutor = Executors.newCachedThreadPool();
}
public static synchronized void start() {
public static synchronized void start(String[] serverNames) {
if (isStarted)
return;
isStarted = true;
instanceExecutor = Executors.newSingleThreadExecutor();
instance = new NTP();
instance = new NTP(serverNames);
instanceExecutor.execute(instance);
}
@@ -137,9 +132,9 @@ public class NTP implements Runnable {
instanceExecutor.shutdownNow();
}
public static synchronized void testMode() {
// Fix offset to match system time
NTP.offset = 0L;
public static synchronized void setFixedOffset(Long offset) {
// Fix offset, e.g. for testing
NTP.offset = offset;
}
/**
@@ -151,7 +146,7 @@ public class NTP implements Runnable {
if (NTP.offset == null)
return null;
return System.currentTimeMillis() + NTP.offset + Settings.getInstance().getTestNtpOffset();
return System.currentTimeMillis() + NTP.offset;
}
public void run() {
@@ -161,103 +156,120 @@ public class NTP implements Runnable {
while (!isStopping) {
Thread.sleep(1000);
CompletionService<Boolean> ecs = new ExecutorCompletionService<>(serverExecutor);
for (NTPServer server : ntpServers)
ecs.submit(() -> server.doPoll(client));
boolean haveUpdates = pollServers();
if (!haveUpdates)
continue;
boolean hasUpdate = false;
for (int i = 0; i < ntpServers.size(); ++i) {
if (isStopping)
return;
try {
hasUpdate = ecs.take().get() || hasUpdate;
} catch (ExecutionException e) {
// skip
}
}
if (hasUpdate) {
double s0 = 0;
double s1 = 0;
double s2 = 0;
for (NTPServer server : ntpServers) {
if (server.offset == null) {
server.usage = ' ';
continue;
}
server.usage = '+';
double value = server.offset * (double) server.stratum;
s0 += 1;
s1 += value;
s2 += value * value;
}
if (s0 < ntpServers.size() / 3 + 1) {
LOGGER.debug(String.format("Not enough replies (%d) to calculate network time", s0));
} else {
double thresholdStddev = Math.sqrt(((s0 * s2) - (s1 * s1)) / (s0 * (s0 - 1)));
double mean = s1 / s0;
// Now only consider offsets within 1 stddev?
s0 = 0;
s1 = 0;
s2 = 0;
for (NTPServer server : ntpServers) {
if (server.offset == null || server.reach == 0)
continue;
if (Math.abs(server.offset * (double)server.stratum - mean) > thresholdStddev)
continue;
server.usage = '*';
s0 += 1;
s1 += server.offset;
s2 += server.offset * server.offset;
}
if (s0 <= 1) {
LOGGER.debug(String.format("Not enough useful values (%d) to calculate network time. (stddev: %7.4f)", s0, thresholdStddev));
} else {
double filteredMean = s1 / s0;
double filteredStddev = Math.sqrt(((s0 * s2) - (s1 * s1)) / (s0 * (s0 - 1)));
LOGGER.trace(String.format("Threshold stddev: %7.3f, mean: %7.3f, stddev: %7.3f, nValues: %.0f / %d",
thresholdStddev, filteredMean, filteredStddev, s0, ntpServers.size()));
NTP.offset = (long) filteredMean;
LOGGER.debug(String.format("New NTP offset: %d", NTP.offset));
}
}
if (LOGGER.getLevel().isMoreSpecificThan(Level.TRACE)) {
LOGGER.trace(String.format("%c%16s %16s %2s %c %4s %4s %3s %7s %7s %7s",
' ', "remote", "refid", "st", 't', "when", "poll", "reach", "delay", "offset", "jitter"
));
for (NTPServer server : ntpServers)
LOGGER.trace(String.format("%c%16.16s %16.16s %2s %c %4s %4d %3o %7s %7s %7s",
server.usage,
server.remote,
formatNull("%s", server.refId, ""),
formatNull("%2d", server.stratum, ""),
server.type,
formatNull("%4d", server.getWhen(), "-"),
server.poll,
server.reach,
formatNull("%5dms", server.delay, ""),
formatNull("% 5.0fms", server.offset, ""),
formatNull("%5.2fms", server.jitter, "")
));
}
}
calculateOffset();
}
} catch (InterruptedException e) {
// Exit
// Interrupted - time to exit
return;
}
}
private boolean pollServers() throws InterruptedException {
final long now = System.currentTimeMillis();
List<NTPServer> pendingServers = ntpServers.stream().filter(ntpServer -> now >= ntpServer.nextPoll).collect(Collectors.toList());
CompletionService<Boolean> ecs = new ExecutorCompletionService<>(serverExecutor);
for (NTPServer server : pendingServers)
ecs.submit(() -> server.doPoll(client, now));
boolean haveUpdate = false;
for (int i = 0; i < pendingServers.size(); ++i) {
if (isStopping)
return false;
try {
haveUpdate = ecs.take().get() || haveUpdate;
} catch (ExecutionException e) {
// skip
}
}
return haveUpdate;
}
private void calculateOffset() {
double s0 = 0;
double s1 = 0;
double s2 = 0;
for (NTPServer server : ntpServers) {
if (server.offset == null) {
server.usage = ' ';
continue;
}
server.usage = '+';
double value = server.offset * (double) server.stratum;
s0 += 1;
s1 += value;
s2 += value * value;
}
if (s0 < ntpServers.size() / 3 + 1) {
final double numberReplies = s0;
LOGGER.debug(() -> String.format("Not enough replies (%d) to calculate network time", numberReplies));
} else {
double thresholdStddev = Math.sqrt(((s0 * s2) - (s1 * s1)) / (s0 * (s0 - 1)));
double mean = s1 / s0;
// Now only consider offsets within 1 stddev?
s0 = 0;
s1 = 0;
s2 = 0;
for (NTPServer server : ntpServers) {
if (server.offset == null || server.reach == 0)
continue;
if (Math.abs(server.offset * (double)server.stratum - mean) > thresholdStddev)
continue;
server.usage = '*';
s0 += 1;
s1 += server.offset;
s2 += server.offset * server.offset;
}
final double numberValues = s0;
if (s0 <= 1) {
LOGGER.debug(() -> String.format("Not enough useful values (%d) to calculate network time. (stddev: %7.4f)", numberValues, thresholdStddev));
} else {
double filteredMean = s1 / s0;
double filteredStddev = Math.sqrt(((s0 * s2) - (s1 * s1)) / (s0 * (s0 - 1)));
LOGGER.trace(() -> String.format("Threshold stddev: %7.3f, mean: %7.3f, stddev: %7.3f, nValues: %.0f / %d",
thresholdStddev, filteredMean, filteredStddev, numberValues, ntpServers.size()));
NTP.offset = (long) filteredMean;
LOGGER.debug(() -> String.format("New NTP offset: %d", NTP.offset));
}
}
if (LOGGER.getLevel().isLessSpecificThan(Level.TRACE)) {
LOGGER.trace(() -> String.format("%c%16s %16s %2s %c %4s %4s %3s %7s %7s %7s",
' ', "remote", "refid", "st", 't', "when", "poll", "reach", "delay", "offset", "jitter"
));
for (NTPServer server : ntpServers)
LOGGER.trace(() -> String.format("%c%16.16s %16.16s %2s %c %4s %4d %3o %7s %7s %7s",
server.usage,
server.remote,
formatNull("%s", server.refId, ""),
formatNull("%2d", server.stratum, ""),
server.type,
formatNull("%4d", server.getWhen(), "-"),
server.poll,
server.reach,
formatNull("%5dms", server.delay, ""),
formatNull("% 5.0fms", server.offset, ""),
formatNull("%5.2fms", server.jitter, "")
));
}
}

View File

@@ -0,0 +1,13 @@
<!doctype html>
<html>
<head>
<title>API Documentation disabled</title>
</head>
<body>
API documentation pages are currently disabled.
<p>
To enable, add an entry like this to your <tt>settings.json</tt> file:
<p>
<code>"apiDocumentationEnabled": true,</code>
</body>
</html>

View File

@@ -48,7 +48,7 @@ PUBLIC_KEY_NOT_FOUND = public key not found
REPOSITORY_ISSUE = repository error
TRANSACTION_INVALID = transaction invalid: %s
TRANSACTION_INVALID = transaction invalid: %s (%s)
TRANSACTION_UNKNOWN = transaction unknown

View File

@@ -1,14 +1,24 @@
#Generated by ResourceBundle Editor (http://essiembre.github.io/eclipse-rbe/)
# SysTray pop-up menu
APPLYING_UPDATE_AND_RESTARTING = Applying automatic update and restarting...
AUTO_UPDATE = Auto Update
BLOCK_HEIGHT = height
CHECK_TIME_ACCURACY = Check time accuracy
CONNECTING = Connecting
CONNECTION = connection
CONNECTIONS = connections
CREATING_BACKUP_OF_DB_FILES = Creating backup of database files...
DB_BACKUP = Database Backup
EXIT = Exit
MINTING_DISABLED = NOT minting
@@ -26,4 +36,6 @@ OPEN_UI = Open UI
SYNCHRONIZE_CLOCK = Synchronize clock
SYNCHRONIZING_BLOCKCHAIN = Synchronizing
SYNCHRONIZING_CLOCK = Synchronizing clock

View File

@@ -26,4 +26,6 @@ OPEN_UI = \u5F00\u542F\u754C\u9762
SYNCHRONIZE_CLOCK = \u540C\u6B65\u65F6\u949F
SYNCHRONIZING_BLOCKCHAIN = \u540C\u6B65\u533A\u5757\u94FE
SYNCHRONIZING_CLOCK = \u540C\u6B65\u7740\u65F6\u949F

View File

@@ -1,4 +1,6 @@
ACCOUNT_ALREADY_EXISTS = account already exists
ACCOUNT_CANNOT_REWARD_SHARE = account cannot reward-share
ALREADY_GROUP_ADMIN = already group admin
@@ -139,7 +141,7 @@ NOT_YET_RELEASED = NOT_YET_RELEASED
NO_BALANCE = NO_BALANCE
NO_BLOCKCHAIN_LOCK = NO_BLOCKCHAIN_LOCK
NO_BLOCKCHAIN_LOCK = node's blockchain currently busy
NO_FLAG_PERMISSION = NO_FLAG_PERMISSION

Binary file not shown.

Before

Width:  |  Height:  |  Size: 22 KiB

After

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 770 B

After

Width:  |  Height:  |  Size: 852 B

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.3 KiB

After

Width:  |  Height:  |  Size: 2.6 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.9 KiB

After

Width:  |  Height:  |  Size: 8.4 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 250 KiB

After

Width:  |  Height:  |  Size: 250 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 74 KiB

After

Width:  |  Height:  |  Size: 77 KiB

View File

@@ -14,15 +14,10 @@ import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.qortal.account.Account;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.account.PublicKeyAccount;
import org.qortal.asset.Asset;
import org.qortal.block.BlockChain;
import org.qortal.data.account.AccountBalanceData;
import org.qortal.data.account.AccountData;
import org.qortal.data.transaction.BaseTransactionData;
import org.qortal.data.transaction.PaymentTransactionData;
import org.qortal.data.transaction.TransactionData;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
@@ -30,7 +25,6 @@ import org.qortal.repository.AccountRepository.BalanceOrdering;
import org.qortal.test.common.BlockUtils;
import org.qortal.test.common.Common;
import org.qortal.test.common.TestAccount;
import org.qortal.test.common.TransactionUtils;
public class AccountBalanceTests extends Common {
@@ -88,123 +82,6 @@ public class AccountBalanceTests extends Common {
}
}
/** Tests we can fetch initial balance when newer balance exists. */
@Test
public void testGetBalanceAtHeight() throws DataException {
try (final Repository repository = RepositoryManager.getRepository()) {
TestAccount alice = Common.getTestAccount(repository, "alice");
BigDecimal initialBalance = testNewerBalance(repository, alice);
// Fetch all historic balances
List<AccountBalanceData> historicBalances = repository.getAccountRepository().getHistoricBalances(alice.getAddress(), Asset.QORT);
for (AccountBalanceData historicBalance : historicBalances)
System.out.println(String.format("Balance at height %d: %s", historicBalance.getHeight(), historicBalance.getBalance().toPlainString()));
// Fetch balance at height 1, even though newer balance exists
AccountBalanceData accountBalanceData = repository.getAccountRepository().getBalance(alice.getAddress(), Asset.QORT, 1);
BigDecimal genesisBalance = accountBalanceData.getBalance();
// Confirm genesis balance is same as initial
assertEqualBigDecimals("Genesis balance should match initial", initialBalance, genesisBalance);
}
}
/** Tests we can fetch balance with a height where no balance change occurred. */
@Test
public void testGetBalanceAtNearestHeight() throws DataException {
Random random = new Random();
byte[] publicKey = new byte[32];
random.nextBytes(publicKey);
try (final Repository repository = RepositoryManager.getRepository()) {
PublicKeyAccount recipientAccount = new PublicKeyAccount(repository, publicKey);
System.out.println(String.format("Test recipient: %s", recipientAccount.getAddress()));
// Mint a few blocks
for (int i = 0; i < 10; ++i)
BlockUtils.mintBlock(repository);
// Confirm recipient balance is zero
BigDecimal balance = recipientAccount.getConfirmedBalance(Asset.QORT);
assertEqualBigDecimals("recipient's balance should be zero", BigDecimal.ZERO, balance);
// Confirm recipient has no historic balances
List<AccountBalanceData> historicBalances = repository.getAccountRepository().getHistoricBalances(recipientAccount.getAddress(), Asset.QORT);
for (AccountBalanceData historicBalance : historicBalances)
System.err.println(String.format("Block %d: %s", historicBalance.getHeight(), historicBalance.getBalance().toPlainString()));
assertTrue("recipient should not have historic balances yet", historicBalances.isEmpty());
// Send 1 QORT to recipient
TestAccount sendingAccount = Common.getTestAccount(repository, "alice");
pay(repository, sendingAccount, recipientAccount, BigDecimal.ONE);
// Mint some more blocks
for (int i = 0; i < 10; ++i)
BlockUtils.mintBlock(repository);
// Send more QORT to recipient
BigDecimal amount = BigDecimal.valueOf(random.nextInt(123456));
pay(repository, sendingAccount, recipientAccount, amount);
BigDecimal totalAmount = BigDecimal.ONE.add(amount);
// Mint some more blocks
for (int i = 0; i < 10; ++i)
BlockUtils.mintBlock(repository);
// Confirm recipient balance is as expected
balance = recipientAccount.getConfirmedBalance(Asset.QORT);
assertEqualBigDecimals("recipient's balance incorrect", totalAmount, balance);
historicBalances = repository.getAccountRepository().getHistoricBalances(recipientAccount.getAddress(), Asset.QORT);
for (AccountBalanceData historicBalance : historicBalances)
System.out.println(String.format("Block %d: %s", historicBalance.getHeight(), historicBalance.getBalance().toPlainString()));
// Confirm balance as of 2 blocks ago
int height = repository.getBlockRepository().getBlockchainHeight();
balance = repository.getAccountRepository().getBalance(recipientAccount.getAddress(), Asset.QORT, height - 2).getBalance();
assertEqualBigDecimals("recipient's historic balance incorrect", totalAmount, balance);
// Confirm balance prior to last payment
balance = repository.getAccountRepository().getBalance(recipientAccount.getAddress(), Asset.QORT, height - 15).getBalance();
assertEqualBigDecimals("recipient's historic balance incorrect", BigDecimal.ONE, balance);
// Orphan blocks to before last payment
BlockUtils.orphanBlocks(repository, 10 + 5);
// Re-check balance from (now) invalid height
AccountBalanceData accountBalanceData = repository.getAccountRepository().getBalance(recipientAccount.getAddress(), Asset.QORT, height - 2);
balance = accountBalanceData.getBalance();
assertEqualBigDecimals("recipient's invalid-height balance should be one", BigDecimal.ONE, balance);
// Orphan blocks to before initial 1 QORT payment
BlockUtils.orphanBlocks(repository, 10 + 5);
// Re-check balance from (now) invalid height
accountBalanceData = repository.getAccountRepository().getBalance(recipientAccount.getAddress(), Asset.QORT, height - 2);
assertNull("recipient's invalid-height balance data should be null", accountBalanceData);
// Confirm recipient has no historic balances
historicBalances = repository.getAccountRepository().getHistoricBalances(recipientAccount.getAddress(), Asset.QORT);
for (AccountBalanceData historicBalance : historicBalances)
System.err.println(String.format("Block %d: %s", historicBalance.getHeight(), historicBalance.getBalance().toPlainString()));
assertTrue("recipient should have no remaining historic balances", historicBalances.isEmpty());
}
}
private void pay(Repository repository, PrivateKeyAccount sendingAccount, Account recipientAccount, BigDecimal amount) throws DataException {
byte[] reference = sendingAccount.getLastReference();
long timestamp = repository.getTransactionRepository().fromSignature(reference).getTimestamp() + 1;
int txGroupId = 0;
BigDecimal fee = BlockChain.getInstance().getUnitFee();
BaseTransactionData baseTransactionData = new BaseTransactionData(timestamp, txGroupId, reference, sendingAccount.getPublicKey(), fee, null);
TransactionData transactionData = new PaymentTransactionData(baseTransactionData, recipientAccount.getAddress(), amount);
TransactionUtils.signAndMint(repository, transactionData, sendingAccount);
}
/** Tests SQL query speed for account balance fetches. */
@Test
public void testRepositorySpeed() throws DataException, SQLException {

View File

@@ -3,6 +3,7 @@ package org.qortal.test;
import org.junit.Test;
import org.qortal.data.transaction.TransactionData;
import org.qortal.repository.DataException;
import org.qortal.settings.Settings;
import org.qortal.test.common.Common;
import org.qortal.transaction.CreateAssetOrderTransaction;
import org.qortal.transaction.CreatePollTransaction;
@@ -22,7 +23,7 @@ public class CompatibilityTests extends Common {
@Before
public void beforeTest() throws DataException {
Common.useSettings("test-settings-v1.json");
NTP.testMode();
NTP.setFixedOffset(Settings.getInstance().getTestNtpOffset());
}
@Test

View File

@@ -11,6 +11,7 @@ import java.util.concurrent.TimeUnit;
import org.junit.Test;
import org.qortal.utils.ExecuteProduceConsume;
import org.qortal.utils.ExecuteProduceConsume.StatsSnapshot;
public class EPCTests {
@@ -60,13 +61,10 @@ public class EPCTests {
ScheduledExecutorService statusExecutor = Executors.newSingleThreadScheduledExecutor();
statusExecutor.scheduleAtFixedRate(() -> {
synchronized (testEPC) {
final long seconds = (System.currentTimeMillis() - start) / 1000L;
System.out.println(String.format("After %d second%s, active threads: %d, greatest thread count: %d, tasks produced: %d, tasks consumed: %d",
seconds, (seconds != 1 ? "s" : ""),
testEPC.getActiveThreadCount(), testEPC.getGreatestActiveThreadCount(),
testEPC.getTasksProduced(), testEPC.getTasksConsumed()));
}
final StatsSnapshot snapshot = testEPC.getStatsSnapshot();
final long seconds = (System.currentTimeMillis() - start) / 1000L;
System.out.print(String.format("After %d second%s, ", seconds, (seconds != 1 ? "s" : "")));
printSnapshot(snapshot);
}, 1L, 1L, TimeUnit.SECONDS);
// Let it run for a minute
@@ -78,10 +76,17 @@ public class EPCTests {
final long after = System.currentTimeMillis();
System.out.println(String.format("Shutdown took %d milliseconds", after - before));
System.out.println(String.format("Greatest thread count: %d", testEPC.getGreatestActiveThreadCount()));
System.out.println(String.format("Tasks produced: %d", testEPC.getTasksProduced()));
System.out.println(String.format("Tasks consumed: %d", testEPC.getTasksConsumed()));
final StatsSnapshot snapshot = testEPC.getStatsSnapshot();
System.out.print("After shutdown, ");
printSnapshot(snapshot);
}
private void printSnapshot(final StatsSnapshot snapshot) {
System.out.println(String.format("threads: %d active (%d max, %d exhaustion%s), tasks: %d produced / %d consumed",
snapshot.activeThreadCount, snapshot.greatestActiveThreadCount,
snapshot.spawnFailures, (snapshot.spawnFailures != 1 ? "s": ""),
snapshot.tasksProduced, snapshot.tasksConsumed));
}
@Test

View File

@@ -13,8 +13,10 @@ import org.qortal.transaction.Transaction.ValidationResult;
public class CheckTranslations {
private static final String[] SUPPORTED_LANGS = new String[] { "en", "de", "zh", "ru" };
private static final Set<String> SYSTRAY_KEYS = Set.of("BLOCK_HEIGHT", "CHECK_TIME_ACCURACY", "CONNECTION", "CONNECTIONS",
"EXIT", "MINTING_DISABLED", "MINTING_ENABLED", "OPEN_UI", "SYNCHRONIZE_CLOCK", "SYNCHRONIZING_CLOCK");
private static final Set<String> SYSTRAY_KEYS = Set.of("AUTO_UPDATE", "APPLYING_UPDATE_AND_RESTARTING", "BLOCK_HEIGHT",
"CHECK_TIME_ACCURACY", "CONNECTING", "CONNECTION", "CONNECTIONS", "CREATING_BACKUP_OF_DB_FILES", "DB_BACKUP", "EXIT",
"MINTING_DISABLED", "MINTING_ENABLED", "NTP_NAG_CAPTION", "NTP_NAG_TEXT_UNIX", "NTP_NAG_TEXT_WINDOWS",
"OPEN_UI", "SYNCHRONIZE_CLOCK", "SYNCHRONIZING_BLOCKCHAIN", "SYNCHRONIZING_CLOCK");
private static String failurePrefix;

View File

@@ -1,200 +1,44 @@
package org.qortal.test.apps;
import java.io.IOException;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Deque;
import java.util.LinkedList;
import java.util.List;
import java.util.concurrent.CompletionService;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorCompletionService;
import java.util.concurrent.Executors;
import org.apache.commons.net.ntp.NTPUDPClient;
import org.apache.commons.net.ntp.NtpV3Packet;
import org.apache.commons.net.ntp.TimeInfo;
import org.apache.logging.log4j.Level;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.core.config.Configuration;
import org.apache.logging.log4j.core.config.LoggerConfig;
import org.apache.logging.log4j.core.LoggerContext;
import org.qortal.utils.NTP;
public class NTPTests {
private static final List<String> CC_TLDS = Arrays.asList("oceania", "europe", "cn", "asia", "africa");
public static void main(String[] args) throws UnknownHostException, IOException, InterruptedException {
NTPUDPClient client = new NTPUDPClient();
client.setDefaultTimeout(2000);
class NTPServer {
private static final int MIN_POLL = 8;
public char usage = ' ';
public String remote;
public String refId;
public Integer stratum;
public char type = 'u'; // unicast
public int poll = MIN_POLL;
public byte reach = 0;
public Long delay;
public Double offset;
public Double jitter;
private Deque<Double> offsets = new LinkedList<>();
private double totalSquareOffsets = 0.0;
private long nextPoll;
private Long lastGood;
public NTPServer(String remote) {
this.remote = remote;
}
public boolean poll(NTPUDPClient client) {
final long now = System.currentTimeMillis();
if (now < this.nextPoll)
return false;
boolean isUpdated = false;
try {
TimeInfo timeInfo = client.getTime(InetAddress.getByName(remote));
timeInfo.computeDetails();
NtpV3Packet ntpMessage = timeInfo.getMessage();
this.refId = ntpMessage.getReferenceIdString();
this.stratum = ntpMessage.getStratum();
this.poll = Math.max(MIN_POLL, 1 << ntpMessage.getPoll());
this.delay = timeInfo.getDelay();
this.offset = (double) timeInfo.getOffset();
if (this.offsets.size() == 8) {
double oldOffset = this.offsets.removeFirst();
this.totalSquareOffsets -= oldOffset * oldOffset;
}
this.offsets.addLast(this.offset);
this.totalSquareOffsets += this.offset * this.offset;
this.jitter = Math.sqrt(this.totalSquareOffsets / this.offsets.size());
this.reach = (byte) ((this.reach << 1) | 1);
this.lastGood = now;
isUpdated = true;
} catch (IOException e) {
this.reach <<= 1;
}
this.nextPoll = now + this.poll * 1000;
return isUpdated;
}
public Integer getWhen() {
if (this.lastGood == null)
return null;
return (int) ((System.currentTimeMillis() - this.lastGood) / 1000);
}
}
List<NTPServer> ntpServers = new ArrayList<>();
public static void main(String[] args) throws InterruptedException {
List<String> ntpServers = new ArrayList<>();
for (String ccTld : CC_TLDS)
for (int subpool = 0; subpool <=3; ++subpool)
ntpServers.add(new NTPServer(subpool + "." + ccTld + ".pool.ntp.org"));
for (int subpool = 0; subpool <= 3; ++subpool)
ntpServers.add(new String(subpool + "." + ccTld + ".pool.ntp.org"));
while (true) {
Thread.sleep(1000);
Runtime.getRuntime().addShutdownHook(new Thread(() -> {
NTP.shutdownNow();
}));
CompletionService<Boolean> ecs = new ExecutorCompletionService<Boolean>(Executors.newCachedThreadPool());
for (NTPServer server : ntpServers)
ecs.submit(() -> server.poll(client));
Logger ntpLogger = LogManager.getLogger(NTP.class);
LoggerContext loggerContext = (LoggerContext) LogManager.getContext(false);
Configuration config = loggerContext.getConfiguration();
LoggerConfig loggerConfig = config.getLoggerConfig(ntpLogger.getName());
boolean showReport = false;
for (int i = 0; i < ntpServers.size(); ++i)
try {
showReport = ecs.take().get() || showReport;
} catch (ExecutionException e) {
// skip
}
loggerConfig.setLevel(Level.TRACE);
loggerContext.updateLoggers(config);
if (showReport) {
double s0 = 0;
double s1 = 0;
double s2 = 0;
NTP.start(ntpServers.toArray(new String[0]));
for (NTPServer server : ntpServers) {
if (server.offset == null) {
server.usage = ' ';
continue;
}
server.usage = '+';
double value = server.offset * (double) server.stratum;
s0 += 1;
s1 += value;
s2 += value * value;
}
if (s0 < ntpServers.size() / 3 + 1) {
System.out.println("Not enough replies to calculate network time");
} else {
double filterStddev = Math.sqrt(((s0 * s2) - (s1 * s1)) / (s0 * (s0 - 1)));
double filterMean = s1 / s0;
// Now only consider offsets within 1 stddev?
s0 = 0;
s1 = 0;
s2 = 0;
for (NTPServer server : ntpServers) {
if (server.offset == null || server.reach == 0)
continue;
if (Math.abs(server.offset * (double)server.stratum - filterMean) > filterStddev)
continue;
server.usage = '*';
s0 += 1;
s1 += server.offset;
s2 += server.offset * server.offset;
}
if (s0 <= 1) {
System.out.println(String.format("Not enough values to calculate network time. stddev: %7.4f", filterStddev));
} else {
double mean = s1 / s0;
double newStddev = Math.sqrt(((s0 * s2) - (s1 * s1)) / (s0 * (s0 - 1)));
System.out.println(String.format("filtering stddev: %7.3f, mean: %7.3f, new stddev: %7.3f, nValues: %.0f / %d", filterStddev, mean, newStddev, s0, ntpServers.size()));
}
}
System.out.println(String.format("%c%16s %16s %2s %c %4s %4s %3s %7s %7s %7s",
' ', "remote", "refid", "st", 't', "when", "poll", "reach", "delay", "offset", "jitter"
));
for (NTPServer server : ntpServers)
System.out.println(String.format("%c%16.16s %16.16s %2s %c %4s %4d %3o %7s %7s %7s",
server.usage,
server.remote,
formatNull("%s", server.refId, ""),
formatNull("%2d", server.stratum, ""),
server.type,
formatNull("%4d", server.getWhen(), "-"),
server.poll,
server.reach,
formatNull("%5dms", server.delay, ""),
formatNull("% 5.0fms", server.offset, ""),
formatNull("%5.2fms", server.jitter, "")
));
}
}
}
private static String formatNull(String format, Object arg, String nullOutput) {
return arg != null ? String.format(format, arg) : nullOutput;
// Endless sleep
Thread.sleep(1000000000L);
}
}

View File

@@ -6,6 +6,7 @@ import org.junit.Test;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.settings.Settings;
import org.qortal.test.common.AccountUtils;
import org.qortal.test.common.AssetUtils;
import org.qortal.test.common.Common;
@@ -19,7 +20,7 @@ public class OldTradingTests extends Common {
@Before
public void beforeTest() throws DataException {
Common.useSettings("test-settings-old-asset.json");
NTP.testMode();
NTP.setFixedOffset(Settings.getInstance().getTestNtpOffset());
}
@After

View File

@@ -116,7 +116,7 @@ public class Common {
public static void useDefaultSettings() throws DataException {
useSettings(testSettingsFilename);
NTP.testMode();
NTP.setFixedOffset(Settings.getInstance().getTestNtpOffset());
}
public static void resetBlockchain() throws DataException {

View File

@@ -0,0 +1,111 @@
package org.qortal.test.group;
import static org.junit.Assert.*;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.qortal.account.PrivateKeyAccount;
import org.qortal.data.transaction.CreateGroupTransactionData;
import org.qortal.data.transaction.UpdateGroupTransactionData;
import org.qortal.group.Group.ApprovalThreshold;
import org.qortal.repository.DataException;
import org.qortal.repository.Repository;
import org.qortal.repository.RepositoryManager;
import org.qortal.test.common.Common;
import org.qortal.test.common.GroupUtils;
import org.qortal.test.common.TestAccount;
import org.qortal.test.common.transaction.TestTransaction;
import org.qortal.transaction.CreateGroupTransaction;
import org.qortal.transaction.Transaction;
import org.qortal.transaction.Transaction.ValidationResult;
import org.qortal.transaction.UpdateGroupTransaction;
public class GroupBlockDelayTests extends Common {
@Before
public void beforeTest() throws DataException {
Common.useDefaultSettings();
}
@After
public void afterTest() throws DataException {
Common.orphanCheck();
}
@Test
public void testCreateGroupBlockDelayValues() throws DataException {
try (final Repository repository = RepositoryManager.getRepository()) {
TestAccount alice = Common.getTestAccount(repository, "alice");
// Check we're starting with something valid
Transaction transaction = buildCreateGroupWithDelays(repository, alice, 10, 40);
assertEquals(ValidationResult.OK, transaction.isValid());
transaction = buildCreateGroupWithDelays(repository, alice, -1, 40);
assertNotSame("Negative minimum block delay should be invalid", ValidationResult.OK, transaction.isValid());
transaction = buildCreateGroupWithDelays(repository, alice, 10, -1);
assertNotSame("Negative maximum block delay should be invalid", ValidationResult.OK, transaction.isValid());
transaction = buildCreateGroupWithDelays(repository, alice, 10, 0);
assertNotSame("Zero maximum block delay should be invalid", ValidationResult.OK, transaction.isValid());
transaction = buildCreateGroupWithDelays(repository, alice, 40, 10);
assertNotSame("Maximum block delay smaller than minimum block delay should be invalid", ValidationResult.OK, transaction.isValid());
transaction = buildCreateGroupWithDelays(repository, alice, 40, 40);
assertEquals("Maximum block delay same as minimum block delay should be OK", ValidationResult.OK, transaction.isValid());
}
}
private CreateGroupTransaction buildCreateGroupWithDelays(Repository repository, PrivateKeyAccount account, int minimumBlockDelay, int maximumBlockDelay) throws DataException {
String owner = account.getAddress();
String groupName = "test group";
String description = "random test group";
final boolean isOpen = false;
ApprovalThreshold approvalThreshold = ApprovalThreshold.PCT40;
CreateGroupTransactionData transactionData = new CreateGroupTransactionData(TestTransaction.generateBase(account), owner, groupName, description, isOpen, approvalThreshold, minimumBlockDelay, maximumBlockDelay);
return new CreateGroupTransaction(repository, transactionData);
}
@Test
public void testUpdateGroupBlockDelayValues() throws DataException {
try (final Repository repository = RepositoryManager.getRepository()) {
TestAccount alice = Common.getTestAccount(repository, "alice");
int groupId = GroupUtils.createGroup(repository, "alice", "test", true, ApprovalThreshold.ONE, 10, 40);
// Check we're starting with something valid
Transaction transaction = buildUpdateGroupWithDelays(repository, alice, groupId, 10, 40);
assertEquals(ValidationResult.OK, transaction.isValid());
transaction = buildUpdateGroupWithDelays(repository, alice, groupId, -1, 40);
assertNotSame("Negative minimum block delay should be invalid", ValidationResult.OK, transaction.isValid());
transaction = buildUpdateGroupWithDelays(repository, alice, groupId, 10, -1);
assertNotSame("Negative maximum block delay should be invalid", ValidationResult.OK, transaction.isValid());
transaction = buildUpdateGroupWithDelays(repository, alice, groupId, 10, 0);
assertNotSame("Zero maximum block delay should be invalid", ValidationResult.OK, transaction.isValid());
transaction = buildUpdateGroupWithDelays(repository, alice, groupId, 40, 10);
assertNotSame("Maximum block delay smaller than minimum block delay should be invalid", ValidationResult.OK, transaction.isValid());
transaction = buildUpdateGroupWithDelays(repository, alice, groupId, 40, 40);
assertEquals("Maximum block delay same as minimum block delay should be OK", ValidationResult.OK, transaction.isValid());
}
}
private UpdateGroupTransaction buildUpdateGroupWithDelays(Repository repository, PrivateKeyAccount account, int groupId, int newMinimumBlockDelay, int newMaximumBlockDelay) throws DataException {
String newOwner = account.getAddress();
String newDescription = "random test group";
final boolean newIsOpen = false;
ApprovalThreshold newApprovalThreshold = ApprovalThreshold.PCT40;
UpdateGroupTransactionData transactionData = new UpdateGroupTransactionData(TestTransaction.generateBase(account), groupId, newOwner, newDescription, newIsOpen, newApprovalThreshold, newMinimumBlockDelay, newMaximumBlockDelay);
return new UpdateGroupTransaction(repository, transactionData);
}
}

View File

@@ -4,9 +4,11 @@ import static org.junit.Assert.*;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import org.bitcoinj.core.Base58;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
@@ -204,4 +206,55 @@ public class RewardTests extends Common {
}
}
/** Test rewards to founders, one in reward-share, the other is self-share. */
@Test
public void testFounderRewards() throws DataException {
Common.useSettings("test-settings-v2-founder-rewards.json");
BigDecimal perHundred = BigDecimal.valueOf(100L);
try (final Repository repository = RepositoryManager.getRepository()) {
BigDecimal blockReward = BlockUtils.getNextBlockReward(repository);
List<PrivateKeyAccount> mintingAndOnlineAccounts = new ArrayList<>();
// Alice to mint, therefore online
PrivateKeyAccount aliceSelfShare = Common.getTestAccount(repository, "alice-reward-share");
mintingAndOnlineAccounts.add(aliceSelfShare);
// Bob self-share NOT online
// Chloe self-share and reward-share with Dilbert both online
PrivateKeyAccount chloeSelfShare = Common.getTestAccount(repository, "chloe-reward-share");
mintingAndOnlineAccounts.add(chloeSelfShare);
PrivateKeyAccount chloeDilbertRewardShare = new PrivateKeyAccount(repository, Base58.decode("HuiyqLipUN1V9p1HZfLhyEwmEA6BTaT2qEfjgkwPViV4"));
mintingAndOnlineAccounts.add(chloeDilbertRewardShare);
BlockMinter.mintTestingBlock(repository, mintingAndOnlineAccounts.toArray(new PrivateKeyAccount[0]));
// 3 founders (online or not) so blockReward divided by 3
BigDecimal founderCount = BigDecimal.valueOf(3L);
BigDecimal perFounderReward = blockReward.divide(founderCount, RoundingMode.DOWN);
// Alice simple self-share so her reward is perFounderReward
AccountUtils.assertBalance(repository, "alice", Asset.QORT, perFounderReward);
// Bob not online so his reward is simply perFounderReward
AccountUtils.assertBalance(repository, "bob", Asset.QORT, perFounderReward);
// Chloe has two reward-shares, so her reward is divided by 2
BigDecimal chloeSharesCount = BigDecimal.valueOf(2L);
BigDecimal chloePerShareReward = perFounderReward.divide(chloeSharesCount, RoundingMode.DOWN);
// Her self-share gets chloePerShareReward
BigDecimal chloeExpectedBalance = chloePerShareReward;
// Her reward-share with Dilbert: 25% goes to Dilbert
BigDecimal dilbertSharePercent = BigDecimal.valueOf(25L);
BigDecimal dilbertExpectedBalance = chloePerShareReward.multiply(dilbertSharePercent).divide(perHundred, RoundingMode.DOWN);
// The remaining 75% goes to Chloe
BigDecimal rewardShareRemaining = chloePerShareReward.subtract(dilbertExpectedBalance);
chloeExpectedBalance = chloeExpectedBalance.add(rewardShareRemaining);
AccountUtils.assertBalance(repository, "chloe", Asset.QORT, chloeExpectedBalance);
}
}
}

View File

@@ -0,0 +1,63 @@
{
"isTestChain": true,
"blockTimestampMargin": 500,
"transactionExpiryPeriod": 86400000,
"maxBlockSize": 2097152,
"maxBytesPerUnitFee": 1024,
"unitFee": "0.1",
"requireGroupForApproval": false,
"minAccountLevelToRewardShare": 5,
"maxRewardSharesPerMintingAccount": 20,
"founderEffectiveMintingLevel": 10,
"onlineAccountSignaturesMinLifetime": 3600000,
"onlineAccountSignaturesMaxLifetime": 86400000,
"rewardsByHeight": [
{ "height": 1, "reward": 100 },
{ "height": 11, "reward": 10 },
{ "height": 21, "reward": 1 }
],
"sharesByLevel": [
{ "levels": [ 1, 2 ], "share": 0.05 },
{ "levels": [ 3, 4 ], "share": 0.10 },
{ "levels": [ 5, 6 ], "share": 0.15 },
{ "levels": [ 7, 8 ], "share": 0.20 },
{ "levels": [ 9, 10 ], "share": 0.25 }
],
"qoraHoldersShare": 0.20,
"qoraPerQortReward": 250,
"blocksNeededByLevel": [ 10, 20, 30, 40, 50, 60, 70, 80, 90, 100 ],
"blockTimingsByHeight": [
{ "height": 1, "target": 60000, "deviation": 30000, "power": 0.2 }
],
"featureTriggers": {
"messageHeight": 0,
"atHeight": 0,
"assetsTimestamp": 0,
"votingTimestamp": 0,
"arbitraryTimestamp": 0,
"powfixTimestamp": 0,
"qortalTimestamp": 0,
"newAssetPricingTimestamp": 0,
"groupApprovalTimestamp": 0
},
"genesisInfo": {
"version": 4,
"timestamp": 0,
"transactions": [
{ "type": "ISSUE_ASSET", "owner": "QcFmNxSArv5tWEzCtTKb2Lqc5QkKuQ7RNs", "assetName": "QORT", "description": "QORT native coin", "data": "", "quantity": 0, "isDivisible": true, "fee": 0, "reference": "3Verk6ZKBJc3WTTVfxFC9icSjKdM8b92eeJEpJP8qNizG4ZszNFq8wdDYdSjJXq2iogDFR1njyhsBdVpbvDfjzU7" },
{ "type": "ISSUE_ASSET", "owner": "QUwGVHPPxJNJ2dq95abQNe79EyBN2K26zM", "assetName": "Legacy-QORA", "description": "Representative legacy QORA", "quantity": 0, "isDivisible": true, "data": "{}", "isUnspendable": true },
{ "type": "ISSUE_ASSET", "owner": "QUwGVHPPxJNJ2dq95abQNe79EyBN2K26zM", "assetName": "QORT-from-QORA", "description": "QORT gained from holding legacy QORA", "quantity": 0, "isDivisible": true, "data": "{}", "isUnspendable": true },
{ "type": "ACCOUNT_FLAGS", "target": "QgV4s3xnzLhVBEJxcYui4u4q11yhUHsd9v", "andMask": -1, "orMask": 1, "xorMask": 0 },
{ "type": "REWARD_SHARE", "minterPublicKey": "2tiMr5LTpaWCgbRvkPK8TFd7k63DyHJMMFFsz9uBf1ZP", "recipient": "QgV4s3xnzLhVBEJxcYui4u4q11yhUHsd9v", "rewardSharePublicKey": "7PpfnvLSG7y4HPh8hE7KoqAjLCkv7Ui6xw4mKAkbZtox", "sharePercent": 100 },
{ "type": "ACCOUNT_FLAGS", "target": "QixPbJUwsaHsVEofJdozU9zgVqkK6aYhrK", "andMask": -1, "orMask": 1, "xorMask": 0 },
{ "type": "REWARD_SHARE", "minterPublicKey": "C6wuddsBV3HzRrXUtezE7P5MoRXp5m3mEDokRDGZB6ry", "recipient": "QixPbJUwsaHsVEofJdozU9zgVqkK6aYhrK", "rewardSharePublicKey": "CcABzvk26TFEHG7Yok84jxyd4oBtLkx8RJdGFVz2csvp", "sharePercent": 100 },
{ "type": "ACCOUNT_FLAGS", "target": "QaUpHNhT3Ygx6avRiKobuLdusppR5biXjL", "andMask": -1, "orMask": 1, "xorMask": 0 },
{ "type": "REWARD_SHARE", "minterPublicKey": "7KNBj2MnEb6zq1vvKY1q8G2Voctcc2Z1X4avFyEH2eJC", "recipient": "QaUpHNhT3Ygx6avRiKobuLdusppR5biXjL", "rewardSharePublicKey": "6bnEKqZbsCSWryUQnbBT9Umufdu3CapFvxfAni6afhFb", "sharePercent": 100 },
{ "type": "REWARD_SHARE", "minterPublicKey": "7KNBj2MnEb6zq1vvKY1q8G2Voctcc2Z1X4avFyEH2eJC", "recipient": "Qci5m9k4rcwe4ruKrZZQKka4FzUUMut3er", "rewardSharePublicKey": "Hebh14YXUdJA66Vq8KyffNXHx3NSDUAZaNH9qbfEvf5M", "sharePercent": 25 }
]
}
}

View File

@@ -0,0 +1,6 @@
{
"restrictedApi": false,
"blockchainConfig": "src/test/resources/test-chain-v2-founder-rewards.json",
"wipeUnconfirmedOnStart": false,
"minPeers": 0
}

View File

@@ -2,5 +2,6 @@
"restrictedApi": false,
"blockchainConfig": "src/test/resources/test-chain-v2.json",
"wipeUnconfirmedOnStart": false,
"testNtpOffset": 0,
"minPeers": 0
}

35
stop.sh
View File

@@ -1,10 +1,41 @@
#!/bin/sh
#!/usr/bin/env bash
# Check for color support
if [ -t 1 ]; then
ncolors=$( tput colors )
if [ -n "${ncolors}" -a "${ncolors}" -ge 8 ]; then
if normal="$( tput sgr0 )"; then
# use terminfo names
red="$( tput setaf 1 )"
green="$( tput setaf 2)"
else
# use termcap names for FreeBSD compat
normal="$( tput me )"
red="$( tput AF 1 )"
green="$( tput AF 2)"
fi
fi
fi
# Track the pid if we can find it
read pid 2>/dev/null <run.pid
is_pid_valid=$?
echo 'Calling GET /admin/stop on local Qortal node'
if curl --url http://localhost:12391/admin/stop 1>/dev/null 2>&1; then
echo "Qortal node responded and should be shutting down"
if [ "${is_pid_valid}" -eq 0 ]; then
echo -n "Monitoring for Qortal node to end"
while s=`ps -p $pid -o stat=` && [[ "$s" && "$s" != 'Z' ]]; do
echo -n .
sleep 1
done
echo
echo "${green}Qortal ended gracefully${normal}"
rm -f run.pid
fi
exit 0
else
echo "No response from Qortal node - not running?"
echo "${red}No response from Qortal node - not running?${normal}"
exit 1
fi

57
tools/build-zip.sh Executable file
View File

@@ -0,0 +1,57 @@
#!/usr/bin/env bash
set -e
saved_pwd=$PWD
# Check we are within a git repo
git_dir=$( git rev-parse --show-toplevel )
if [ -z "${git_dir}" ]; then
echo "Cannot determine top-level directory for git repo"
exit 1
fi
# Change to git top-level
cd ${git_dir}
# Check we are in 'master' branch
branch_name=$( git symbolic-ref -q HEAD )
branch_name=${branch_name##refs/heads/}
echo "Current git branch: ${branch_name}"
if [ "${branch_name}" != "master" ]; then
echo "Unexpected current branch '${branch_name}' - expecting 'master'"
exit 1
fi
# Determine project name
project=$( perl -n -e 'if (m/<artifactId>(\w+)<.artifactId>/) { print $1; exit }' pom.xml $)
if [ -z "${project}" ]; then
echo "Unable to determine project name from pom.xml?"
exit 1
fi
# Extract git tag
git_tag=$( git tag --points-at HEAD )
if [ -z "${git_tag}" ]; then
echo "Unable to extract git tag"
exit 1
fi
build_dir=/tmp/${project}
commit_ts=$( git show --no-patch --format=%cI )
/bin/rm -fr ${build_dir}
mkdir -p ${build_dir}
cp target/${project}*.jar ${build_dir}/${project}.jar
git show HEAD:log4j2.properties > ${build_dir}/log4j2.properties
git show HEAD:run.sh > ${build_dir}/run.sh
printf "{\n}\n" > ${build_dir}/settings.json
touch -d ${commit_ts%%+??:??} ${build_dir} ${build_dir}/*
rm -f ${saved_pwd}/${project}.zip
(cd ${build_dir}/..; 7z a -r -tzip ${saved_pwd}/${project}-${git_tag#v}.zip ${project}/)

View File

@@ -86,11 +86,11 @@ die("Can't convert base58 public key to hex:\n$pubkey_hex\n") unless $pubkey_hex
printf "Public key hex: %s\n", $pubkey_hex;
my $address = `curl --silent --url http://localhost:${port}/addresses/convert/${pubkey}`;
die("Can't convert base58 public key to address:\n$address\n") unless $address =~ m/^\w{34}$/;
die("Can't convert base58 public key to address:\n$address\n") unless $address =~ m/^\w{33,34}$/;
printf "Address: %s\n", $address;
my $reference = `curl --silent --url http://localhost:${port}/addresses/lastreference/${address}`;
die("Can't fetch last reference for $address:\n$reference\n") unless $reference =~ m/^\w{88}$/;
die("Can't fetch last reference for $address:\n$reference\n") unless $reference =~ m/^\w{87,88}$/;
printf "Last reference: %s\n", $reference;
my $reference_hex = `curl --silent --url http://localhost:${port}/utils/frombase58 --data ${reference}`;