Fixed Synchronizer.getBlockSummaries() which was expecting BLOCK_SUMMARIES, but updated peers send BLOCK_SUMMARIES_V2

Touches quite a few files because:

* Deprecate HEIGHT_V2 because it doesn't contain enough info to be fully useful during sync.
Newer peers will re-use BLOCK_SUMMARIES_V2.

* For newer peers, instead of sending / broadcasting HEIGHT_V2,
send top N block summaries instead, to avoid requests for minor reorgs.

* When responding to GET_BLOCK, and we don't actually have the requested block,
we currently send an empty BLOCK_SUMMARIES message instead of not responding,
which would cause a slow timeout in Synchronizer.

This pattern has spread to other network message response code,
so now we introduce a generic 'unknown' message type for all these cases.

* Remove PeerChainTipData class entirely and re-use BlockSummaryData instead.

* Each Peer instance used to hold PeerChainTipData - essentially single latest block summary - but now holds a List of latest block summaries.

* PeerChainTipData getter/setter methods modified for compatibility at this point in time.

* Repository methods that return BlockSummaryData (or lists of) now try to fully populate them,
including newly added block reference field.

* Re-worked Peer.canUseCommonBlockData() to be more readable

* Cherry-picked patch to Message.fromByteBuffer() to pass an empty, read-only ByteBuffer to subclass fromByteBuffer() methods, instead of null.
This allows natural use of BufferUnderflowException if a subclass tries to use read(), or hasRemaining(), etc. from an empty data-payload message.
Previously this could have caused an NPE.

.gitignore

@@ -28,6 +28,7 @@
 /WindowsInstaller/Install Files/qortal.jar
 /*.7z
 /tmp
+/wallets
 /data*
 /src/test/resources/arbitrary/*/.qortal/cache
 apikey.txt

Dockerfile

@@ -0,0 +1,26 @@
+FROM maven:3-openjdk-11 as builder
+
+WORKDIR /work
+COPY ./ /work/
+RUN mvn clean package
+
+###
+FROM openjdk:11
+
+RUN useradd -r -u 1000 -g users qortal && \
+    mkdir /usr/local/qortal /qortal && \
+    chown 1000:100 /qortal
+
+COPY --from=builder /work/log4j2.properties /usr/local/qortal/
+COPY --from=builder /work/target/qortal*.jar /usr/local/qortal/qortal.jar
+
+USER 1000:100
+
+EXPOSE 12391 12392
+HEALTHCHECK --start-period=5m CMD curl -sf http://127.0.0.1:12391/admin/info || exit 1
+
+WORKDIR /qortal
+VOLUME /qortal
+
+ENTRYPOINT ["java"]
+CMD ["-Djava.net.preferIPv4Stack=false", "-jar", "/usr/local/qortal/qortal.jar"]

README.md

@@ -1,10 +1,4 @@
-# Qortal Data Node
-
-## Important
-
-This code is unfinished, and we haven't had the official genesis block for the data chain yet.
-Therefore it is only possible to use this code if you first create your own test chain. I would
-highly recommend waiting until the code is in a more complete state before trying to run this.
+# Qortal Project - Official Repo
 
 ## Build / run
 

TestNets.md

@@ -41,13 +41,39 @@
 - Start up at least as many nodes as `minBlockchainPeers` (or adjust this value instead)
 - Probably best to perform API call `DELETE /peers/known`
 - Add other nodes via API call `POST /peers <peer-hostname-or-IP>`
-- Add minting private key to node(s) via API call `POST /admin/mintingaccounts <minting-private-key>`
-	This key must have corresponding `REWARD_SHARE` transaction in testnet genesis block
+- Add minting private key to nodes via API call `POST /admin/mintingaccounts <minting-private-key>`
+    The keys must have corresponding `REWARD_SHARE` transactions in testnet genesis block
+- You must have at least 2 separate minting keys and two separate nodes. Assign one minting key to each node.
+- Alternatively, comment out the `if (mintedLastBlock) { }` conditional in BlockMinter.java to allow for a single node and key.
 - Wait for genesis block timestamp to pass
 - A node should mint block 2 approximately 60 seconds after genesis block timestamp
 - Other testnet nodes will sync *as long as there is at least `minBlockchainPeers` peers with an "up-to-date" chain`
 - You can also use API call `POST /admin/forcesync <connected-peer-IP-and-port>` on stuck nodes
 
+## Single-node testnet
+
+A single-node testnet is possible with code modifications, for basic testing, or to more easily start a new testnet.
+To do so, follow these steps:
+- Comment out the `if (mintedLastBlock) { }` conditional in BlockMinter.java
+- Comment out the `minBlockchainPeers` validation in Settings.validate()
+- Set `minBlockchainPeers` to 0 in settings.json
+- Set `Synchronizer.RECOVERY_MODE_TIMEOUT` to `0`
+- All other steps should remain the same. Only a single reward share key is needed.
+- Remember to put these values back after introducing other nodes
+
+## Fixed network
+
+To restrict a testnet to a set of private nodes, you can use the "fixed network" feature.
+This ensures that the testnet nodes only communicate with each other and not other known peers.
+To do this, add the following setting to each testnet node, substituting the IP addresses:
+```
+"fixedNetwork": [
+  "192.168.0.101:62392",
+  "192.168.0.102:62392",
+  "192.168.0.103:62392"
+]
+```
+
 ## Dealing with stuck chain
 
 Maybe your nodes have been offline and no-one has minted a recent testnet block.

WindowsInstaller/Qortal.aip

@@ -2,7 +2,7 @@
 <DOCUMENT Type="Advanced Installer" CreateVersion="14.9" version="18.2" Modules="enterprise" RootPath="." Language="en_GB" Id="{713E21E0-28FC-422F-8A95-823D01A5F80B}">
   <COMPONENT cid="caphyon.advinst.msicomp.MsiPropsComponent">
     <ROW Property="AI_BITMAP_DISPLAY_MODE" Value="0"/>
-    <ROW Property="AI_CURRENT_YEAR" Value="2021" ValueLocId="-"/>
+    <ROW Property="AI_CURRENT_YEAR" Value="2022" ValueLocId="-"/>
     <ROW Property="AI_FINDEXE_TITLE" Value="Select the installation package for [|ProductName]" ValueLocId="AI.Property.FindExeTitle"/>
     <ROW Property="AI_PRODUCTNAME_ARP" Value="Qortal"/>
     <ROW Property="AI_RUN_AS_ADMIN" Value="0"/>
@@ -17,10 +17,10 @@
     <ROW Property="Manufacturer" Value="Qortal"/>
     <ROW Property="MsiLogging" MultiBuildValue="DefaultBuild:vp"/>
     <ROW Property="NTP_GOOD" Value="false"/>
-    <ROW Property="ProductCode" Value="1033:{51EFA0B0-C304-4043-AF6D-2C17C783A998} 1049:{C4662BB2-A247-426E-A128-B7DBD12ECE78} 2052:{1AF44520-C8AB-4261-BCFE-EEA941439718} 2057:{C096EB6A-F43F-45EE-921C-D20F9B993E80} " Type="16"/>
+    <ROW Property="ProductCode" Value="1033:{E5597539-098E-4BA6-99DF-4D22018BC0D3} 1049:{2B5E55A2-142A-4BED-B3B9-5657162282B7} 2052:{6F19171F-4743-4127-B191-AAFA3FA885D2} 2057:{A1B3108D-EC5D-47A1-AEE4-DBD956E682FB} " Type="16"/>
     <ROW Property="ProductLanguage" Value="2057"/>
     <ROW Property="ProductName" Value="Qortal"/>
-    <ROW Property="ProductVersion" Value="2.1.3" Type="32"/>
+    <ROW Property="ProductVersion" Value="3.4.3" Type="32"/>
     <ROW Property="RECONFIG_NTP" Value="true"/>
     <ROW Property="REMOVE_BLOCKCHAIN" Value="YES" Type="4"/>
     <ROW Property="REPAIR_BLOCKCHAIN" Value="YES" Type="4"/>
@@ -212,7 +212,7 @@
     <ROW Component="ADDITIONAL_LICENSE_INFO_71" ComponentId="{12A3ADBE-BB7A-496C-8869-410681E6232F}" Directory_="jdk.zipfs_Dir" Attributes="0" KeyPath="ADDITIONAL_LICENSE_INFO_71" Type="0"/>
     <ROW Component="ADDITIONAL_LICENSE_INFO_8" ComponentId="{D53AD95E-CF96-4999-80FC-5812277A7456}" Directory_="java.naming_Dir" Attributes="0" KeyPath="ADDITIONAL_LICENSE_INFO_8" Type="0"/>
     <ROW Component="ADDITIONAL_LICENSE_INFO_9" ComponentId="{6B7EA9B0-5D17-47A8-B78C-FACE86D15E01}" Directory_="java.net.http_Dir" Attributes="0" KeyPath="ADDITIONAL_LICENSE_INFO_9" Type="0"/>
-    <ROW Component="AI_CustomARPName" ComponentId="{9A243E53-8FC9-4854-B6E2-937493305BB4}" Directory_="APPDIR" Attributes="260" KeyPath="DisplayName" Options="1"/>
+    <ROW Component="AI_CustomARPName" ComponentId="{F17029E8-CCC4-456D-B4AC-1854C81C46B6}" Directory_="APPDIR" Attributes="260" KeyPath="DisplayName" Options="1"/>
     <ROW Component="AI_ExePath" ComponentId="{3644948D-AE0B-41BB-9FAF-A79E70490A08}" Directory_="APPDIR" Attributes="260" KeyPath="AI_ExePath"/>
     <ROW Component="APPDIR" ComponentId="{680DFDDE-3FB4-47A5-8FF5-934F576C6F91}" Directory_="APPDIR" Attributes="0"/>
     <ROW Component="AccessBridgeCallbacks.h" ComponentId="{288055D1-1062-47A3-AA44-5601B4E38AED}" Directory_="bridge_Dir" Attributes="0" KeyPath="AccessBridgeCallbacks.h" Type="0"/>
@@ -1173,7 +1173,7 @@
     <ROW Action="AI_STORE_LOCATION" Type="51" Source="ARPINSTALLLOCATION" Target="[APPDIR]"/>
     <ROW Action="AI_SetPermissions" Type="11265" Source="userAccounts.dll" Target="OnSetPermissions" WithoutSeq="true"/>
     <ROW Action="CustomizeLog4j2PropertiesScript" Type="3109" Target="Script Text" TargetUnformatted="var actionData = Session.Property(&quot;CustomActionData&quot;);&#13;&#10;var actionDataArray = actionData.split(&quot;|&quot;);&#13;&#10;var appDir = actionDataArray[0];&#13;&#10;var dataFolder = actionDataArray[1] + actionDataArray[2] + &quot;\\&quot;;&#13;&#10;&#13;&#10;var ForReading = 1, ForWriting = 2, ForAppending = 8;&#13;&#10;var fso = new ActiveXObject(&quot;Scripting.FileSystemObject&quot;);&#13;&#10;&#13;&#10;// Make copy&#13;&#10;fso.CopyFile(appDir + &quot;log4j2.properties&quot;, appDir + &quot;log4j2-orig.properties&quot;, true); // overwrite&#13;&#10;&#13;&#10;// Rewrite %AppDir%\log4j2.properties to update logfile storage path&#13;&#10;var fin = fso.OpenTextFile(appDir + &quot;log4j2-orig.properties&quot;, ForReading, false); // no create&#13;&#10;var fout = fso.OpenTextFile(appDir + &quot;log4j2.properties&quot;, ForWriting, true); // can create&#13;&#10;&#13;&#10;// Copy lines with rewriting where necessary&#13;&#10;while( !fin.AtEndOfStream ) {&#13;&#10;&#9;var line = fin.ReadLine();&#13;&#10;&#13;&#10;&#9;var start = line.indexOf(&quot;property.dirname&quot;);&#13;&#10;&#9;if (start &gt; 0) {&#13;&#10;&#9;&#9;// line: # property.dirname = ...appdata...&#13;&#10;&#9;&#9;// uncomment/replace this line for Windows&#13;&#10;&#9;&#9;fout.WriteLine( &quot;property.dirname = &quot; + dataFolder.split(&apos;\\&apos;).join(&apos;\\\\&apos;) );&#13;&#10;&#9;} else {&#13;&#10;&#9;&#9;// not found - output verbatim&#13;&#10;&#9;&#9;fout.WriteLine( line );&#13;&#10;&#9;}&#13;&#10;}&#13;&#10;&#13;&#10;fin.Close();&#13;&#10;fout.Close();&#13;&#10;" AdditionalSeq="AI_DATA_SETTER_4"/>
-    <ROW Action="CustomizeSettingsJsonScript" Type="3109" Target="Script Text" TargetUnformatted="var actionData = Session.Property(&quot;CustomActionData&quot;);&#13;&#10;var actionDataArray = actionData.split(&quot;|&quot;);&#13;&#10;var appDir = actionDataArray[0];&#13;&#10;var dataFolder = actionDataArray[1] + actionDataArray[2] + &quot;\\&quot;;&#13;&#10;&#13;&#10;var ForReading = 1, ForWriting = 2, ForAppending = 8;&#13;&#10;var fso = new ActiveXObject(&quot;Scripting.FileSystemObject&quot;);&#13;&#10;&#13;&#10;// Create basic %APPDIR%\settings.json with path to real settings.json in dataFolder&#13;&#10;var fts = fso.OpenTextFile(appDir + &quot;settings.json&quot;, ForWriting, true);&#13;&#10;&#13;&#10;fts.WriteLine( &quot;{&quot; );&#13;&#10;// We need to escape Windows path backslashes to keep JSON valid&#13;&#10;fts.WriteLine( &quot;  \&quot;userPath\&quot;: \&quot;&quot; + dataFolder.split(&apos;\\&apos;).join(&apos;\\\\&apos;) + &quot;\&quot;&quot; );&#13;&#10;fts.WriteLine( &quot;}&quot; );&#13;&#10;&#13;&#10;fts.Close();&#13;&#10;&#13;&#10;// Make copy&#13;&#10;fso.CopyFile(dataFolder + &quot;settings.json&quot;, dataFolder + &quot;settings-orig.json&quot;, true); // overwrite&#13;&#10;&#13;&#10;// Rewrite settings.json to update repository path&#13;&#10;var fin = fso.OpenTextFile(dataFolder + &quot;settings-orig.json&quot;, ForReading, false);&#13;&#10;var fout = fso.OpenTextFile(dataFolder + &quot;settings.json&quot;, ForWriting, true);&#13;&#10;&#13;&#10;// First line should contain opening brace&#13;&#10;fout.WriteLine( fin.ReadLine() );&#13;&#10;&#13;&#10;// Append our entries&#13;&#10;fout.WriteLine( &quot;  \&quot;repositoryPath\&quot;: \&quot;&quot; + dataFolder.split(&apos;\\&apos;).join(&apos;\\\\&apos;) + &quot;db\&quot;,&quot; );&#13;&#10;&#13;&#10;// copy rest of settings&#13;&#10;while( !fin.AtEndOfStream ) {&#13;&#10;&#9;fout.WriteLine( fin.ReadLine() );&#13;&#10;}&#13;&#10;&#13;&#10;fin.Close();&#13;&#10;fout.Close();&#13;&#10;" AdditionalSeq="AI_DATA_SETTER_3"/>
+    <ROW Action="CustomizeSettingsJsonScript" Type="3109" Target="Script Text" TargetUnformatted="var actionData = Session.Property(&quot;CustomActionData&quot;);&#13;&#10;var actionDataArray = actionData.split(&quot;|&quot;);&#13;&#10;var appDir = actionDataArray[0];&#13;&#10;var dataFolder = actionDataArray[1] + actionDataArray[2] + &quot;\\&quot;;&#13;&#10;&#13;&#10;var ForReading = 1, ForWriting = 2, ForAppending = 8;&#13;&#10;var fso = new ActiveXObject(&quot;Scripting.FileSystemObject&quot;);&#13;&#10;&#13;&#10;// Create basic %APPDIR%\settings.json with path to real settings.json in dataFolder&#13;&#10;var fts = fso.OpenTextFile(appDir + &quot;settings.json&quot;, ForWriting, true);&#13;&#10;&#13;&#10;fts.WriteLine( &quot;{&quot; );&#13;&#10;// We need to escape Windows path backslashes to keep JSON valid&#13;&#10;fts.WriteLine( &quot;  \&quot;userPath\&quot;: \&quot;&quot; + dataFolder.split(&apos;\\&apos;).join(&apos;\\\\&apos;) + &quot;\&quot;&quot; );&#13;&#10;fts.WriteLine( &quot;}&quot; );&#13;&#10;&#13;&#10;fts.Close();&#13;&#10;&#13;&#10;// Make copy&#13;&#10;fso.CopyFile(dataFolder + &quot;settings.json&quot;, dataFolder + &quot;settings-orig.json&quot;, true); // overwrite&#13;&#10;&#13;&#10;// Rewrite settings.json to update repository path&#13;&#10;var fin = fso.OpenTextFile(dataFolder + &quot;settings-orig.json&quot;, ForReading, false);&#13;&#10;var fout = fso.OpenTextFile(dataFolder + &quot;settings.json&quot;, ForWriting, true);&#13;&#10;&#13;&#10;// First line should contain opening brace&#13;&#10;fout.WriteLine( fin.ReadLine() );&#13;&#10;&#13;&#10;// Append our entries&#13;&#10;fout.WriteLine( &quot;  \&quot;repositoryPath\&quot;: \&quot;&quot; + dataFolder.split(&apos;\\&apos;).join(&apos;\\\\&apos;) + &quot;db\&quot;,&quot; );&#13;&#10;fout.WriteLine( &quot;  \&quot;dataPath\&quot;: \&quot;&quot; + dataFolder.split(&apos;\\&apos;).join(&apos;\\\\&apos;) + &quot;data\&quot;,&quot; );&#13;&#10;&#13;&#10;// copy rest of settings&#13;&#10;while( !fin.AtEndOfStream ) {&#13;&#10;&#9;fout.WriteLine( fin.ReadLine() );&#13;&#10;}&#13;&#10;&#13;&#10;fin.Close();&#13;&#10;fout.Close();&#13;&#10;" AdditionalSeq="AI_DATA_SETTER_3"/>
     <ROW Action="DetectRunningProcess" Type="1" Source="aicustact.dll" Target="DetectProcess" Options="3" AdditionalSeq="AI_DATA_SETTER_8"/>
     <ROW Action="DetectW32Time" Type="1" Source="aicustact.dll" Target="DetectService" Options="3" AdditionalSeq="AI_DATA_SETTER_11"/>
     <ROW Action="NTP_config" Type="3090" Source="ntpcfg.bat"/>

lib/com/dosse/WaifUPnP/1.1/WaifUPnP-1.1.pom

@@ -0,0 +1,9 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
+    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
+  <modelVersion>4.0.0</modelVersion>
+  <groupId>com.dosse</groupId>
+  <artifactId>WaifUPnP</artifactId>
+  <version>1.1</version>
+  <description>POM was created from install:install-file</description>
+</project>

lib/com/dosse/WaifUPnP/maven-metadata-local.xml

@@ -0,0 +1,12 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<metadata>
+  <groupId>com.dosse</groupId>
+  <artifactId>WaifUPnP</artifactId>
+  <versioning>
+    <release>1.1</release>
+    <versions>
+      <version>1.1</version>
+    </versions>
+    <lastUpdated>20220218200127</lastUpdated>
+  </versioning>
+</metadata>

pom.xml

@@ -3,13 +3,13 @@
 	<modelVersion>4.0.0</modelVersion>
 	<groupId>org.qortal</groupId>
 	<artifactId>qortal</artifactId>
-	<version>3.0.0</version>
+	<version>3.6.0</version>
 	<packaging>jar</packaging>
 	<properties>
 		<skipTests>true</skipTests>
-		<altcoinj.version>bf9fb80</altcoinj.version>
+		<altcoinj.version>7dc8c6f</altcoinj.version>
 		<bitcoinj.version>0.15.10</bitcoinj.version>
-		<bouncycastle.version>1.64</bouncycastle.version>
+		<bouncycastle.version>1.69</bouncycastle.version>
 		<build.timestamp>${maven.build.timestamp}</build.timestamp>
 		<ciyam-at.version>1.3.8</ciyam-at.version>
 		<commons-net.version>3.6</commons-net.version>
@@ -21,9 +21,12 @@
 		<dagger.version>1.2.2</dagger.version>
 		<guava.version>28.1-jre</guava.version>
 		<hsqldb.version>2.5.1</hsqldb.version>
+		<homoglyph.version>1.2.1</homoglyph.version>
+		<icu4j.version>70.1</icu4j.version>
+		<upnp.version>1.1</upnp.version>
 		<jersey.version>2.29.1</jersey.version>
 		<jetty.version>9.4.29.v20200521</jetty.version>
-		<log4j.version>2.12.1</log4j.version>
+		<log4j.version>2.17.1</log4j.version>
 		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
 		<slf4j.version>1.7.12</slf4j.version>
 		<swagger-api.version>2.0.9</swagger-api.version>
@@ -31,6 +34,8 @@
 		<package-info-maven-plugin.version>1.1.0</package-info-maven-plugin.version>
 		<jsoup.version>1.13.1</jsoup.version>
 		<java-diff-utils.version>4.10</java-diff-utils.version>
+		<grpc.version>1.45.1</grpc.version>
+		<protobuf.version>3.19.4</protobuf.version>
 	</properties>
 	<build>
 		<sourceDirectory>src/main/java</sourceDirectory>
@@ -427,6 +432,12 @@
 			<artifactId>AT</artifactId>
 			<version>${ciyam-at.version}</version>
 		</dependency>
+		<!-- UPnP support -->
+		<dependency>
+			<groupId>com.dosse</groupId>
+			<artifactId>WaifUPnP</artifactId>
+			<version>${upnp.version}</version>
+		</dependency>
 		<!-- Bitcoin support -->
 		<dependency>
 			<groupId>org.bitcoinj</groupId>
@@ -435,7 +446,7 @@
 		</dependency>
 		<!-- For Litecoin, etc. support, requires bitcoinj -->
 		<dependency>
-			<groupId>com.github.jjos2372</groupId>
+			<groupId>com.github.qortal</groupId>
 			<artifactId>altcoinj</artifactId>
 			<version>${altcoinj.version}</version>
 		</dependency>
@@ -561,7 +572,18 @@
 		<dependency>
 			<groupId>net.codebox</groupId>
 			<artifactId>homoglyph</artifactId>
-			<version>1.2.0</version>
+			<version>${homoglyph.version}</version>
+		</dependency>
+		<!-- Unicode support -->
+		<dependency>
+			<groupId>com.ibm.icu</groupId>
+			<artifactId>icu4j</artifactId>
+			<version>${icu4j.version}</version>
+		</dependency>
+		<dependency>
+			<groupId>com.ibm.icu</groupId>
+			<artifactId>icu4j-charset</artifactId>
+			<version>${icu4j.version}</version>
 		</dependency>
 		<!-- Jetty -->
 		<dependency>
@@ -685,5 +707,25 @@
 			<artifactId>java-diff-utils</artifactId>
 			<version>${java-diff-utils.version}</version>
 		</dependency>
+		<dependency>
+			<groupId>io.grpc</groupId>
+			<artifactId>grpc-netty</artifactId>
+			<version>${grpc.version}</version>
+		</dependency>
+		<dependency>
+			<groupId>io.grpc</groupId>
+			<artifactId>grpc-protobuf</artifactId>
+			<version>${grpc.version}</version>
+		</dependency>
+		<dependency>
+			<groupId>io.grpc</groupId>
+			<artifactId>grpc-stub</artifactId>
+			<version>${grpc.version}</version>
+		</dependency>
+		<dependency>
+			<groupId>com.google.protobuf</groupId>
+			<artifactId>protobuf-java</artifactId>
+			<version>${protobuf.version}</version>
+		</dependency>
 	</dependencies>
 </project>

src/main/java/com/rust/litewalletjni/LiteWalletJni.java

@@ -0,0 +1,100 @@
+/*
+ * Copyright (C) 2009 The Android Open Source Project
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/*
+ * LiteWalletJni code based on https://github.com/PirateNetwork/cordova-plugin-litewallet
+ *
+ * MIT License
+ *
+ * Copyright (c) 2020 Zero Currency Coin
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in all
+ * copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+package com.rust.litewalletjni;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.qortal.controller.PirateChainWalletController;
+
+import java.nio.file.Path;
+import java.nio.file.Paths;
+
+public class LiteWalletJni {
+
+    protected static final Logger LOGGER = LogManager.getLogger(LiteWalletJni.class);
+
+    public static native String initlogging();
+    public static native String initnew(final String serveruri, final String params, final String saplingOutputb64, final String saplingSpendb64);
+    public static native String initfromseed(final String serveruri, final String params, final String seed, final String birthday, final String saplingOutputb64, final String saplingSpendb64);
+    public static native String initfromb64(final String serveruri, final String params, final String datab64, final String saplingOutputb64, final String saplingSpendb64);
+    public static native String save();
+
+    public static native String execute(final String cmd, final String args);
+    public static native String getseedphrase();
+    public static native String getseedphrasefromentropyb64(final String entropy64);
+    public static native String checkseedphrase(final String input);
+
+
+    private static boolean loaded = false;
+
+    public static void loadLibrary() {
+        if (loaded) {
+            return;
+        }
+        String osName = System.getProperty("os.name");
+        String osArchitecture = System.getProperty("os.arch");
+
+        LOGGER.info("OS Name: {}", osName);
+        LOGGER.info("OS Architecture: {}", osArchitecture);
+
+        try {
+            String libFileName = PirateChainWalletController.getRustLibFilename();
+            if (libFileName == null) {
+                LOGGER.info("Library not found for OS: {}, arch: {}", osName, osArchitecture);
+                return;
+            }
+
+            Path libPath = Paths.get(PirateChainWalletController.getRustLibOuterDirectory().toString(), libFileName);
+            System.load(libPath.toAbsolutePath().toString());
+            loaded = true;
+        }
+        catch (UnsatisfiedLinkError e) {
+            LOGGER.info("Unable to load library");
+        }
+    }
+
+    public static boolean isLoaded() {
+        return loaded;
+    }
+
+}

src/main/java/org/qortal/ApplyUpdate.java

@@ -7,18 +7,20 @@ import java.nio.file.Path;
 import java.nio.file.Paths;
 import java.nio.file.StandardCopyOption;
 import java.security.Security;
-import java.util.ArrayList;
-import java.util.Arrays;
-import java.util.List;
+import java.util.*;
+import java.util.stream.Collectors;
 
 import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
 import org.bouncycastle.jce.provider.BouncyCastleProvider;
 import org.bouncycastle.jsse.provider.BouncyCastleJsseProvider;
+import org.qortal.api.ApiKey;
 import org.qortal.api.ApiRequest;
 import org.qortal.controller.AutoUpdate;
 import org.qortal.settings.Settings;
 
+import static org.qortal.controller.AutoUpdate.AGENTLIB_JVM_HOLDER_ARG;
+
 public class ApplyUpdate {
 
 	static {
@@ -38,7 +40,7 @@ public class ApplyUpdate {
 	private static final String JAVA_TOOL_OPTIONS_NAME = "JAVA_TOOL_OPTIONS";
 	private static final String JAVA_TOOL_OPTIONS_VALUE = "-XX:MaxRAMFraction=4";
 
-	private static final long CHECK_INTERVAL = 10 * 1000L; // ms
+	private static final long CHECK_INTERVAL = 30 * 1000L; // ms
 	private static final int MAX_ATTEMPTS = 12;
 
 	public static void main(String[] args) {
@@ -70,14 +72,40 @@ public class ApplyUpdate {
 		String baseUri = "http://localhost:" + Settings.getInstance().getApiPort() + "/";
 		LOGGER.info(() -> String.format("Shutting down node using API via %s", baseUri));
 
+		// The /admin/stop endpoint requires an API key, which may or may not be already generated
+		boolean apiKeyNewlyGenerated = false;
+		ApiKey apiKey = null;
+		try {
+			apiKey = new ApiKey();
+			if (!apiKey.generated()) {
+				apiKey.generate();
+				apiKeyNewlyGenerated = true;
+				LOGGER.info("Generated API key");
+			}
+		} catch (IOException e) {
+			LOGGER.info("Error loading API key: {}", e.getMessage());
+		}
+
+		// Create GET params
+		Map<String, String> params = new HashMap<>();
+		if (apiKey != null) {
+			params.put("apiKey", apiKey.toString());
+		}
+
+		// Attempt to stop the node
 		int attempt;
 		for (attempt = 0; attempt < MAX_ATTEMPTS; ++attempt) {
 			final int attemptForLogging = attempt;
 			LOGGER.info(() -> String.format("Attempt #%d out of %d to shutdown node", attemptForLogging + 1, MAX_ATTEMPTS));
-			String response = ApiRequest.perform(baseUri + "admin/stop", null);
-			if (response == null)
+			String response = ApiRequest.perform(baseUri + "admin/stop", params);
+			if (response == null) {
 				// No response - consider node shut down
+				if (apiKeyNewlyGenerated) {
+					// API key was newly generated for this auto update, so we need to remove it
+					ApplyUpdate.removeGeneratedApiKey();
+				}
 				return true;
+			}
 
 			LOGGER.info(() -> String.format("Response from API: %s", response));
 
@@ -89,6 +117,11 @@ public class ApplyUpdate {
 			}
 		}
 
+		if (apiKeyNewlyGenerated) {
+			// API key was newly generated for this auto update, so we need to remove it
+			ApplyUpdate.removeGeneratedApiKey();
+		}
+
 		if (attempt == MAX_ATTEMPTS) {
 			LOGGER.error("Failed to shutdown node - giving up");
 			return false;
@@ -97,6 +130,19 @@ public class ApplyUpdate {
 		return true;
 	}
 
+	private static void removeGeneratedApiKey() {
+		try {
+			LOGGER.info("Removing newly generated API key...");
+
+			// Delete the API key since it was only generated for this auto update
+			ApiKey apiKey = new ApiKey();
+			apiKey.delete();
+
+		} catch (IOException e) {
+			LOGGER.info("Error loading or deleting API key: {}", e.getMessage());
+		}
+	}
+
 	private static void replaceJar() {
 		// Assuming current working directory contains the JAR files
 		Path realJar = Paths.get(JAR_FILENAME);
@@ -154,6 +200,11 @@ public class ApplyUpdate {
 			// JVM arguments
 			javaCmd.addAll(ManagementFactory.getRuntimeMXBean().getInputArguments());
 
+			// Reapply any retained, but disabled, -agentlib JVM arg
+			javaCmd = javaCmd.stream()
+					.map(arg -> arg.replace(AGENTLIB_JVM_HOLDER_ARG, "-agentlib"))
+					.collect(Collectors.toList());
+
 			// Call mainClass in JAR
 			javaCmd.addAll(Arrays.asList("-jar", JAR_FILENAME));
 
@@ -162,7 +213,7 @@ public class ApplyUpdate {
 		}
 
 		try {
-			LOGGER.info(() -> String.format("Restarting node with: %s", String.join(" ", javaCmd)));
+			LOGGER.info(String.format("Restarting node with: %s", String.join(" ", javaCmd)));
 
 			ProcessBuilder processBuilder = new ProcessBuilder(javaCmd);
 
@@ -171,8 +222,15 @@ public class ApplyUpdate {
 				processBuilder.environment().put(JAVA_TOOL_OPTIONS_NAME, JAVA_TOOL_OPTIONS_VALUE);
 			}
 
-			processBuilder.start();
-		} catch (IOException e) {
+			// New process will inherit our stdout and stderr
+			processBuilder.redirectOutput(ProcessBuilder.Redirect.INHERIT);
+			processBuilder.redirectError(ProcessBuilder.Redirect.INHERIT);
+
+			Process process = processBuilder.start();
+
+			// Nothing to pipe to new process, so close output stream (process's stdin)
+			process.getOutputStream().close();
+		} catch (Exception e) {
 			LOGGER.error(String.format("Failed to restart node (BAD): %s", e.getMessage()));
 		}
 	}

src/main/java/org/qortal/account/Account.java

@@ -8,11 +8,13 @@ import javax.xml.bind.annotation.XmlAccessorType;
 import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
 import org.qortal.block.BlockChain;
+import org.qortal.controller.LiteNode;
 import org.qortal.data.account.AccountBalanceData;
 import org.qortal.data.account.AccountData;
 import org.qortal.data.account.RewardShareData;
 import org.qortal.repository.DataException;
 import org.qortal.repository.Repository;
+import org.qortal.settings.Settings;
 import org.qortal.utils.Base58;
 
 @XmlAccessorType(XmlAccessType.NONE) // Stops JAX-RS errors when unmarshalling blockchain config
@@ -59,7 +61,17 @@ public class Account {
 	// Balance manipulations - assetId is 0 for QORT
 
 	public long getConfirmedBalance(long assetId) throws DataException {
-		AccountBalanceData accountBalanceData = this.repository.getAccountRepository().getBalance(this.address, assetId);
+		AccountBalanceData accountBalanceData;
+
+		if (Settings.getInstance().isLite()) {
+			// Lite nodes request data from peers instead of the local db
+			accountBalanceData = LiteNode.getInstance().fetchAccountBalance(this.address, assetId);
+		}
+		else {
+			// All other node types fetch from the local db
+			accountBalanceData = this.repository.getAccountRepository().getBalance(this.address, assetId);
+		}
+
 		if (accountBalanceData == null)
 			return 0;
 
@@ -205,6 +217,12 @@ public class Account {
 		return false;
 	}
 
+	/** Returns account's blockMinted (0+) or null if account not found in repository. */
+	public Integer getBlocksMinted() throws DataException {
+		return this.repository.getAccountRepository().getMintedBlockCount(this.address);
+	}
+
+
 	/** Returns whether account can build reward-shares.
 	 * <p>
 	 * To be able to create reward-shares, the account needs to pass at least one of these tests:<br>
@@ -272,7 +290,7 @@ public class Account {
 	/**
 	 * Returns 'effective' minting level, or zero if reward-share does not exist.
 	 * <p>
-	 * For founder accounts, this returns "founderEffectiveMintingLevel" from blockchain config.
+	 * this is being used on src/main/java/org/qortal/api/resource/AddressesResource.java to fulfil the online accounts api call
 	 * 
 	 * @param repository
 	 * @param rewardSharePublicKey
@@ -288,5 +306,26 @@ public class Account {
 		Account rewardShareMinter = new Account(repository, rewardShareData.getMinter());
 		return rewardShareMinter.getEffectiveMintingLevel();
 	}
+	/**
+	 * Returns 'effective' minting level, with a fix for the zero level.
+	 * <p>
+	 * For founder accounts, this returns "founderEffectiveMintingLevel" from blockchain config.
+	 *
+	 * @param repository
+	 * @param rewardSharePublicKey
+	 * @return 0+
+	 * @throws DataException
+	 */
+	public static int getRewardShareEffectiveMintingLevelIncludingLevelZero(Repository repository, byte[] rewardSharePublicKey) throws DataException {
+		// Find actual minter and get their effective minting level
+		RewardShareData rewardShareData = repository.getAccountRepository().getRewardShare(rewardSharePublicKey);
+		if (rewardShareData == null)
+			return 0;
 
+		else if(!rewardShareData.getMinter().equals(rewardShareData.getRecipient()))//the minter is different than the recipient this means sponsorship
+			return 0;
+
+		Account rewardShareMinter = new Account(repository, rewardShareData.getMinter());
+		return rewardShareMinter.getEffectiveMintingLevel();
+	}
 }

src/main/java/org/qortal/account/PrivateKeyAccount.java

@@ -11,15 +11,15 @@ public class PrivateKeyAccount extends PublicKeyAccount {
 	private final Ed25519PrivateKeyParameters edPrivateKeyParams;
 
 	/**
-	 * Create PrivateKeyAccount using byte[32] seed.
+	 * Create PrivateKeyAccount using byte[32] private key.
 	 * 
-	 * @param seed
+	 * @param privateKey
 	 *            byte[32] used to create private/public key pair
 	 * @throws IllegalArgumentException
-	 *             if passed invalid seed
+	 *             if passed invalid privateKey
 	 */
-	public PrivateKeyAccount(Repository repository, byte[] seed) {
-		this(repository, new Ed25519PrivateKeyParameters(seed, 0));
+	public PrivateKeyAccount(Repository repository, byte[] privateKey) {
+		this(repository, new Ed25519PrivateKeyParameters(privateKey, 0));
 	}
 
 	private PrivateKeyAccount(Repository repository, Ed25519PrivateKeyParameters edPrivateKeyParams) {
@@ -37,10 +37,6 @@ public class PrivateKeyAccount extends PublicKeyAccount {
 		return this.privateKey;
 	}
 
-	public static byte[] toPublicKey(byte[] seed) {
-		return new Ed25519PrivateKeyParameters(seed, 0).generatePublicKey().getEncoded();
-	}
-
 	public byte[] sign(byte[] message) {
 		return Crypto.sign(this.edPrivateKeyParams, message);
 	}

src/main/java/org/qortal/api/ApiKey.java

@@ -81,6 +81,15 @@ public class ApiKey {
         writer.close();
     }
 
+    public void delete() throws IOException {
+        this.apiKey = null;
+
+        Path filePath = this.getFilePath();
+        if (Files.exists(filePath)) {
+            Files.delete(filePath);
+        }
+    }
+
 
     public boolean generated() {
         return (this.apiKey != null);

src/main/java/org/qortal/api/ApiService.java

@@ -40,13 +40,7 @@ import org.glassfish.jersey.server.ResourceConfig;
 import org.glassfish.jersey.servlet.ServletContainer;
 import org.qortal.api.resource.AnnotationPostProcessor;
 import org.qortal.api.resource.ApiDefinition;
-import org.qortal.api.websocket.ActiveChatsWebSocket;
-import org.qortal.api.websocket.AdminStatusWebSocket;
-import org.qortal.api.websocket.BlocksWebSocket;
-import org.qortal.api.websocket.ChatMessagesWebSocket;
-import org.qortal.api.websocket.PresenceWebSocket;
-import org.qortal.api.websocket.TradeBotWebSocket;
-import org.qortal.api.websocket.TradeOffersWebSocket;
+import org.qortal.api.websocket.*;
 import org.qortal.settings.Settings;
 
 public class ApiService {
@@ -212,6 +206,9 @@ public class ApiService {
 			context.addServlet(ChatMessagesWebSocket.class, "/websockets/chat/messages");
 			context.addServlet(TradeOffersWebSocket.class, "/websockets/crosschain/tradeoffers");
 			context.addServlet(TradeBotWebSocket.class, "/websockets/crosschain/tradebot");
+			context.addServlet(TradePresenceWebSocket.class, "/websockets/crosschain/tradepresence");
+
+			// Deprecated
 			context.addServlet(PresenceWebSocket.class, "/websockets/presence");
 
 			// Start server

src/main/java/org/qortal/api/HTMLParser.java

@@ -19,14 +19,20 @@ public class HTMLParser {
         this.data = data;
     }
 
-    public void setDocumentBaseUrl() {
+    public void addAdditionalHeaderTags() {
         String fileContents = new String(data);
         Document document = Jsoup.parse(fileContents);
         String baseUrl = this.linkPrefix + "/";
         Elements head = document.getElementsByTag("head");
         if (!head.isEmpty()) {
+            // Add base href tag
             String baseElement = String.format("<base href=\"%s\">", baseUrl);
             head.get(0).prepend(baseElement);
+
+            // Add meta charset tag
+            String metaCharsetElement = "<meta charset=\"UTF-8\">";
+            head.get(0).prepend(metaCharsetElement);
+
         }
         String html = document.html();
         this.data = html.getBytes();

src/main/java/org/qortal/api/domainmap/resource/DomainMapResource.java

@@ -17,7 +17,7 @@ import java.util.Map;
 
 
 @Path("/")
-@Tag(name = "Gateway")
+@Tag(name = "Domain Map")
 public class DomainMapResource {
 
     @Context HttpServletRequest request;

src/main/java/org/qortal/api/gateway/resource/GatewayResource.java

@@ -65,7 +65,7 @@ public class GatewayResource {
         }
 
         ArbitraryDataResource resource = new ArbitraryDataResource(name, ResourceIdType.NAME, service, identifier);
-        return resource.getStatus();
+        return resource.getStatus(false);
     }
 
 

src/main/java/org/qortal/api/model/ConnectedPeer.java

@@ -1,7 +1,7 @@
 package org.qortal.api.model;
 
 import io.swagger.v3.oas.annotations.media.Schema;
-import org.qortal.data.network.PeerChainTipData;
+import org.qortal.data.block.BlockSummaryData;
 import org.qortal.data.network.PeerData;
 import org.qortal.network.Handshake;
 import org.qortal.network.Peer;
@@ -63,11 +63,11 @@ public class ConnectedPeer {
             this.age = "connecting...";
         }
 
-        PeerChainTipData peerChainTipData = peer.getChainTipData();
+        BlockSummaryData peerChainTipData = peer.getChainTipData();
         if (peerChainTipData != null) {
-            this.lastHeight = peerChainTipData.getLastHeight();
-            this.lastBlockSignature = peerChainTipData.getLastBlockSignature();
-            this.lastBlockTimestamp = peerChainTipData.getLastBlockTimestamp();
+            this.lastHeight = peerChainTipData.getHeight();
+            this.lastBlockSignature = peerChainTipData.getSignature();
+            this.lastBlockTimestamp = peerChainTipData.getTimestamp();
         }
     }
 

src/main/java/org/qortal/api/model/NodeInfo.java

@@ -12,6 +12,7 @@ public class NodeInfo {
 	public long buildTimestamp;
 	public String nodeId;
 	public boolean isTestNet;
+	public String type;
 
 	public NodeInfo() {
 	}

src/main/java/org/qortal/api/model/NodeStatus.java

@@ -4,6 +4,8 @@ import javax.xml.bind.annotation.XmlAccessType;
 import javax.xml.bind.annotation.XmlAccessorType;
 
 import org.qortal.controller.Controller;
+import org.qortal.controller.OnlineAccountsManager;
+import org.qortal.controller.Synchronizer;
 import org.qortal.network.Network;
 
 @XmlAccessorType(XmlAccessType.FIELD)
@@ -20,12 +22,12 @@ public class NodeStatus {
 	public final int height;
 
 	public NodeStatus() {
-		this.isMintingPossible = Controller.getInstance().isMintingPossible();
+		this.isMintingPossible = OnlineAccountsManager.getInstance().hasActiveOnlineAccountSignatures();
 
-		this.syncPercent = Controller.getInstance().getSyncPercent();
-		this.isSynchronizing = this.syncPercent != null;
+		this.syncPercent = Synchronizer.getInstance().getSyncPercent();
+		this.isSynchronizing = Synchronizer.getInstance().isSynchronizing();
 
-		this.numberOfConnections = Network.getInstance().getHandshakedPeers().size();
+		this.numberOfConnections = Network.getInstance().getImmutableHandshakedPeers().size();
 
 		this.height = Controller.getInstance().getChainHeight();
 	}

src/main/java/org/qortal/api/model/crosschain/DigibyteSendRequest.java

@@ -0,0 +1,29 @@
+package org.qortal.api.model.crosschain;
+
+import javax.xml.bind.annotation.XmlAccessType;
+import javax.xml.bind.annotation.XmlAccessorType;
+import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
+
+import io.swagger.v3.oas.annotations.media.Schema;
+
+@XmlAccessorType(XmlAccessType.FIELD)
+public class DigibyteSendRequest {
+
+	@Schema(description = "Digibyte BIP32 extended private key", example = "tprv___________________________________________________________________________________________________________")
+	public String xprv58;
+
+	@Schema(description = "Recipient's Digibyte address ('legacy' P2PKH only)", example = "1DigByteEaterAddressDontSendf59kuE")
+	public String receivingAddress;
+
+	@Schema(description = "Amount of DGB to send", type = "number")
+	@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
+	public long digibyteAmount;
+
+	@Schema(description = "Transaction fee per byte (optional). Default is 0.00000100 DGB (100 sats) per byte", example = "0.00000100", type = "number")
+	@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
+	public Long feePerByte;
+
+	public DigibyteSendRequest() {
+	}
+
+}

src/main/java/org/qortal/api/model/crosschain/PirateChainSendRequest.java

@@ -0,0 +1,32 @@
+package org.qortal.api.model.crosschain;
+
+import io.swagger.v3.oas.annotations.media.Schema;
+
+import javax.xml.bind.annotation.XmlAccessType;
+import javax.xml.bind.annotation.XmlAccessorType;
+import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
+
+@XmlAccessorType(XmlAccessType.FIELD)
+public class PirateChainSendRequest {
+
+	@Schema(description = "32 bytes of entropy, Base58 encoded", example = "5oSXF53qENtdUyKhqSxYzP57m6RhVFP9BJKRr9E5kRGV")
+	public String entropy58;
+
+	@Schema(description = "Recipient's Pirate Chain address", example = "zc...")
+	public String receivingAddress;
+
+	@Schema(description = "Amount of ARRR to send", type = "number")
+	@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
+	public long arrrAmount;
+
+	@Schema(description = "Transaction fee per byte (optional). Default is 0.00000100 ARRR (100 sats) per byte", example = "0.00000100", type = "number")
+	@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
+	public Long feePerByte;
+
+	@Schema(description = "Optional memo to include information for the recipient", example = "zc...")
+	public String memo;
+
+	public PirateChainSendRequest() {
+	}
+
+}

src/main/java/org/qortal/api/model/crosschain/RavencoinSendRequest.java

@@ -0,0 +1,29 @@
+package org.qortal.api.model.crosschain;
+
+import javax.xml.bind.annotation.XmlAccessType;
+import javax.xml.bind.annotation.XmlAccessorType;
+import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
+
+import io.swagger.v3.oas.annotations.media.Schema;
+
+@XmlAccessorType(XmlAccessType.FIELD)
+public class RavencoinSendRequest {
+
+	@Schema(description = "Ravencoin BIP32 extended private key", example = "tprv___________________________________________________________________________________________________________")
+	public String xprv58;
+
+	@Schema(description = "Recipient's Ravencoin address ('legacy' P2PKH only)", example = "1RvnCoinEaterAddressDontSendf59kuE")
+	public String receivingAddress;
+
+	@Schema(description = "Amount of RVN to send", type = "number")
+	@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
+	public long ravencoinAmount;
+
+	@Schema(description = "Transaction fee per byte (optional). Default is 0.00000100 RVN (100 sats) per byte", example = "0.00000100", type = "number")
+	@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
+	public Long feePerByte;
+
+	public RavencoinSendRequest() {
+	}
+
+}

src/main/java/org/qortal/api/resource/AddressesResource.java

@@ -30,7 +30,8 @@ import org.qortal.api.Security;
 import org.qortal.api.model.ApiOnlineAccount;
 import org.qortal.api.model.RewardShareKeyRequest;
 import org.qortal.asset.Asset;
-import org.qortal.controller.Controller;
+import org.qortal.controller.LiteNode;
+import org.qortal.controller.OnlineAccountsManager;
 import org.qortal.crypto.Crypto;
 import org.qortal.data.account.AccountData;
 import org.qortal.data.account.RewardShareData;
@@ -109,18 +110,26 @@ public class AddressesResource {
 		if (!Crypto.isValidAddress(address))
 			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
 
-		byte[] lastReference = null;
+		AccountData accountData;
 
-		try (final Repository repository = RepositoryManager.getRepository()) {
-			AccountData accountData = repository.getAccountRepository().getAccount(address);
-			// Not found?
-			if (accountData == null)
-				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.ADDRESS_UNKNOWN);
-
-			lastReference = accountData.getReference();
-		} catch (DataException e) {
-			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
+		if (Settings.getInstance().isLite()) {
+			// Lite nodes request data from peers instead of the local db
+			accountData = LiteNode.getInstance().fetchAccountData(address);
 		}
+		else {
+			// All other node types request data from local db
+			try (final Repository repository = RepositoryManager.getRepository()) {
+				accountData = repository.getAccountRepository().getAccount(address);
+			} catch (DataException e) {
+				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
+			}
+		}
+
+		// Not found?
+		if (accountData == null)
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.ADDRESS_UNKNOWN);
+
+		byte[] lastReference = accountData.getReference();
 
 		if (lastReference == null || lastReference.length == 0)
 			return "false";
@@ -156,7 +165,7 @@ public class AddressesResource {
 	)
 	@ApiErrors({ApiError.PUBLIC_KEY_NOT_FOUND, ApiError.REPOSITORY_ISSUE})
 	public List<ApiOnlineAccount> getOnlineAccounts() {
-		List<OnlineAccountData> onlineAccounts = Controller.getInstance().getOnlineAccounts();
+		List<OnlineAccountData> onlineAccounts = OnlineAccountsManager.getInstance().getOnlineAccounts();
 
 		// Map OnlineAccountData entries to OnlineAccount via reward-share data
 		try (final Repository repository = RepositoryManager.getRepository()) {
@@ -191,14 +200,14 @@ public class AddressesResource {
 	)
 	@ApiErrors({ApiError.PUBLIC_KEY_NOT_FOUND, ApiError.REPOSITORY_ISSUE})
 	public List<OnlineAccountLevel> getOnlineAccountsByLevel() {
-		List<OnlineAccountData> onlineAccounts = Controller.getInstance().getOnlineAccounts();
+		List<OnlineAccountData> onlineAccounts = OnlineAccountsManager.getInstance().getOnlineAccounts();
 
 		try (final Repository repository = RepositoryManager.getRepository()) {
 			List<OnlineAccountLevel> onlineAccountLevels = new ArrayList<>();
 
 			for (OnlineAccountData onlineAccountData : onlineAccounts) {
 				try {
-					final int minterLevel = Account.getRewardShareEffectiveMintingLevel(repository, onlineAccountData.getPublicKey());
+					final int minterLevel = Account.getRewardShareEffectiveMintingLevelIncludingLevelZero(repository, onlineAccountData.getPublicKey());
 
 					OnlineAccountLevel onlineAccountLevel = onlineAccountLevels.stream()
 							.filter(a -> a.getLevel() == minterLevel)

src/main/java/org/qortal/api/resource/AdminResource.java

@@ -35,7 +35,6 @@ import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
 import org.apache.logging.log4j.core.LoggerContext;
 import org.apache.logging.log4j.core.appender.RollingFileAppender;
-import org.checkerframework.checker.units.qual.A;
 import org.qortal.account.Account;
 import org.qortal.account.PrivateKeyAccount;
 import org.qortal.api.*;
@@ -44,6 +43,7 @@ import org.qortal.api.model.NodeInfo;
 import org.qortal.api.model.NodeStatus;
 import org.qortal.block.BlockChain;
 import org.qortal.controller.Controller;
+import org.qortal.controller.Synchronizer;
 import org.qortal.controller.Synchronizer.SynchronizationResult;
 import org.qortal.data.account.MintingAccountData;
 import org.qortal.data.account.RewardShareData;
@@ -119,10 +119,23 @@ public class AdminResource {
 		nodeInfo.buildTimestamp = Controller.getInstance().getBuildTimestamp();
 		nodeInfo.nodeId = Network.getInstance().getOurNodeId();
 		nodeInfo.isTestNet = Settings.getInstance().isTestNet();
+		nodeInfo.type = getNodeType();
 
 		return nodeInfo;
 	}
 
+	private String getNodeType() {
+		if (Settings.getInstance().isLite()) {
+			return "lite";
+		}
+		else if (Settings.getInstance().isTopOnly()) {
+			return "topOnly";
+		}
+		else {
+			return "full";
+		}
+	}
+
 	@GET
 	@Path("/status")
 	@Operation(
@@ -314,6 +327,7 @@ public class AdminResource {
 
 			repository.getAccountRepository().save(mintingAccountData);
 			repository.saveChanges();
+			repository.exportNodeLocalData();//after adding new minting account let's persist it to the backup  MintingAccounts.json 
 		} catch (IllegalArgumentException e) {
 			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY, e);
 		} catch (DataException e) {
@@ -354,6 +368,7 @@ public class AdminResource {
 				return "false";
 
 			repository.saveChanges();
+			repository.exportNodeLocalData();//after removing new minting account let's persist it to the backup  MintingAccounts.json 
 		} catch (IllegalArgumentException e) {
 			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY, e);
 		} catch (DataException e) {
@@ -379,6 +394,10 @@ public class AdminResource {
 			) @QueryParam("limit") Integer limit, @Parameter(
 				ref = "offset"
 			) @QueryParam("offset") Integer offset, @Parameter(
+				name = "tail",
+				description = "Fetch most recent log lines",
+				schema = @Schema(type = "boolean")
+			) @QueryParam("tail") Boolean tail, @Parameter(
 				ref = "reverse"
 			) @QueryParam("reverse") Boolean reverse) {
 		LoggerContext loggerContext = (LoggerContext) LogManager.getContext();
@@ -394,6 +413,13 @@ public class AdminResource {
 			if (reverse != null && reverse)
 				logLines = Lists.reverse(logLines);
 
+			// Tail mode - return the last X lines (where X = limit)
+			if (tail != null && tail) {
+				if (limit != null && limit > 0) {
+					offset = logLines.size() - limit;
+				}
+			}
+
 			// offset out of bounds?
 			if (offset != null && (offset < 0 || offset >= logLines.size()))
 				return "";
@@ -414,7 +440,7 @@ public class AdminResource {
 
 			limit = Math.min(limit, logLines.size());
 
-			logLines.subList(limit - 1, logLines.size()).clear();
+			logLines.subList(limit, logLines.size()).clear();
 
 			return String.join("\n", logLines);
 		} catch (IOException e) {
@@ -511,7 +537,7 @@ public class AdminResource {
 			PeerAddress peerAddress = PeerAddress.fromString(targetPeerAddress);
 			InetSocketAddress resolvedAddress = peerAddress.toSocketAddress();
 
-			List<Peer> peers = Network.getInstance().getHandshakedPeers();
+			List<Peer> peers = Network.getInstance().getImmutableHandshakedPeers();
 			Peer targetPeer = peers.stream().filter(peer -> peer.getResolvedAddress().equals(resolvedAddress)).findFirst().orElse(null);
 
 			if (targetPeer == null)
@@ -525,7 +551,7 @@ public class AdminResource {
 			SynchronizationResult syncResult;
 			try {
 				do {
-					syncResult = Controller.getInstance().actuallySynchronize(targetPeer, true);
+					syncResult = Synchronizer.getInstance().actuallySynchronize(targetPeer, true);
 				} while (syncResult == SynchronizationResult.OK);
 			} finally {
 				blockchainLock.unlock();
@@ -545,7 +571,7 @@ public class AdminResource {
 	@Path("/repository/data")
 	@Operation(
 		summary = "Export sensitive/node-local data from repository.",
-		description = "Exports data to .script files on local machine"
+		description = "Exports data to .json files on local machine"
 	)
 	@ApiErrors({ApiError.INVALID_DATA, ApiError.REPOSITORY_ISSUE})
 	@SecurityRequirement(name = "apiKey")
@@ -586,10 +612,6 @@ public class AdminResource {
 	public String importRepository(@HeaderParam(Security.API_KEY_HEADER) String apiKey, String filename) {
 		Security.checkApiCallAllowed(request);
 
-		// Hard-coded because it's too dangerous to allow user-supplied filenames in weaker security contexts
-		if (Settings.getInstance().getApiKey() == null)
-			filename = "qortal-backup/TradeBotStates.json";
-
 		try (final Repository repository = RepositoryManager.getRepository()) {
 			ReentrantLock blockchainLock = Controller.getInstance().getBlockchainLock();
 
@@ -706,6 +728,49 @@ public class AdminResource {
 		}
 	}
 
+	@POST
+	@Path("/repository/importarchivedtrades")
+	@Operation(
+			summary = "Imports archived trades from TradeBotStatesArchive.json",
+			description = "This can be used to recover trades that exist in the archive only, which may be needed if a<br />" +
+					"problem occurred during the proof-of-work computation stage of a buy request.",
+			responses = {
+					@ApiResponse(
+							content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "boolean"))
+					)
+			}
+	)
+	@ApiErrors({ApiError.REPOSITORY_ISSUE})
+	@SecurityRequirement(name = "apiKey")
+	public boolean importArchivedTrades(@HeaderParam(Security.API_KEY_HEADER) String apiKey) {
+		Security.checkApiCallAllowed(request);
+
+		try (final Repository repository = RepositoryManager.getRepository()) {
+			ReentrantLock blockchainLock = Controller.getInstance().getBlockchainLock();
+
+			blockchainLock.lockInterruptibly();
+
+			try {
+				repository.importDataFromFile("qortal-backup/TradeBotStatesArchive.json");
+				repository.saveChanges();
+
+				return true;
+
+			} catch (IOException e) {
+				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA, e);
+
+			} finally {
+				blockchainLock.unlock();
+			}
+		} catch (InterruptedException e) {
+			// We couldn't lock blockchain to perform import
+			return false;
+		} catch (DataException e) {
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
+		}
+	}
+
+
 
 	@POST
 	@Path("/apikey/generate")

src/main/java/org/qortal/api/resource/ArbitraryResource.java

@@ -15,6 +15,7 @@ import java.io.*;
 import java.nio.file.Files;
 import java.nio.file.Paths;
 import java.util.ArrayList;
+import java.util.Arrays;
 import java.util.List;
 
 import javax.servlet.ServletContext;
@@ -33,16 +34,18 @@ import org.qortal.api.resource.TransactionsResource.ConfirmationStatus;
 import org.qortal.arbitrary.*;
 import org.qortal.arbitrary.ArbitraryDataFile.ResourceIdType;
 import org.qortal.arbitrary.exception.MissingDataException;
+import org.qortal.arbitrary.metadata.ArbitraryDataTransactionMetadata;
+import org.qortal.arbitrary.misc.Category;
 import org.qortal.arbitrary.misc.Service;
 import org.qortal.controller.Controller;
 import org.qortal.controller.arbitrary.ArbitraryDataStorageManager;
+import org.qortal.controller.arbitrary.ArbitraryMetadataManager;
 import org.qortal.data.account.AccountData;
-import org.qortal.data.arbitrary.ArbitraryResourceInfo;
-import org.qortal.data.arbitrary.ArbitraryResourceNameInfo;
-import org.qortal.data.arbitrary.ArbitraryResourceStatus;
+import org.qortal.data.arbitrary.*;
 import org.qortal.data.naming.NameData;
 import org.qortal.data.transaction.ArbitraryTransactionData;
 import org.qortal.data.transaction.TransactionData;
+import org.qortal.list.ResourceListManager;
 import org.qortal.repository.DataException;
 import org.qortal.repository.Repository;
 import org.qortal.repository.RepositoryManager;
@@ -54,7 +57,9 @@ import org.qortal.transaction.Transaction.ValidationResult;
 import org.qortal.transform.TransformationException;
 import org.qortal.transform.transaction.ArbitraryTransactionTransformer;
 import org.qortal.transform.transaction.TransactionTransformer;
+import org.qortal.utils.ArbitraryTransactionUtils;
 import org.qortal.utils.Base58;
+import org.qortal.utils.NTP;
 import org.qortal.utils.ZipUtils;
 
 @Path("/arbitrary")
@@ -88,7 +93,9 @@ public class ArbitraryResource {
 			@Parameter(ref = "limit") @QueryParam("limit") Integer limit,
 			@Parameter(ref = "offset") @QueryParam("offset") Integer offset,
 			@Parameter(ref = "reverse") @QueryParam("reverse") Boolean reverse,
-			@Parameter(description = "Include status") @QueryParam("includestatus") Boolean includeStatus) {
+			@Parameter(description = "Filter names by list") @QueryParam("namefilter") String nameFilter,
+			@Parameter(description = "Include status") @QueryParam("includestatus") Boolean includeStatus,
+			@Parameter(description = "Include metadata") @QueryParam("includemetadata") Boolean includeMetadata) {
 
 		try (final Repository repository = RepositoryManager.getRepository()) {
 
@@ -103,16 +110,76 @@ public class ArbitraryResource {
 				throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, "identifier cannot be specified when requesting a default resource");
 			}
 
+			// Load filter from list if needed
+			List<String> names = null;
+			if (nameFilter != null) {
+				names = ResourceListManager.getInstance().getStringsInList(nameFilter);
+				if (names.isEmpty()) {
+					// List doesn't exist or is empty - so there will be no matches
+					return new ArrayList<>();
+				}
+			}
+
 			List<ArbitraryResourceInfo> resources = repository.getArbitraryRepository()
-					.getArbitraryResources(service, identifier, null, defaultRes, limit, offset, reverse);
+					.getArbitraryResources(service, identifier, names, defaultRes, limit, offset, reverse);
 
 			if (resources == null) {
 				return new ArrayList<>();
 			}
 
-			if (includeStatus != null && includeStatus == true) {
+			if (includeStatus != null && includeStatus) {
 				resources = this.addStatusToResources(resources);
 			}
+			if (includeMetadata != null && includeMetadata) {
+				resources = this.addMetadataToResources(resources);
+			}
+
+			return resources;
+
+		} catch (DataException e) {
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
+		}
+	}
+
+	@GET
+	@Path("/resources/search")
+	@Operation(
+			summary = "Search arbitrary resources available on chain, optionally filtered by service.\n" +
+					"If default is set to true, only resources without identifiers will be returned.",
+			responses = {
+					@ApiResponse(
+							content = @Content(mediaType = MediaType.APPLICATION_JSON, schema = @Schema(implementation = ArbitraryResourceInfo.class))
+					)
+			}
+	)
+	@ApiErrors({ApiError.REPOSITORY_ISSUE})
+	public List<ArbitraryResourceInfo> searchResources(
+			@QueryParam("service") Service service,
+			@QueryParam("query") String query,
+			@Parameter(description = "Default resources (without identifiers) only") @QueryParam("default") Boolean defaultResource,
+			@Parameter(ref = "limit") @QueryParam("limit") Integer limit,
+			@Parameter(ref = "offset") @QueryParam("offset") Integer offset,
+			@Parameter(ref = "reverse") @QueryParam("reverse") Boolean reverse,
+			@Parameter(description = "Include status") @QueryParam("includestatus") Boolean includeStatus,
+			@Parameter(description = "Include metadata") @QueryParam("includemetadata") Boolean includeMetadata) {
+
+		try (final Repository repository = RepositoryManager.getRepository()) {
+
+			boolean defaultRes = Boolean.TRUE.equals(defaultResource);
+
+			List<ArbitraryResourceInfo> resources = repository.getArbitraryRepository()
+					.searchArbitraryResources(service, query, defaultRes, limit, offset, reverse);
+
+			if (resources == null) {
+				return new ArrayList<>();
+			}
+
+			if (includeStatus != null && includeStatus) {
+				resources = this.addStatusToResources(resources);
+			}
+			if (includeMetadata != null && includeMetadata) {
+				resources = this.addMetadataToResources(resources);
+			}
 
 			return resources;
 
@@ -139,7 +206,8 @@ public class ArbitraryResource {
 			@Parameter(ref = "limit") @QueryParam("limit") Integer limit,
 			@Parameter(ref = "offset") @QueryParam("offset") Integer offset,
 			@Parameter(ref = "reverse") @QueryParam("reverse") Boolean reverse,
-			@Parameter(description = "Include status") @QueryParam("includestatus") Boolean includeStatus) {
+			@Parameter(description = "Include status") @QueryParam("includestatus") Boolean includeStatus,
+			@Parameter(description = "Include metadata") @QueryParam("includemetadata") Boolean includeMetadata) {
 
 		try (final Repository repository = RepositoryManager.getRepository()) {
 
@@ -161,11 +229,15 @@ public class ArbitraryResource {
 				String name = creatorName.name;
 				if (name != null) {
 					List<ArbitraryResourceInfo> resources = repository.getArbitraryRepository()
-							.getArbitraryResources(service, identifier, name, defaultRes, null, null, reverse);
+							.getArbitraryResources(service, identifier, Arrays.asList(name), defaultRes, null, null, reverse);
 
-					if (includeStatus != null && includeStatus == true) {
+					if (includeStatus != null && includeStatus) {
 						resources = this.addStatusToResources(resources);
 					}
+					if (includeMetadata != null && includeMetadata) {
+						resources = this.addMetadataToResources(resources);
+					}
+
 					creatorName.resources = resources;
 				}
 			}
@@ -189,12 +261,13 @@ public class ArbitraryResource {
 			}
 	)
 	@SecurityRequirement(name = "apiKey")
-	public ArbitraryResourceStatus getDefaultResourceStatus(@PathParam("service") Service service,
-													  		 @PathParam("name") String name,
-															 @QueryParam("build") Boolean build) {
+	public ArbitraryResourceStatus getDefaultResourceStatus(@HeaderParam(Security.API_KEY_HEADER) String apiKey,
+															@PathParam("service") Service service,
+															@PathParam("name") String name,
+															@QueryParam("build") Boolean build) {
 
 		Security.requirePriorAuthorizationOrApiKey(request, name, service, null);
-		return this.getStatus(service, name, null, build);
+		return ArbitraryTransactionUtils.getStatus(service, name, null, build);
 	}
 
 	@GET
@@ -209,13 +282,14 @@ public class ArbitraryResource {
 			}
 	)
 	@SecurityRequirement(name = "apiKey")
-	public ArbitraryResourceStatus getResourceStatus(@PathParam("service") Service service,
-													  @PathParam("name") String name,
-													  @PathParam("identifier") String identifier,
-													  @QueryParam("build") Boolean build) {
+	public ArbitraryResourceStatus getResourceStatus(@HeaderParam(Security.API_KEY_HEADER) String apiKey,
+													 @PathParam("service") Service service,
+													 @PathParam("name") String name,
+													 @PathParam("identifier") String identifier,
+													 @QueryParam("build") Boolean build) {
 
 		Security.requirePriorAuthorizationOrApiKey(request, name, service, identifier);
-		return this.getStatus(service, name, identifier, build);
+		return ArbitraryTransactionUtils.getStatus(service, name, identifier, build);
 	}
 
 
@@ -345,6 +419,28 @@ public class ArbitraryResource {
 		return Settings.getInstance().isRelayModeEnabled();
 	}
 
+	@GET
+	@Path("/categories")
+	@Operation(
+			summary = "List arbitrary transaction categories",
+			responses = {
+					@ApiResponse(
+							content = @Content(mediaType = MediaType.APPLICATION_JSON, schema = @Schema(implementation = ArbitraryCategoryInfo.class))
+					)
+			}
+	)
+	@ApiErrors({ApiError.REPOSITORY_ISSUE})
+	public List<ArbitraryCategoryInfo> getCategories() {
+		List<ArbitraryCategoryInfo> categories = new ArrayList<>();
+		for (Category category : Category.values()) {
+			ArbitraryCategoryInfo arbitraryCategory = new ArbitraryCategoryInfo();
+			arbitraryCategory.id = category.toString();
+			arbitraryCategory.name = category.getName();
+			categories.add(arbitraryCategory);
+		}
+		return categories;
+	}
+
 	@GET
 	@Path("/hosted/transactions")
 	@Operation(
@@ -356,12 +452,14 @@ public class ArbitraryResource {
 			}
 	)
 	@ApiErrors({ApiError.REPOSITORY_ISSUE})
-	public List<ArbitraryTransactionData> getHostedTransactions(@HeaderParam(Security.API_KEY_HEADER) String apiKey) {
+	public List<ArbitraryTransactionData> getHostedTransactions(@HeaderParam(Security.API_KEY_HEADER) String apiKey,
+																@Parameter(ref = "limit") @QueryParam("limit") Integer limit,
+																@Parameter(ref = "offset") @QueryParam("offset") Integer offset) {
 		Security.checkApiCallAllowed(request);
 
 		try (final Repository repository = RepositoryManager.getRepository()) {
 
-			List<ArbitraryTransactionData> hostedTransactions = ArbitraryDataStorageManager.getInstance().listAllHostedTransactions(repository);
+			List<ArbitraryTransactionData> hostedTransactions = ArbitraryDataStorageManager.getInstance().listAllHostedTransactions(repository, limit, offset);
 
 			return hostedTransactions;
 
@@ -383,34 +481,41 @@ public class ArbitraryResource {
 	@ApiErrors({ApiError.REPOSITORY_ISSUE})
 	public List<ArbitraryResourceInfo> getHostedResources(
 			@HeaderParam(Security.API_KEY_HEADER) String apiKey,
-			@Parameter(description = "Include status") @QueryParam("includestatus") Boolean includeStatus) {
+			@Parameter(description = "Include status") @QueryParam("includestatus") Boolean includeStatus,
+			@Parameter(description = "Include metadata") @QueryParam("includemetadata") Boolean includeMetadata,
+			@Parameter(ref = "limit") @QueryParam("limit") Integer limit,
+			@Parameter(ref = "offset") @QueryParam("offset") Integer offset,
+			@QueryParam("query") String query) {
 		Security.checkApiCallAllowed(request);
 
 		List<ArbitraryResourceInfo> resources = new ArrayList<>();
 
 		try (final Repository repository = RepositoryManager.getRepository()) {
+			
+			List<ArbitraryTransactionData> transactionDataList;
+
+			if (query == null || query.equals("")) {
+				transactionDataList = ArbitraryDataStorageManager.getInstance().listAllHostedTransactions(repository, limit, offset);
+			} else {
+				transactionDataList = ArbitraryDataStorageManager.getInstance().searchHostedTransactions(repository,query, limit, offset);
+			}
 
-			List<ArbitraryTransactionData> transactionDataList = ArbitraryDataStorageManager.getInstance().listAllHostedTransactions(repository);
 			for (ArbitraryTransactionData transactionData : transactionDataList) {
-				ArbitraryTransaction transaction = new ArbitraryTransaction(repository, transactionData);
-				if (transaction.isDataLocal()) {
-					String name = transactionData.getName();
-					Service service = transactionData.getService();
-					String identifier = transactionData.getIdentifier();
-
-					if (transactionData.getName() != null) {
-						List<ArbitraryResourceInfo> transactionResources = repository.getArbitraryRepository()
-								.getArbitraryResources(service, identifier, name, (identifier == null), null, null, false);
-						if (transactionResources != null) {
-							resources.addAll(transactionResources);
-						}
-					}
+				ArbitraryResourceInfo arbitraryResourceInfo = new ArbitraryResourceInfo();
+				arbitraryResourceInfo.name = transactionData.getName();
+				arbitraryResourceInfo.service = transactionData.getService();
+				arbitraryResourceInfo.identifier = transactionData.getIdentifier();
+				if (!resources.contains(arbitraryResourceInfo)) {
+					resources.add(arbitraryResourceInfo);
 				}
 			}
 
-			if (includeStatus != null && includeStatus == true) {
+			if (includeStatus != null && includeStatus) {
 				resources = this.addStatusToResources(resources);
 			}
+			if (includeMetadata != null && includeMetadata) {
+				resources = this.addMetadataToResources(resources);
+			}
 
 			return resources;
 
@@ -419,6 +524,8 @@ public class ArbitraryResource {
 		}
 	}
 
+
+
 	@DELETE
 	@Path("/resource/{service}/{name}/{identifier}")
 	@Operation(
@@ -534,10 +641,16 @@ public class ArbitraryResource {
 								   @PathParam("service") Service service,
 								   @PathParam("name") String name,
 								   @QueryParam("filepath") String filepath,
-								   @QueryParam("rebuild") boolean rebuild) {
-		Security.checkApiCallAllowed(request);
+								   @QueryParam("rebuild") boolean rebuild,
+								   @QueryParam("async") boolean async,
+								   @QueryParam("attempts") Integer attempts) {
 
-		return this.download(service, name, null, filepath, rebuild);
+		// Authentication can be bypassed in the settings, for those running public QDN nodes
+		if (!Settings.getInstance().isQDNAuthBypassEnabled()) {
+			Security.checkApiCallAllowed(request);
+		}
+
+		return this.download(service, name, null, filepath, rebuild, async, attempts);
 	}
 
 	@GET
@@ -563,10 +676,64 @@ public class ArbitraryResource {
 								   @PathParam("name") String name,
 								   @PathParam("identifier") String identifier,
 								   @QueryParam("filepath") String filepath,
-								   @QueryParam("rebuild") boolean rebuild) {
+								   @QueryParam("rebuild") boolean rebuild,
+								   @QueryParam("async") boolean async,
+								   @QueryParam("attempts") Integer attempts) {
+
+		// Authentication can be bypassed in the settings, for those running public QDN nodes
+		if (!Settings.getInstance().isQDNAuthBypassEnabled()) {
+			Security.checkApiCallAllowed(request);
+		}
+
+		return this.download(service, name, identifier, filepath, rebuild, async, attempts);
+	}
+
+
+	// Metadata
+
+	@GET
+	@Path("/metadata/{service}/{name}/{identifier}")
+	@Operation(
+			summary = "Fetch raw metadata from resource with supplied service, name, identifier, and relative path",
+			responses = {
+					@ApiResponse(
+							description = "Path to file structure containing requested data",
+							content = @Content(
+									mediaType = MediaType.APPLICATION_JSON,
+									schema = @Schema(
+											implementation = ArbitraryDataTransactionMetadata.class
+									)
+							)
+					)
+			}
+	)
+	@SecurityRequirement(name = "apiKey")
+	public ArbitraryResourceMetadata getMetadata(@HeaderParam(Security.API_KEY_HEADER) String apiKey,
+							  @PathParam("service") Service service,
+							  @PathParam("name") String name,
+							  @PathParam("identifier") String identifier) {
 		Security.checkApiCallAllowed(request);
 
-		return this.download(service, name, identifier, filepath, rebuild);
+		ArbitraryDataResource resource = new ArbitraryDataResource(name, ResourceIdType.NAME, service, identifier);
+
+		try {
+			ArbitraryDataTransactionMetadata transactionMetadata = ArbitraryMetadataManager.getInstance().fetchMetadata(resource, false);
+			if (transactionMetadata != null) {
+				ArbitraryResourceMetadata resourceMetadata = ArbitraryResourceMetadata.fromTransactionMetadata(transactionMetadata);
+				if (resourceMetadata != null) {
+					return resourceMetadata;
+				}
+				else {
+					// The metadata file doesn't contain title, description, category, or tags
+					throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FILE_NOT_FOUND);
+				}
+			}
+		} catch (IllegalArgumentException e) {
+			// No metadata exists for this resource
+			throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.FILE_NOT_FOUND, e.getMessage());
+		}
+
+		throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
 	}
 
 
@@ -602,6 +769,10 @@ public class ArbitraryResource {
 	public String post(@HeaderParam(Security.API_KEY_HEADER) String apiKey,
 					   @PathParam("service") String serviceString,
 					   @PathParam("name") String name,
+					   @QueryParam("title") String title,
+					   @QueryParam("description") String description,
+					   @QueryParam("tags") List<String> tags,
+					   @QueryParam("category") Category category,
 					   String path) {
 		Security.checkApiCallAllowed(request);
 
@@ -609,7 +780,8 @@ public class ArbitraryResource {
 			throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, "Path not supplied");
 		}
 
-		return this.upload(Service.valueOf(serviceString), name, null, path, null, null, false);
+		return this.upload(Service.valueOf(serviceString), name, null, path, null, null, false,
+				title, description, tags, category);
 	}
 
 	@POST
@@ -642,6 +814,10 @@ public class ArbitraryResource {
 					   @PathParam("service") String serviceString,
 					   @PathParam("name") String name,
 					   @PathParam("identifier") String identifier,
+					   @QueryParam("title") String title,
+					   @QueryParam("description") String description,
+					   @QueryParam("tags") List<String> tags,
+					   @QueryParam("category") Category category,
 					   String path) {
 		Security.checkApiCallAllowed(request);
 
@@ -649,7 +825,8 @@ public class ArbitraryResource {
 			throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, "Path not supplied");
 		}
 
-		return this.upload(Service.valueOf(serviceString), name, identifier, path, null, null, false);
+		return this.upload(Service.valueOf(serviceString), name, identifier, path, null, null, false,
+				title, description, tags, category);
 	}
 
 
@@ -683,6 +860,10 @@ public class ArbitraryResource {
 	public String postBase64EncodedData(@HeaderParam(Security.API_KEY_HEADER) String apiKey,
 										@PathParam("service") String serviceString,
 										@PathParam("name") String name,
+										@QueryParam("title") String title,
+										@QueryParam("description") String description,
+										@QueryParam("tags") List<String> tags,
+										@QueryParam("category") Category category,
 										String base64) {
 		Security.checkApiCallAllowed(request);
 
@@ -690,7 +871,8 @@ public class ArbitraryResource {
 			throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, "Data not supplied");
 		}
 
-		return this.upload(Service.valueOf(serviceString), name, null, null, null, base64, false);
+		return this.upload(Service.valueOf(serviceString), name, null, null, null, base64, false,
+				title, description, tags, category);
 	}
 
 	@POST
@@ -721,6 +903,10 @@ public class ArbitraryResource {
 										@PathParam("service") String serviceString,
 										@PathParam("name") String name,
 										@PathParam("identifier") String identifier,
+										@QueryParam("title") String title,
+										@QueryParam("description") String description,
+										@QueryParam("tags") List<String> tags,
+										@QueryParam("category") Category category,
 										String base64) {
 		Security.checkApiCallAllowed(request);
 
@@ -728,7 +914,8 @@ public class ArbitraryResource {
 			throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, "Data not supplied");
 		}
 
-		return this.upload(Service.valueOf(serviceString), name, identifier, null, null, base64, false);
+		return this.upload(Service.valueOf(serviceString), name, identifier, null, null, base64, false,
+				title, description, tags, category);
 	}
 
 
@@ -761,6 +948,10 @@ public class ArbitraryResource {
 	public String postZippedData(@HeaderParam(Security.API_KEY_HEADER) String apiKey,
 								 @PathParam("service") String serviceString,
 								 @PathParam("name") String name,
+								 @QueryParam("title") String title,
+								 @QueryParam("description") String description,
+								 @QueryParam("tags") List<String> tags,
+								 @QueryParam("category") Category category,
 								 String base64Zip) {
 		Security.checkApiCallAllowed(request);
 
@@ -768,7 +959,8 @@ public class ArbitraryResource {
 			throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, "Data not supplied");
 		}
 
-		return this.upload(Service.valueOf(serviceString), name, null, null, null, base64Zip, true);
+		return this.upload(Service.valueOf(serviceString), name, null, null, null, base64Zip, true,
+				title, description, tags, category);
 	}
 
 	@POST
@@ -799,6 +991,10 @@ public class ArbitraryResource {
 								 @PathParam("service") String serviceString,
 								 @PathParam("name") String name,
 								 @PathParam("identifier") String identifier,
+								 @QueryParam("title") String title,
+								 @QueryParam("description") String description,
+								 @QueryParam("tags") List<String> tags,
+								 @QueryParam("category") Category category,
 								 String base64Zip) {
 		Security.checkApiCallAllowed(request);
 
@@ -806,7 +1002,8 @@ public class ArbitraryResource {
 			throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, "Data not supplied");
 		}
 
-		return this.upload(Service.valueOf(serviceString), name, identifier, null, null, base64Zip, true);
+		return this.upload(Service.valueOf(serviceString), name, identifier, null, null, base64Zip, true,
+				title, description, tags, category);
 	}
 
 
@@ -842,6 +1039,10 @@ public class ArbitraryResource {
 	public String postString(@HeaderParam(Security.API_KEY_HEADER) String apiKey,
 							 @PathParam("service") String serviceString,
 							 @PathParam("name") String name,
+							 @QueryParam("title") String title,
+							 @QueryParam("description") String description,
+							 @QueryParam("tags") List<String> tags,
+							 @QueryParam("category") Category category,
 							 String string) {
 		Security.checkApiCallAllowed(request);
 
@@ -849,7 +1050,8 @@ public class ArbitraryResource {
 			throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, "Data string not supplied");
 		}
 
-		return this.upload(Service.valueOf(serviceString), name, null, null, string, null, false);
+		return this.upload(Service.valueOf(serviceString), name, null, null, string, null, false,
+				title, description, tags, category);
 	}
 
 	@POST
@@ -882,6 +1084,10 @@ public class ArbitraryResource {
 							 @PathParam("service") String serviceString,
 							 @PathParam("name") String name,
 							 @PathParam("identifier") String identifier,
+							 @QueryParam("title") String title,
+							 @QueryParam("description") String description,
+							 @QueryParam("tags") List<String> tags,
+							 @QueryParam("category") Category category,
 							 String string) {
 		Security.checkApiCallAllowed(request);
 
@@ -889,13 +1095,16 @@ public class ArbitraryResource {
 			throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, "Data string not supplied");
 		}
 
-		return this.upload(Service.valueOf(serviceString), name, identifier, null, string, null, false);
+		return this.upload(Service.valueOf(serviceString), name, identifier, null, string, null, false,
+				title, description, tags, category);
 	}
 
 
 	// Shared methods
 
-	private String upload(Service service, String name, String identifier, String path, String string, String base64, boolean zipped) {
+	private String upload(Service service, String name, String identifier,
+						  String path, String string, String base64, boolean zipped,
+						  String title, String description, List<String> tags, Category category) {
 		// Fetch public key from registered name
 		try (final Repository repository = RepositoryManager.getRepository()) {
 			NameData nameData = repository.getNameRepository().fromName(name);
@@ -904,7 +1113,8 @@ public class ArbitraryResource {
 				throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, error);
 			}
 
-			if (!Controller.getInstance().isUpToDate()) {
+			final Long minLatestBlockTimestamp = NTP.getTime() - (60 * 60 * 1000L);
+			if (!Controller.getInstance().isUpToDate(minLatestBlockTimestamp)) {
 				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCKCHAIN_NEEDS_SYNC);
 			}
 
@@ -959,7 +1169,8 @@ public class ArbitraryResource {
 
 			try {
 				ArbitraryDataTransactionBuilder transactionBuilder = new ArbitraryDataTransactionBuilder(
-						repository, publicKey58, Paths.get(path), name, null, service, identifier
+						repository, publicKey58, Paths.get(path), name, null, service, identifier,
+						title, description, tags, category
 				);
 
 				transactionBuilder.build();
@@ -977,30 +1188,45 @@ public class ArbitraryResource {
 		}
 	}
 
-	private HttpServletResponse download(Service service, String name, String identifier, String filepath, boolean rebuild) {
+	private HttpServletResponse download(Service service, String name, String identifier, String filepath, boolean rebuild, boolean async, Integer maxAttempts) {
 
 		ArbitraryDataReader arbitraryDataReader = new ArbitraryDataReader(name, ArbitraryDataFile.ResourceIdType.NAME, service, identifier);
 		try {
 
 			int attempts = 0;
+			if (maxAttempts == null) {
+				maxAttempts = 5;
+			}
 
 			// Loop until we have data
-			while (!Controller.isStopping()) {
-				attempts++;
-				if (!arbitraryDataReader.isBuilding()) {
-					try {
-						arbitraryDataReader.loadSynchronously(rebuild);
-						break;
-					} catch (MissingDataException e) {
-						if (attempts > 5) {
-							// Give up after 5 attempts
-							throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, "Data unavailable. Please try again later.");
+			if (async) {
+				// Asynchronous
+				arbitraryDataReader.loadAsynchronously(false, 1);
+			}
+			else {
+				// Synchronous
+				while (!Controller.isStopping()) {
+					attempts++;
+					if (!arbitraryDataReader.isBuilding()) {
+						try {
+							arbitraryDataReader.loadSynchronously(rebuild);
+							break;
+						} catch (MissingDataException e) {
+							if (attempts > maxAttempts) {
+								// Give up after 5 attempts
+								throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, "Data unavailable. Please try again later.");
+							}
 						}
 					}
+					Thread.sleep(3000L);
 				}
-				Thread.sleep(3000L);
 			}
+
 			java.nio.file.Path outputPath = arbitraryDataReader.getFilePath();
+			if (outputPath == null) {
+				// Assume the resource doesn't exist
+				throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.FILE_NOT_FOUND, "File not found");
+			}
 
 			if (filepath == null || filepath.isEmpty()) {
 				// No file path supplied - so check if this is a single file resource
@@ -1009,6 +1235,10 @@ public class ArbitraryResource {
 					// This is a single file resource
 					filepath = files[0];
 				}
+				else {
+					throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA,
+							"filepath is required for resources containing more than one file");
+				}
 			}
 
 			// TODO: limit file size that can be read into memory
@@ -1024,39 +1254,43 @@ public class ArbitraryResource {
 
 			return response;
 		} catch (Exception e) {
-			LOGGER.info(String.format("Unable to load %s %s: %s", service, name, e.getMessage()));
+			LOGGER.debug(String.format("Unable to load %s %s: %s", service, name, e.getMessage()));
 			throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.FILE_NOT_FOUND, e.getMessage());
 		}
 	}
 
 
-	private ArbitraryResourceStatus getStatus(Service service, String name, String identifier, Boolean build) {
-
-		// If "build=true" has been specified in the query string, build the resource before returning its status
-		if (build != null && build == true) {
-			ArbitraryDataReader reader = new ArbitraryDataReader(name, ArbitraryDataFile.ResourceIdType.NAME, service, null);
-			try {
-				if (!reader.isBuilding()) {
-					reader.loadSynchronously(false);
-				}
-			} catch (Exception e) {
-				// No need to handle exception, as it will be reflected in the status
-			}
-		}
-
-		ArbitraryDataResource resource = new ArbitraryDataResource(name, ResourceIdType.NAME, service, identifier);
-		return resource.getStatus();
-	}
-
 	private List<ArbitraryResourceInfo> addStatusToResources(List<ArbitraryResourceInfo> resources) {
 		// Determine and add the status of each resource
 		List<ArbitraryResourceInfo> updatedResources = new ArrayList<>();
+		for (ArbitraryResourceInfo resourceInfo : resources) {
+			try {
+				ArbitraryDataResource resource = new ArbitraryDataResource(resourceInfo.name, ResourceIdType.NAME,
+						resourceInfo.service, resourceInfo.identifier);
+				ArbitraryResourceStatus status = resource.getStatus(true);
+				if (status != null) {
+					resourceInfo.status = status;
+				}
+				updatedResources.add(resourceInfo);
+
+			} catch (Exception e) {
+				// Catch and log all exceptions, since some systems are experiencing 500 errors when including statuses
+				LOGGER.info("Caught exception when adding status to resource %s: %s", resourceInfo, e.toString());
+			}
+		}
+		return updatedResources;
+	}
+
+	private List<ArbitraryResourceInfo> addMetadataToResources(List<ArbitraryResourceInfo> resources) {
+		// Add metadata fields to each resource if they exist
+		List<ArbitraryResourceInfo> updatedResources = new ArrayList<>();
 		for (ArbitraryResourceInfo resourceInfo : resources) {
 			ArbitraryDataResource resource = new ArbitraryDataResource(resourceInfo.name, ResourceIdType.NAME,
 					resourceInfo.service, resourceInfo.identifier);
-			ArbitraryResourceStatus status = resource.getStatus();
-			if (status != null) {
-				resourceInfo.status = status;
+			ArbitraryDataTransactionMetadata transactionMetadata = resource.getLatestTransactionMetadata();
+			ArbitraryResourceMetadata resourceMetadata = ArbitraryResourceMetadata.fromTransactionMetadata(transactionMetadata);
+			if (resourceMetadata != null) {
+				resourceInfo.metadata = resourceMetadata;
 			}
 			updatedResources.add(resourceInfo);
 		}

src/main/java/org/qortal/api/resource/BlocksResource.java

@@ -114,7 +114,7 @@ public class BlocksResource {
 	@Path("/signature/{signature}/data")
 	@Operation(
 			summary = "Fetch serialized, base58 encoded block data using base58 signature",
-			description = "Returns serialized data for the block that matches the given signature",
+			description = "Returns serialized data for the block that matches the given signature, and an optional block serialization version",
 			responses = {
 					@ApiResponse(
 							description = "the block data",
@@ -125,7 +125,7 @@ public class BlocksResource {
 	@ApiErrors({
 			ApiError.INVALID_SIGNATURE, ApiError.BLOCK_UNKNOWN, ApiError.INVALID_DATA, ApiError.REPOSITORY_ISSUE
 	})
-	public String getSerializedBlockData(@PathParam("signature") String signature58) {
+	public String getSerializedBlockData(@PathParam("signature") String signature58, @QueryParam("version") Integer version) {
 		// Decode signature
 		byte[] signature;
 		try {
@@ -136,20 +136,41 @@ public class BlocksResource {
 
 		try (final Repository repository = RepositoryManager.getRepository()) {
 
+			// Default to version 1
+			if (version == null) {
+				version = 1;
+			}
+
             // Check the database first
 			BlockData blockData = repository.getBlockRepository().fromSignature(signature);
 			if (blockData != null) {
                 Block block = new Block(repository, blockData);
                 ByteArrayOutputStream bytes = new ByteArrayOutputStream();
                 bytes.write(Ints.toByteArray(block.getBlockData().getHeight()));
-                bytes.write(BlockTransformer.toBytes(block));
+
+				switch (version) {
+					case 1:
+						bytes.write(BlockTransformer.toBytes(block));
+						break;
+
+					case 2:
+						bytes.write(BlockTransformer.toBytesV2(block));
+						break;
+
+					default:
+						throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
+				}
+
                 return Base58.encode(bytes.toByteArray());
             }
 
             // Not found, so try the block archive
             byte[] bytes = BlockArchiveReader.getInstance().fetchSerializedBlockBytesForSignature(signature, false, repository);
             if (bytes != null) {
-                return Base58.encode(bytes);
+				if (version != 1) {
+					throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, "Archived blocks require version 1");
+				}
+				return Base58.encode(bytes);
             }
 
             throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
@@ -268,9 +289,15 @@ public class BlocksResource {
 	@ApiErrors({
 		ApiError.REPOSITORY_ISSUE
 	})
-	public BlockData getLastBlock() {
+	public BlockData getLastBlock(@QueryParam("includeOnlineSignatures") Boolean includeOnlineSignatures) {
 		try (final Repository repository = RepositoryManager.getRepository()) {
-			return repository.getBlockRepository().getLastBlock();
+			BlockData blockData = repository.getBlockRepository().getLastBlock();
+
+			if (includeOnlineSignatures == null || includeOnlineSignatures == false) {
+				blockData.setOnlineAccountsSignatures(null);
+			}
+
+			return blockData;
 		} catch (DataException e) {
 			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
 		}
@@ -548,20 +575,25 @@ public class BlocksResource {
 	@ApiErrors({
 		ApiError.BLOCK_UNKNOWN, ApiError.REPOSITORY_ISSUE
 	})
-	public BlockData getByTimestamp(@PathParam("timestamp") long timestamp) {
+	public BlockData getByTimestamp(@PathParam("timestamp") long timestamp,
+									@QueryParam("includeOnlineSignatures") Boolean includeOnlineSignatures) {
 		try (final Repository repository = RepositoryManager.getRepository()) {
 			BlockData blockData = null;
 
 			// Try the Blocks table
 			int height = repository.getBlockRepository().getHeightFromTimestamp(timestamp);
-			if (height > 0) {
+			if (height > 1) {
 				// Found match in Blocks table
-				return repository.getBlockRepository().fromHeight(height);
+				blockData = repository.getBlockRepository().fromHeight(height);
+				if (includeOnlineSignatures == null || includeOnlineSignatures == false) {
+					blockData.setOnlineAccountsSignatures(null);
+				}
+				return blockData;
 			}
 
 			// Not found in Blocks table, so try the archive
 			height = repository.getBlockArchiveRepository().getHeightFromTimestamp(timestamp);
-			if (height > 0) {
+			if (height > 1) {
 				// Found match in archive
 				blockData = repository.getBlockArchiveRepository().fromHeight(height);
 			}
@@ -571,6 +603,10 @@ public class BlocksResource {
 				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCK_UNKNOWN);
 			}
 
+			if (includeOnlineSignatures == null || includeOnlineSignatures == false) {
+				blockData.setOnlineAccountsSignatures(null);
+			}
+
 			return blockData;
 		} catch (DataException e) {
 			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);

src/main/java/org/qortal/api/resource/ChatResource.java

@@ -99,6 +99,38 @@ public class ChatResource {
 		}
 	}
 
+	@GET
+	@Path("/message/{signature}")
+	@Operation(
+			summary = "Find chat message by signature",
+			responses = {
+					@ApiResponse(
+							description = "CHAT message",
+							content = @Content(
+										schema = @Schema(
+												implementation = ChatMessage.class
+										)
+							)
+					)
+			}
+	)
+	@ApiErrors({ApiError.INVALID_CRITERIA, ApiError.INVALID_ADDRESS, ApiError.REPOSITORY_ISSUE})
+	public ChatMessage getMessageBySignature(@PathParam("signature") String signature58) {
+		byte[] signature = Base58.decode(signature58);
+
+		try (final Repository repository = RepositoryManager.getRepository()) {
+
+			ChatTransactionData chatTransactionData = (ChatTransactionData) repository.getTransactionRepository().fromSignature(signature);
+			if (chatTransactionData == null) {
+				throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.INVALID_CRITERIA, "Message not found");
+			}
+
+			return repository.getChatRepository().toChatMessage(chatTransactionData);
+		} catch (DataException e) {
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
+		}
+	}
+
 	@GET
 	@Path("/active/{address}")
 	@Operation(

src/main/java/org/qortal/api/resource/CrossChainBitcoinResource.java

@@ -67,11 +67,16 @@ public class CrossChainBitcoinResource {
 		if (!bitcoin.isValidDeterministicKey(key58))
 			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
 
-		Long balance = bitcoin.getWalletBalance(key58);
-		if (balance == null)
-			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
+		try {
+			Long balance = bitcoin.getWalletBalanceFromTransactions(key58);
+			if (balance == null)
+				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
 
-		return balance.toString();
+			return balance.toString();
+
+		} catch (ForeignBlockchainException e) {
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
+		}
 	}
 
 	@POST
@@ -117,7 +122,7 @@ public class CrossChainBitcoinResource {
 	@Path("/send")
 	@Operation(
 		summary = "Sends BTC from hierarchical, deterministic BIP32 wallet to specific address",
-		description = "Currently only supports 'legacy' P2PKH Bitcoin addresses. Supply BIP32 'm' private key in base58, starting with 'xprv' for mainnet, 'tprv' for testnet",
+		description = "Currently supports 'legacy' P2PKH Bitcoin addresses and Native SegWit (P2WPKH) addresses. Supply BIP32 'm' private key in base58, starting with 'xprv' for mainnet, 'tprv' for testnet",
 		requestBody = @RequestBody(
 			required = true,
 			content = @Content(

src/main/java/org/qortal/api/resource/CrossChainDigibyteResource.java

@@ -0,0 +1,177 @@
+package org.qortal.api.resource;
+
+import io.swagger.v3.oas.annotations.Operation;
+import io.swagger.v3.oas.annotations.media.ArraySchema;
+import io.swagger.v3.oas.annotations.media.Content;
+import io.swagger.v3.oas.annotations.media.Schema;
+import io.swagger.v3.oas.annotations.parameters.RequestBody;
+import io.swagger.v3.oas.annotations.responses.ApiResponse;
+import io.swagger.v3.oas.annotations.security.SecurityRequirement;
+import io.swagger.v3.oas.annotations.tags.Tag;
+
+import java.util.List;
+
+import javax.servlet.http.HttpServletRequest;
+import javax.ws.rs.HeaderParam;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.core.Context;
+import javax.ws.rs.core.MediaType;
+
+import org.bitcoinj.core.Transaction;
+import org.qortal.api.ApiError;
+import org.qortal.api.ApiErrors;
+import org.qortal.api.ApiExceptionFactory;
+import org.qortal.api.Security;
+import org.qortal.api.model.crosschain.DigibyteSendRequest;
+import org.qortal.crosschain.Digibyte;
+import org.qortal.crosschain.ForeignBlockchainException;
+import org.qortal.crosschain.SimpleTransaction;
+
+@Path("/crosschain/dgb")
+@Tag(name = "Cross-Chain (Digibyte)")
+public class CrossChainDigibyteResource {
+
+	@Context
+	HttpServletRequest request;
+
+	@POST
+	@Path("/walletbalance")
+	@Operation(
+		summary = "Returns DGB balance for hierarchical, deterministic BIP32 wallet",
+		description = "Supply BIP32 'm' private/public key in base58, starting with 'xprv'/'xpub' for mainnet, 'tprv'/'tpub' for testnet",
+		requestBody = @RequestBody(
+			required = true,
+			content = @Content(
+				mediaType = MediaType.TEXT_PLAIN,
+				schema = @Schema(
+					type = "string",
+					description = "BIP32 'm' private/public key in base58",
+					example = "tpubD6NzVbkrYhZ4XTPc4btCZ6SMgn8CxmWkj6VBVZ1tfcJfMq4UwAjZbG8U74gGSypL9XBYk2R2BLbDBe8pcEyBKM1edsGQEPKXNbEskZozeZc"
+				)
+			)
+		),
+		responses = {
+			@ApiResponse(
+				content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string", description = "balance (satoshis)"))
+			)
+		}
+	)
+	@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
+	@SecurityRequirement(name = "apiKey")
+	public String getDigibyteWalletBalance(@HeaderParam(Security.API_KEY_HEADER) String apiKey, String key58) {
+		Security.checkApiCallAllowed(request);
+
+		Digibyte digibyte = Digibyte.getInstance();
+
+		if (!digibyte.isValidDeterministicKey(key58))
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
+
+		try {
+			Long balance = digibyte.getWalletBalanceFromTransactions(key58);
+			if (balance == null)
+				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
+
+			return balance.toString();
+
+		} catch (ForeignBlockchainException e) {
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
+		}
+	}
+
+	@POST
+	@Path("/wallettransactions")
+	@Operation(
+		summary = "Returns transactions for hierarchical, deterministic BIP32 wallet",
+		description = "Supply BIP32 'm' private/public key in base58, starting with 'xprv'/'xpub' for mainnet, 'tprv'/'tpub' for testnet",
+		requestBody = @RequestBody(
+			required = true,
+			content = @Content(
+				mediaType = MediaType.TEXT_PLAIN,
+				schema = @Schema(
+					type = "string",
+					description = "BIP32 'm' private/public key in base58",
+					example = "tpubD6NzVbkrYhZ4XTPc4btCZ6SMgn8CxmWkj6VBVZ1tfcJfMq4UwAjZbG8U74gGSypL9XBYk2R2BLbDBe8pcEyBKM1edsGQEPKXNbEskZozeZc"
+				)
+			)
+		),
+		responses = {
+			@ApiResponse(
+				content = @Content(array = @ArraySchema( schema = @Schema( implementation = SimpleTransaction.class ) ) )
+			)
+		}
+	)
+	@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
+	@SecurityRequirement(name = "apiKey")
+	public List<SimpleTransaction> getDigibyteWalletTransactions(@HeaderParam(Security.API_KEY_HEADER) String apiKey, String key58) {
+		Security.checkApiCallAllowed(request);
+
+		Digibyte digibyte = Digibyte.getInstance();
+
+		if (!digibyte.isValidDeterministicKey(key58))
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
+
+		try {
+			return digibyte.getWalletTransactions(key58);
+		} catch (ForeignBlockchainException e) {
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
+		}
+	}
+
+	@POST
+	@Path("/send")
+	@Operation(
+		summary = "Sends DGB from hierarchical, deterministic BIP32 wallet to specific address",
+		description = "Currently supports 'legacy' P2PKH Digibyte addresses and Native SegWit (P2WPKH) addresses. Supply BIP32 'm' private key in base58, starting with 'xprv' for mainnet, 'tprv' for testnet",
+		requestBody = @RequestBody(
+			required = true,
+			content = @Content(
+				mediaType = MediaType.APPLICATION_JSON,
+				schema = @Schema(
+					implementation = DigibyteSendRequest.class
+				)
+			)
+		),
+		responses = {
+			@ApiResponse(
+				content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string", description = "transaction hash"))
+			)
+		}
+	)
+	@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.INVALID_CRITERIA, ApiError.INVALID_ADDRESS, ApiError.FOREIGN_BLOCKCHAIN_BALANCE_ISSUE, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
+	@SecurityRequirement(name = "apiKey")
+	public String sendBitcoin(@HeaderParam(Security.API_KEY_HEADER) String apiKey, DigibyteSendRequest digibyteSendRequest) {
+		Security.checkApiCallAllowed(request);
+
+		if (digibyteSendRequest.digibyteAmount <= 0)
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
+
+		if (digibyteSendRequest.feePerByte != null && digibyteSendRequest.feePerByte <= 0)
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
+
+		Digibyte digibyte = Digibyte.getInstance();
+
+		if (!digibyte.isValidAddress(digibyteSendRequest.receivingAddress))
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
+
+		if (!digibyte.isValidDeterministicKey(digibyteSendRequest.xprv58))
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
+
+		Transaction spendTransaction = digibyte.buildSpend(digibyteSendRequest.xprv58,
+				digibyteSendRequest.receivingAddress,
+				digibyteSendRequest.digibyteAmount,
+				digibyteSendRequest.feePerByte);
+
+		if (spendTransaction == null)
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_BALANCE_ISSUE);
+
+		try {
+			digibyte.broadcastTransaction(spendTransaction);
+		} catch (ForeignBlockchainException e) {
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
+		}
+
+		return spendTransaction.getTxId().toString();
+	}
+
+}

src/main/java/org/qortal/api/resource/CrossChainDogecoinResource.java

@@ -65,11 +65,16 @@ public class CrossChainDogecoinResource {
 		if (!dogecoin.isValidDeterministicKey(key58))
 			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
 
-		Long balance = dogecoin.getWalletBalance(key58);
-		if (balance == null)
-			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
+		try {
+			Long balance = dogecoin.getWalletBalanceFromTransactions(key58);
+			if (balance == null)
+				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
 
-		return balance.toString();
+			return balance.toString();
+
+		} catch (ForeignBlockchainException e) {
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
+		}
 	}
 
 	@POST

src/main/java/org/qortal/api/resource/CrossChainHtlcResource.java

@@ -1,5 +1,6 @@
 package org.qortal.api.resource;
 
+import com.google.common.hash.HashCode;
 import io.swagger.v3.oas.annotations.Operation;
 import io.swagger.v3.oas.annotations.media.Content;
 import io.swagger.v3.oas.annotations.media.Schema;
@@ -7,8 +8,11 @@ import io.swagger.v3.oas.annotations.responses.ApiResponse;
 import io.swagger.v3.oas.annotations.security.SecurityRequirement;
 import io.swagger.v3.oas.annotations.tags.Tag;
 
+import java.io.IOException;
 import java.math.BigDecimal;
 import java.util.List;
+import java.util.Objects;
+import java.util.concurrent.locks.ReentrantLock;
 
 import javax.servlet.http.HttpServletRequest;
 import javax.ws.rs.*;
@@ -21,6 +25,7 @@ import org.bitcoinj.core.*;
 import org.bitcoinj.script.Script;
 import org.qortal.api.*;
 import org.qortal.api.model.CrossChainBitcoinyHTLCStatus;
+import org.qortal.controller.Controller;
 import org.qortal.crosschain.*;
 import org.qortal.crypto.Crypto;
 import org.qortal.data.at.ATData;
@@ -284,6 +289,12 @@ public class CrossChainHtlcResource {
 					continue;
 				}
 
+				Bitcoiny bitcoiny = (Bitcoiny) acct.getBlockchain();
+				if (Objects.equals(bitcoiny.getCurrencyCode(), "ARRR")) {
+					LOGGER.info("Skipping AT {} because ARRR is currently unsupported", atAddress);
+					continue;
+				}
+
 				CrossChainTradeData crossChainTradeData = acct.populateTradeData(repository, atData);
 				if (crossChainTradeData == null) {
 					LOGGER.info("Couldn't find crosschain trade data for AT {}", atAddress);
@@ -363,10 +374,6 @@ public class CrossChainHtlcResource {
 			// Use secret-A to redeem P2SH-A
 
 			Bitcoiny bitcoiny = (Bitcoiny) acct.getBlockchain();
-			if (bitcoiny.getClass() == Bitcoin.class) {
-				LOGGER.info("Redeeming a Bitcoin HTLC is not yet supported");
-				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
-			}
 
 			int lockTime = crossChainTradeData.lockTimeA;
 			byte[] redeemScriptA = BitcoinyHTLC.buildScript(crossChainTradeData.partnerForeignPKH, lockTime, crossChainTradeData.creatorForeignPKH, crossChainTradeData.hashOfSecretA);
@@ -574,7 +581,7 @@ public class CrossChainHtlcResource {
 			// If the AT is "finished" then it will have a zero balance
 			// In these cases we should avoid HTLC refunds if tbe QORT haven't been returned to the seller
 			if (atData.getIsFinished() && crossChainTradeData.mode != AcctMode.REFUNDED && crossChainTradeData.mode != AcctMode.CANCELLED) {
-				LOGGER.info(String.format("Skipping AT %s because the QORT has already been redemed", atAddress));
+				LOGGER.info(String.format("Skipping AT %s because the QORT has already been redeemed by the buyer", atAddress));
 				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
 			}
 
@@ -584,11 +591,6 @@ public class CrossChainHtlcResource {
 				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
 
 			Bitcoiny bitcoiny = (Bitcoiny) acct.getBlockchain();
-			if (bitcoiny.getClass() == Bitcoin.class) {
-				LOGGER.info("Refunding a Bitcoin HTLC is not yet supported");
-				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
-			}
-
 			int lockTime = tradeBotData.getLockTimeA();
 
 			// We can't refund P2SH-A until lockTime-A has passed
@@ -600,15 +602,26 @@ public class CrossChainHtlcResource {
 			if (medianBlockTime <= lockTime)
 				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_TOO_SOON);
 
-			byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTime, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
-			String p2shAddressA = bitcoiny.deriveP2shAddress(redeemScriptA);
-			LOGGER.info(String.format("Refunding P2SH address: %s", p2shAddressA));
-
 			// Fee for redeem/refund is subtracted from P2SH-A balance.
 			long feeTimestamp = calcFeeTimestamp(lockTime, crossChainTradeData.tradeTimeout);
 			long p2shFee = bitcoiny.getP2shFee(feeTimestamp);
 			long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
-			BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(bitcoiny.getBlockchainProvider(), p2shAddressA, minimumAmountA);
+
+			// Create redeem script based on destination chain
+			byte[] redeemScriptA;
+			String p2shAddressA;
+			BitcoinyHTLC.Status htlcStatusA;
+			if (Objects.equals(bitcoiny.getCurrencyCode(), "ARRR")) {
+				redeemScriptA = PirateChainHTLC.buildScript(tradeBotData.getTradeForeignPublicKey(), lockTime, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
+				p2shAddressA = PirateChain.getInstance().deriveP2shAddressBPrefix(redeemScriptA);
+				htlcStatusA = PirateChainHTLC.determineHtlcStatus(bitcoiny.getBlockchainProvider(), p2shAddressA, minimumAmountA);
+			}
+			else {
+				redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTime, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
+				p2shAddressA = bitcoiny.deriveP2shAddress(redeemScriptA);
+				htlcStatusA = BitcoinyHTLC.determineHtlcStatus(bitcoiny.getBlockchainProvider(), p2shAddressA, minimumAmountA);
+			}
+			LOGGER.info(String.format("Refunding P2SH address: %s", p2shAddressA));
 
 			switch (htlcStatusA) {
 				case UNFUNDED:
@@ -625,18 +638,45 @@ public class CrossChainHtlcResource {
 
 				case FUNDED:{
 					Coin refundAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
-					ECKey refundKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
-					List<TransactionOutput> fundingOutputs = bitcoiny.getUnspentOutputs(p2shAddressA);
 
-					// Validate the destination foreign blockchain address
-					Address receiving = Address.fromString(bitcoiny.getNetworkParameters(), receiveAddress);
-					if (receiving.getOutputScriptType() != Script.ScriptType.P2PKH)
-						throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
+					if (Objects.equals(bitcoiny.getCurrencyCode(), "ARRR")) {
+						// Pirate Chain custom integration
 
-					Transaction p2shRefundTransaction = BitcoinyHTLC.buildRefundTransaction(bitcoiny.getNetworkParameters(), refundAmount, refundKey,
-							fundingOutputs, redeemScriptA, lockTime, receiving.getHash());
+						PirateChain pirateChain = PirateChain.getInstance();
+						String p2shAddressT3 = pirateChain.deriveP2shAddress(redeemScriptA);
+
+						// Get funding txid
+						String fundingTxidHex = PirateChainHTLC.getUnspentFundingTxid(pirateChain.getBlockchainProvider(), p2shAddressA, minimumAmountA);
+						if (fundingTxidHex == null) {
+							throw new ForeignBlockchainException("Missing funding txid when refunding P2SH");
+						}
+						String fundingTxid58 = Base58.encode(HashCode.fromString(fundingTxidHex).asBytes());
+
+						byte[] privateKey = tradeBotData.getTradePrivateKey();
+						String privateKey58 = Base58.encode(privateKey);
+						String redeemScript58 = Base58.encode(redeemScriptA);
+
+						String txid = PirateChain.getInstance().refundP2sh(p2shAddressT3,
+								receiveAddress, refundAmount.value, redeemScript58, fundingTxid58, lockTime, privateKey58);
+						LOGGER.info("Refund txid: {}", txid);
+					}
+					else {
+						// ElectrumX coins
+
+						ECKey refundKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
+						List<TransactionOutput> fundingOutputs = bitcoiny.getUnspentOutputs(p2shAddressA);
+
+						// Validate the destination foreign blockchain address
+						Address receiving = Address.fromString(bitcoiny.getNetworkParameters(), receiveAddress);
+						if (receiving.getOutputScriptType() != Script.ScriptType.P2PKH)
+							throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
+
+						Transaction p2shRefundTransaction = BitcoinyHTLC.buildRefundTransaction(bitcoiny.getNetworkParameters(), refundAmount, refundKey,
+								fundingOutputs, redeemScriptA, lockTime, receiving.getHash());
+
+						bitcoiny.broadcastTransaction(p2shRefundTransaction);
+					}
 
-					bitcoiny.broadcastTransaction(p2shRefundTransaction);
 					return true;
 				}
 			}

src/main/java/org/qortal/api/resource/CrossChainLitecoinResource.java

@@ -67,11 +67,16 @@ public class CrossChainLitecoinResource {
 		if (!litecoin.isValidDeterministicKey(key58))
 			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
 
-		Long balance = litecoin.getWalletBalance(key58);
-		if (balance == null)
-			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
+		try {
+			Long balance = litecoin.getWalletBalanceFromTransactions(key58);
+			if (balance == null)
+				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
 
-		return balance.toString();
+			return balance.toString();
+
+		} catch (ForeignBlockchainException e) {
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
+		}
 	}
 
 	@POST
@@ -117,7 +122,7 @@ public class CrossChainLitecoinResource {
 	@Path("/send")
 	@Operation(
 		summary = "Sends LTC from hierarchical, deterministic BIP32 wallet to specific address",
-		description = "Currently only supports 'legacy' P2PKH Litecoin addresses. Supply BIP32 'm' private key in base58, starting with 'xprv' for mainnet, 'tprv' for testnet",
+		description = "Currently supports 'legacy' P2PKH Litecoin addresses and Native SegWit (P2WPKH) addresses. Supply BIP32 'm' private key in base58, starting with 'xprv' for mainnet, 'tprv' for testnet",
 		requestBody = @RequestBody(
 			required = true,
 			content = @Content(

src/main/java/org/qortal/api/resource/CrossChainPirateChainResource.java

@@ -0,0 +1,229 @@
+package org.qortal.api.resource;
+
+import io.swagger.v3.oas.annotations.Operation;
+import io.swagger.v3.oas.annotations.media.ArraySchema;
+import io.swagger.v3.oas.annotations.media.Content;
+import io.swagger.v3.oas.annotations.media.Schema;
+import io.swagger.v3.oas.annotations.parameters.RequestBody;
+import io.swagger.v3.oas.annotations.responses.ApiResponse;
+import io.swagger.v3.oas.annotations.security.SecurityRequirement;
+import io.swagger.v3.oas.annotations.tags.Tag;
+import org.qortal.api.ApiError;
+import org.qortal.api.ApiErrors;
+import org.qortal.api.ApiExceptionFactory;
+import org.qortal.api.Security;
+import org.qortal.api.model.crosschain.PirateChainSendRequest;
+import org.qortal.crosschain.ForeignBlockchainException;
+import org.qortal.crosschain.PirateChain;
+import org.qortal.crosschain.SimpleTransaction;
+
+import javax.servlet.http.HttpServletRequest;
+import javax.ws.rs.HeaderParam;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.core.Context;
+import javax.ws.rs.core.MediaType;
+import java.util.List;
+
+@Path("/crosschain/arrr")
+@Tag(name = "Cross-Chain (Pirate Chain)")
+public class CrossChainPirateChainResource {
+
+	@Context
+	HttpServletRequest request;
+
+	@POST
+	@Path("/walletbalance")
+	@Operation(
+		summary = "Returns ARRR balance",
+		description = "Supply 32 bytes of entropy, Base58 encoded",
+		requestBody = @RequestBody(
+			required = true,
+			content = @Content(
+				mediaType = MediaType.TEXT_PLAIN,
+				schema = @Schema(
+						type = "string",
+						description = "32 bytes of entropy, Base58 encoded",
+						example = "5oSXF53qENtdUyKhqSxYzP57m6RhVFP9BJKRr9E5kRGV"
+				)
+			)
+		),
+		responses = {
+			@ApiResponse(
+				content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string", description = "balance (satoshis)"))
+			)
+		}
+	)
+	@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
+	@SecurityRequirement(name = "apiKey")
+	public String getPirateChainWalletBalance(@HeaderParam(Security.API_KEY_HEADER) String apiKey, String entropy58) {
+		Security.checkApiCallAllowed(request);
+
+		PirateChain pirateChain = PirateChain.getInstance();
+
+		try {
+			Long balance = pirateChain.getWalletBalance(entropy58);
+			if (balance == null)
+				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
+
+			return balance.toString();
+
+		} catch (ForeignBlockchainException e) {
+			throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE, e.getMessage());
+		}
+	}
+
+	@POST
+	@Path("/wallettransactions")
+	@Operation(
+		summary = "Returns transactions",
+		description = "Supply 32 bytes of entropy, Base58 encoded",
+		requestBody = @RequestBody(
+			required = true,
+			content = @Content(
+				mediaType = MediaType.TEXT_PLAIN,
+				schema = @Schema(
+						type = "string",
+						description = "32 bytes of entropy, Base58 encoded",
+						example = "5oSXF53qENtdUyKhqSxYzP57m6RhVFP9BJKRr9E5kRGV"
+				)
+			)
+		),
+		responses = {
+			@ApiResponse(
+				content = @Content(array = @ArraySchema( schema = @Schema( implementation = SimpleTransaction.class ) ) )
+			)
+		}
+	)
+	@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
+	@SecurityRequirement(name = "apiKey")
+	public List<SimpleTransaction> getPirateChainWalletTransactions(@HeaderParam(Security.API_KEY_HEADER) String apiKey, String entropy58) {
+		Security.checkApiCallAllowed(request);
+
+		PirateChain pirateChain = PirateChain.getInstance();
+
+		try {
+			return pirateChain.getWalletTransactions(entropy58);
+		} catch (ForeignBlockchainException e) {
+			throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE, e.getMessage());
+		}
+	}
+
+	@POST
+	@Path("/send")
+	@Operation(
+		summary = "Sends ARRR from wallet",
+		description = "Currently supports 'legacy' P2PKH PirateChain addresses and Native SegWit (P2WPKH) addresses. Supply BIP32 'm' private key in base58, starting with 'xprv' for mainnet, 'tprv' for testnet",
+		requestBody = @RequestBody(
+			required = true,
+			content = @Content(
+				mediaType = MediaType.TEXT_PLAIN,
+				schema = @Schema(
+						type = "string",
+						description = "32 bytes of entropy, Base58 encoded",
+						example = "5oSXF53qENtdUyKhqSxYzP57m6RhVFP9BJKRr9E5kRGV"
+				)
+			)
+		),
+		responses = {
+			@ApiResponse(
+				content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string", description = "transaction hash"))
+			)
+		}
+	)
+	@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.INVALID_CRITERIA, ApiError.INVALID_ADDRESS, ApiError.FOREIGN_BLOCKCHAIN_BALANCE_ISSUE, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
+	@SecurityRequirement(name = "apiKey")
+	public String sendBitcoin(@HeaderParam(Security.API_KEY_HEADER) String apiKey, PirateChainSendRequest pirateChainSendRequest) {
+		Security.checkApiCallAllowed(request);
+
+		if (pirateChainSendRequest.arrrAmount <= 0)
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
+
+		if (pirateChainSendRequest.feePerByte != null && pirateChainSendRequest.feePerByte <= 0)
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
+
+		PirateChain pirateChain = PirateChain.getInstance();
+
+		try {
+			return pirateChain.sendCoins(pirateChainSendRequest);
+
+		} catch (ForeignBlockchainException e) {
+			// TODO
+			throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE, e.getMessage());
+		}
+	}
+
+
+	@POST
+	@Path("/walletaddress")
+	@Operation(
+			summary = "Returns main wallet address",
+			description = "Supply 32 bytes of entropy, Base58 encoded",
+			requestBody = @RequestBody(
+					required = true,
+					content = @Content(
+							mediaType = MediaType.TEXT_PLAIN,
+							schema = @Schema(
+									type = "string",
+									description = "32 bytes of entropy, Base58 encoded",
+									example = "5oSXF53qENtdUyKhqSxYzP57m6RhVFP9BJKRr9E5kRGV"
+							)
+					)
+			),
+			responses = {
+					@ApiResponse(
+							content = @Content(array = @ArraySchema( schema = @Schema( implementation = SimpleTransaction.class ) ) )
+					)
+			}
+	)
+	@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
+	@SecurityRequirement(name = "apiKey")
+	public String getPirateChainWalletAddress(@HeaderParam(Security.API_KEY_HEADER) String apiKey, String entropy58) {
+		Security.checkApiCallAllowed(request);
+
+		PirateChain pirateChain = PirateChain.getInstance();
+
+		try {
+			return pirateChain.getWalletAddress(entropy58);
+		} catch (ForeignBlockchainException e) {
+			throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE, e.getMessage());
+		}
+	}
+
+
+	@POST
+	@Path("/syncstatus")
+	@Operation(
+			summary = "Returns synchronization status",
+			description = "Supply 32 bytes of entropy, Base58 encoded",
+			requestBody = @RequestBody(
+					required = true,
+					content = @Content(
+							mediaType = MediaType.TEXT_PLAIN,
+							schema = @Schema(
+									type = "string",
+									description = "32 bytes of entropy, Base58 encoded",
+									example = "5oSXF53qENtdUyKhqSxYzP57m6RhVFP9BJKRr9E5kRGV"
+							)
+					)
+			),
+			responses = {
+					@ApiResponse(
+							content = @Content(array = @ArraySchema( schema = @Schema( implementation = SimpleTransaction.class ) ) )
+					)
+			}
+	)
+	@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
+	@SecurityRequirement(name = "apiKey")
+	public String getPirateChainSyncStatus(@HeaderParam(Security.API_KEY_HEADER) String apiKey, String entropy58) {
+		Security.checkApiCallAllowed(request);
+
+		PirateChain pirateChain = PirateChain.getInstance();
+
+		try {
+			return pirateChain.getSyncStatus(entropy58);
+		} catch (ForeignBlockchainException e) {
+			throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE, e.getMessage());
+		}
+	}
+}

src/main/java/org/qortal/api/resource/CrossChainRavencoinResource.java

@@ -0,0 +1,177 @@
+package org.qortal.api.resource;
+
+import io.swagger.v3.oas.annotations.Operation;
+import io.swagger.v3.oas.annotations.media.ArraySchema;
+import io.swagger.v3.oas.annotations.media.Content;
+import io.swagger.v3.oas.annotations.media.Schema;
+import io.swagger.v3.oas.annotations.parameters.RequestBody;
+import io.swagger.v3.oas.annotations.responses.ApiResponse;
+import io.swagger.v3.oas.annotations.security.SecurityRequirement;
+import io.swagger.v3.oas.annotations.tags.Tag;
+
+import java.util.List;
+
+import javax.servlet.http.HttpServletRequest;
+import javax.ws.rs.HeaderParam;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.core.Context;
+import javax.ws.rs.core.MediaType;
+
+import org.bitcoinj.core.Transaction;
+import org.qortal.api.ApiError;
+import org.qortal.api.ApiErrors;
+import org.qortal.api.ApiExceptionFactory;
+import org.qortal.api.Security;
+import org.qortal.api.model.crosschain.RavencoinSendRequest;
+import org.qortal.crosschain.Ravencoin;
+import org.qortal.crosschain.ForeignBlockchainException;
+import org.qortal.crosschain.SimpleTransaction;
+
+@Path("/crosschain/rvn")
+@Tag(name = "Cross-Chain (Ravencoin)")
+public class CrossChainRavencoinResource {
+
+	@Context
+	HttpServletRequest request;
+
+	@POST
+	@Path("/walletbalance")
+	@Operation(
+		summary = "Returns RVN balance for hierarchical, deterministic BIP32 wallet",
+		description = "Supply BIP32 'm' private/public key in base58, starting with 'xprv'/'xpub' for mainnet, 'tprv'/'tpub' for testnet",
+		requestBody = @RequestBody(
+			required = true,
+			content = @Content(
+				mediaType = MediaType.TEXT_PLAIN,
+				schema = @Schema(
+					type = "string",
+					description = "BIP32 'm' private/public key in base58",
+					example = "tpubD6NzVbkrYhZ4XTPc4btCZ6SMgn8CxmWkj6VBVZ1tfcJfMq4UwAjZbG8U74gGSypL9XBYk2R2BLbDBe8pcEyBKM1edsGQEPKXNbEskZozeZc"
+				)
+			)
+		),
+		responses = {
+			@ApiResponse(
+				content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string", description = "balance (satoshis)"))
+			)
+		}
+	)
+	@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
+	@SecurityRequirement(name = "apiKey")
+	public String getRavencoinWalletBalance(@HeaderParam(Security.API_KEY_HEADER) String apiKey, String key58) {
+		Security.checkApiCallAllowed(request);
+
+		Ravencoin ravencoin = Ravencoin.getInstance();
+
+		if (!ravencoin.isValidDeterministicKey(key58))
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
+
+		try {
+			Long balance = ravencoin.getWalletBalanceFromTransactions(key58);
+			if (balance == null)
+				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
+
+			return balance.toString();
+
+		} catch (ForeignBlockchainException e) {
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
+		}
+	}
+
+	@POST
+	@Path("/wallettransactions")
+	@Operation(
+		summary = "Returns transactions for hierarchical, deterministic BIP32 wallet",
+		description = "Supply BIP32 'm' private/public key in base58, starting with 'xprv'/'xpub' for mainnet, 'tprv'/'tpub' for testnet",
+		requestBody = @RequestBody(
+			required = true,
+			content = @Content(
+				mediaType = MediaType.TEXT_PLAIN,
+				schema = @Schema(
+					type = "string",
+					description = "BIP32 'm' private/public key in base58",
+					example = "tpubD6NzVbkrYhZ4XTPc4btCZ6SMgn8CxmWkj6VBVZ1tfcJfMq4UwAjZbG8U74gGSypL9XBYk2R2BLbDBe8pcEyBKM1edsGQEPKXNbEskZozeZc"
+				)
+			)
+		),
+		responses = {
+			@ApiResponse(
+				content = @Content(array = @ArraySchema( schema = @Schema( implementation = SimpleTransaction.class ) ) )
+			)
+		}
+	)
+	@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
+	@SecurityRequirement(name = "apiKey")
+	public List<SimpleTransaction> getRavencoinWalletTransactions(@HeaderParam(Security.API_KEY_HEADER) String apiKey, String key58) {
+		Security.checkApiCallAllowed(request);
+
+		Ravencoin ravencoin = Ravencoin.getInstance();
+
+		if (!ravencoin.isValidDeterministicKey(key58))
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
+
+		try {
+			return ravencoin.getWalletTransactions(key58);
+		} catch (ForeignBlockchainException e) {
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
+		}
+	}
+
+	@POST
+	@Path("/send")
+	@Operation(
+		summary = "Sends RVN from hierarchical, deterministic BIP32 wallet to specific address",
+		description = "Currently only supports 'legacy' P2PKH Ravencoin addresses. Supply BIP32 'm' private key in base58, starting with 'xprv' for mainnet, 'tprv' for testnet",
+		requestBody = @RequestBody(
+			required = true,
+			content = @Content(
+				mediaType = MediaType.APPLICATION_JSON,
+				schema = @Schema(
+					implementation = RavencoinSendRequest.class
+				)
+			)
+		),
+		responses = {
+			@ApiResponse(
+				content = @Content(mediaType = MediaType.TEXT_PLAIN, schema = @Schema(type = "string", description = "transaction hash"))
+			)
+		}
+	)
+	@ApiErrors({ApiError.INVALID_PRIVATE_KEY, ApiError.INVALID_CRITERIA, ApiError.INVALID_ADDRESS, ApiError.FOREIGN_BLOCKCHAIN_BALANCE_ISSUE, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE})
+	@SecurityRequirement(name = "apiKey")
+	public String sendBitcoin(@HeaderParam(Security.API_KEY_HEADER) String apiKey, RavencoinSendRequest ravencoinSendRequest) {
+		Security.checkApiCallAllowed(request);
+
+		if (ravencoinSendRequest.ravencoinAmount <= 0)
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
+
+		if (ravencoinSendRequest.feePerByte != null && ravencoinSendRequest.feePerByte <= 0)
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
+
+		Ravencoin ravencoin = Ravencoin.getInstance();
+
+		if (!ravencoin.isValidAddress(ravencoinSendRequest.receivingAddress))
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
+
+		if (!ravencoin.isValidDeterministicKey(ravencoinSendRequest.xprv58))
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PRIVATE_KEY);
+
+		Transaction spendTransaction = ravencoin.buildSpend(ravencoinSendRequest.xprv58,
+				ravencoinSendRequest.receivingAddress,
+				ravencoinSendRequest.ravencoinAmount,
+				ravencoinSendRequest.feePerByte);
+
+		if (spendTransaction == null)
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_BALANCE_ISSUE);
+
+		try {
+			ravencoin.broadcastTransaction(spendTransaction);
+		} catch (ForeignBlockchainException e) {
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.FOREIGN_BLOCKCHAIN_NETWORK_ISSUE);
+		}
+
+		return spendTransaction.getTxId().toString();
+	}
+
+}

src/main/java/org/qortal/api/resource/CrossChainResource.java

@@ -25,6 +25,7 @@ import org.qortal.api.ApiExceptionFactory;
 import org.qortal.api.Security;
 import org.qortal.api.model.CrossChainCancelRequest;
 import org.qortal.api.model.CrossChainTradeSummary;
+import org.qortal.controller.tradebot.TradeBot;
 import org.qortal.crosschain.SupportedBlockchain;
 import org.qortal.crosschain.ACCT;
 import org.qortal.crosschain.AcctMode;
@@ -120,6 +121,8 @@ public class CrossChainResource {
 				crossChainTrades = crossChainTrades.subList(0, upperLimit);
 			}
 
+			crossChainTrades.stream().forEach(CrossChainResource::decorateTradeDataWithPresence);
+
 			return crossChainTrades;
 		} catch (DataException e) {
 			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
@@ -151,7 +154,11 @@ public class CrossChainResource {
 			if (acct == null)
 				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA);
 
-			return acct.populateTradeData(repository, atData);
+			CrossChainTradeData crossChainTradeData = acct.populateTradeData(repository, atData);
+
+			decorateTradeDataWithPresence(crossChainTradeData);
+
+			return crossChainTradeData;
 		} catch (DataException e) {
 			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
 		}
@@ -486,4 +493,7 @@ public class CrossChainResource {
 		}
 	}
 
+	private static void decorateTradeDataWithPresence(CrossChainTradeData crossChainTradeData) {
+		TradeBot.getInstance().decorateTradeDataWithPresence(crossChainTradeData);
+	}
 }

src/main/java/org/qortal/api/resource/CrossChainTradeBotResource.java

@@ -42,6 +42,7 @@ import org.qortal.repository.DataException;
 import org.qortal.repository.Repository;
 import org.qortal.repository.RepositoryManager;
 import org.qortal.utils.Base58;
+import org.qortal.utils.NTP;
 
 @Path("/crosschain/tradebot")
 @Tag(name = "Cross-Chain (Trade-Bot)")
@@ -137,7 +138,8 @@ public class CrossChainTradeBotResource {
 		if (tradeBotCreateRequest.qortAmount <= 0 || tradeBotCreateRequest.fundingQortAmount <= 0)
 			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.ORDER_SIZE_TOO_SMALL);
 
-		if (!Controller.getInstance().isUpToDate())
+		final Long minLatestBlockTimestamp = NTP.getTime() - (60 * 60 * 1000L);
+		if (!Controller.getInstance().isUpToDate(minLatestBlockTimestamp))
 			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCKCHAIN_NEEDS_SYNC);
 
 		try (final Repository repository = RepositoryManager.getRepository()) {
@@ -153,7 +155,7 @@ public class CrossChainTradeBotResource {
 
 			return Base58.encode(unsignedBytes);
 		} catch (DataException e) {
-			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
+			throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.REPOSITORY_ISSUE, e.getMessage());
 		}
 	}
 
@@ -198,7 +200,8 @@ public class CrossChainTradeBotResource {
 		if (tradeBotRespondRequest.receivingAddress == null || !Crypto.isValidAddress(tradeBotRespondRequest.receivingAddress))
 			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
 
-		if (!Controller.getInstance().isUpToDate())
+		final Long minLatestBlockTimestamp = NTP.getTime() - (60 * 60 * 1000L);
+		if (!Controller.getInstance().isUpToDate(minLatestBlockTimestamp))
 			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCKCHAIN_NEEDS_SYNC);
 
 		// Extract data from cross-chain trading AT
@@ -237,7 +240,7 @@ public class CrossChainTradeBotResource {
 					return "false";
 			}
 		} catch (DataException e) {
-			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
+			throw ApiExceptionFactory.INSTANCE.createCustomException(request, ApiError.REPOSITORY_ISSUE, e.getMessage());
 		}
 	}
 

src/main/java/org/qortal/api/resource/GroupsResource.java

@@ -98,7 +98,15 @@ public class GroupsResource {
 		ref = "reverse"
 	) @QueryParam("reverse") Boolean reverse) {
 		try (final Repository repository = RepositoryManager.getRepository()) {
-			return repository.getGroupRepository().getAllGroups(limit, offset, reverse);
+			List<GroupData> allGroupData = repository.getGroupRepository().getAllGroups(limit, offset, reverse);
+			allGroupData.forEach(groupData -> {
+				try {
+					groupData.memberCount = repository.getGroupRepository().countGroupMembers(groupData.getGroupId());
+				} catch (DataException e) {
+					// Exclude memberCount for this group
+				}
+			});
+			return allGroupData;
 		} catch (DataException e) {
 			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
 		}
@@ -150,7 +158,15 @@ public class GroupsResource {
 			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
 
 		try (final Repository repository = RepositoryManager.getRepository()) {
-			return repository.getGroupRepository().getGroupsWithMember(member);
+			List<GroupData> allGroupData = repository.getGroupRepository().getGroupsWithMember(member);
+			allGroupData.forEach(groupData -> {
+				try {
+					groupData.memberCount = repository.getGroupRepository().countGroupMembers(groupData.getGroupId());
+				} catch (DataException e) {
+					// Exclude memberCount for this group
+				}
+			});
+			return allGroupData;
 		} catch (DataException e) {
 			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
 		}
@@ -177,6 +193,7 @@ public class GroupsResource {
 			if (groupData == null)
 				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.GROUP_UNKNOWN);
 
+			groupData.memberCount = repository.getGroupRepository().countGroupMembers(groupId);
 			return groupData;
 		} catch (DataException e) {
 			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
@@ -922,4 +939,4 @@ public class GroupsResource {
 		}
 	}
 
-}
+}

src/main/java/org/qortal/api/resource/NamesResource.java

@@ -26,6 +26,7 @@ import org.qortal.api.ApiErrors;
 import org.qortal.api.ApiException;
 import org.qortal.api.ApiExceptionFactory;
 import org.qortal.api.model.NameSummary;
+import org.qortal.controller.LiteNode;
 import org.qortal.crypto.Crypto;
 import org.qortal.data.naming.NameData;
 import org.qortal.data.transaction.BuyNameTransactionData;
@@ -101,7 +102,14 @@ public class NamesResource {
 			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
 
 		try (final Repository repository = RepositoryManager.getRepository()) {
-			List<NameData> names = repository.getNameRepository().getNamesByOwner(address, limit, offset, reverse);
+			List<NameData> names;
+
+			if (Settings.getInstance().isLite()) {
+				names = LiteNode.getInstance().fetchAccountNames(address);
+			}
+			else {
+				names = repository.getNameRepository().getNamesByOwner(address, limit, offset, reverse);
+			}
 
 			return names.stream().map(NameSummary::new).collect(Collectors.toList());
 		} catch (DataException e) {
@@ -126,10 +134,18 @@ public class NamesResource {
 	@ApiErrors({ApiError.NAME_UNKNOWN, ApiError.REPOSITORY_ISSUE})
 	public NameData getName(@PathParam("name") String name) {
 		try (final Repository repository = RepositoryManager.getRepository()) {
-			NameData nameData = repository.getNameRepository().fromName(name);
+			NameData nameData;
 
-			if (nameData == null)
+			if (Settings.getInstance().isLite()) {
+				nameData = LiteNode.getInstance().fetchNameData(name);
+			}
+			else {
+				nameData = repository.getNameRepository().fromName(name);
+			}
+
+			if (nameData == null) {
 				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.NAME_UNKNOWN);
+			}
 
 			return nameData;
 		} catch (ApiException e) {

src/main/java/org/qortal/api/resource/PeersResource.java

@@ -20,6 +20,11 @@ import javax.ws.rs.*;
 import javax.ws.rs.core.Context;
 import javax.ws.rs.core.MediaType;
 
+import org.apache.logging.log4j.Level;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.LoggerConfig;
+import org.apache.logging.log4j.core.LoggerContext;
 import org.qortal.api.*;
 import org.qortal.api.model.ConnectedPeer;
 import org.qortal.api.model.PeersSummary;
@@ -61,7 +66,7 @@ public class PeersResource {
 		}
 	)
 	public List<ConnectedPeer> getPeers() {
-		return Network.getInstance().getConnectedPeers().stream().map(ConnectedPeer::new).collect(Collectors.toList());
+		return Network.getInstance().getImmutableConnectedPeers().stream().map(ConnectedPeer::new).collect(Collectors.toList());
 	}
 
 	@GET
@@ -127,9 +132,29 @@ public class PeersResource {
 		}
 	)
 	@SecurityRequirement(name = "apiKey")
-	public ExecuteProduceConsume.StatsSnapshot getEngineStats(@HeaderParam(Security.API_KEY_HEADER) String apiKey) {
+	public ExecuteProduceConsume.StatsSnapshot getEngineStats(@HeaderParam(Security.API_KEY_HEADER) String apiKey, @QueryParam("newLoggingLevel") Level newLoggingLevel) {
 		Security.checkApiCallAllowed(request);
 
+		if (newLoggingLevel != null) {
+			final LoggerContext ctx = (LoggerContext) LogManager.getContext(false);
+			final Configuration config = ctx.getConfiguration();
+
+			String epcClassName = "org.qortal.network.Network.NetworkProcessor";
+			LoggerConfig loggerConfig = config.getLoggerConfig(epcClassName);
+			LoggerConfig specificConfig = loggerConfig;
+
+			// We need a specific configuration for this logger,
+			// otherwise we would change the level of all other loggers
+			// having the original configuration as parent as well
+			if (!loggerConfig.getName().equals(epcClassName)) {
+				specificConfig = new LoggerConfig(epcClassName, newLoggingLevel, true);
+				specificConfig.setParent(loggerConfig);
+				config.addLogger(epcClassName, specificConfig);
+			}
+			specificConfig.setLevel(newLoggingLevel);
+			ctx.updateLoggers();
+		}
+
 		return Network.getInstance().getStatsSnapshot();
 	}
 
@@ -304,7 +329,7 @@ public class PeersResource {
 			PeerAddress peerAddress = PeerAddress.fromString(targetPeerAddress);
 			InetSocketAddress resolvedAddress = peerAddress.toSocketAddress();
 
-			List<Peer> peers = Network.getInstance().getHandshakedPeers();
+			List<Peer> peers = Network.getInstance().getImmutableHandshakedPeers();
 			Peer targetPeer = peers.stream().filter(peer -> peer.getResolvedAddress().equals(resolvedAddress)).findFirst().orElse(null);
 
 			if (targetPeer == null)
@@ -352,9 +377,9 @@ public class PeersResource {
 	public PeersSummary peersSummary() {
 		PeersSummary peersSummary = new PeersSummary();
 
-		List<Peer> connectedPeers = Network.getInstance().getConnectedPeers().stream().collect(Collectors.toList());
+		List<Peer> connectedPeers = Network.getInstance().getImmutableConnectedPeers().stream().collect(Collectors.toList());
 		for (Peer peer : connectedPeers) {
-			if (peer.isOutbound()) {
+			if (!peer.isOutbound()) {
 				peersSummary.inboundConnections++;
 			}
 			else {

src/main/java/org/qortal/api/resource/RenderResource.java

@@ -74,7 +74,9 @@ public class RenderResource {
         Method method = Method.PUT;
         Compression compression = Compression.ZIP;
 
-        ArbitraryDataWriter arbitraryDataWriter = new ArbitraryDataWriter(Paths.get(directoryPath), null, Service.WEBSITE, null, method, compression);
+        ArbitraryDataWriter arbitraryDataWriter = new ArbitraryDataWriter(Paths.get(directoryPath),
+                null, Service.WEBSITE, null, method, compression,
+                null, null, null, null);
         try {
             arbitraryDataWriter.save();
         } catch (IOException | DataException | InterruptedException | MissingDataException e) {
@@ -136,34 +138,38 @@ public class RenderResource {
     @GET
     @Path("/signature/{signature}")
     @SecurityRequirement(name = "apiKey")
-    public HttpServletResponse getIndexBySignature(@PathParam("signature") String signature) {
+    public HttpServletResponse getIndexBySignature(@PathParam("signature") String signature,
+                                                   @QueryParam("theme") String theme) {
         Security.requirePriorAuthorization(request, signature, Service.WEBSITE, null);
-        return this.get(signature, ResourceIdType.SIGNATURE, null, "/", null, "/render/signature", true, true);
+        return this.get(signature, ResourceIdType.SIGNATURE, null, "/", null, "/render/signature", true, true, theme);
     }
 
     @GET
     @Path("/signature/{signature}/{path:.*}")
     @SecurityRequirement(name = "apiKey")
-    public HttpServletResponse getPathBySignature(@PathParam("signature") String signature, @PathParam("path") String inPath) {
+    public HttpServletResponse getPathBySignature(@PathParam("signature") String signature, @PathParam("path") String inPath,
+                                                  @QueryParam("theme") String theme) {
         Security.requirePriorAuthorization(request, signature, Service.WEBSITE, null);
-        return this.get(signature, ResourceIdType.SIGNATURE, null, inPath,null, "/render/signature", true, true);
+        return this.get(signature, ResourceIdType.SIGNATURE, null, inPath,null, "/render/signature", true, true, theme);
     }
 
     @GET
     @Path("/hash/{hash}")
     @SecurityRequirement(name = "apiKey")
-    public HttpServletResponse getIndexByHash(@PathParam("hash") String hash58, @QueryParam("secret") String secret58) {
+    public HttpServletResponse getIndexByHash(@PathParam("hash") String hash58, @QueryParam("secret") String secret58,
+                                              @QueryParam("theme") String theme) {
         Security.requirePriorAuthorization(request, hash58, Service.WEBSITE, null);
-        return this.get(hash58, ResourceIdType.FILE_HASH, Service.WEBSITE, "/", secret58, "/render/hash", true, false);
+        return this.get(hash58, ResourceIdType.FILE_HASH, Service.WEBSITE, "/", secret58, "/render/hash", true, false, theme);
     }
 
     @GET
     @Path("/hash/{hash}/{path:.*}")
     @SecurityRequirement(name = "apiKey")
     public HttpServletResponse getPathByHash(@PathParam("hash") String hash58, @PathParam("path") String inPath,
-                                             @QueryParam("secret") String secret58) {
+                                             @QueryParam("secret") String secret58,
+                                             @QueryParam("theme") String theme) {
         Security.requirePriorAuthorization(request, hash58, Service.WEBSITE, null);
-        return this.get(hash58, ResourceIdType.FILE_HASH, Service.WEBSITE, inPath, secret58, "/render/hash", true, false);
+        return this.get(hash58, ResourceIdType.FILE_HASH, Service.WEBSITE, inPath, secret58, "/render/hash", true, false, theme);
     }
 
     @GET
@@ -171,29 +177,35 @@ public class RenderResource {
     @SecurityRequirement(name = "apiKey")
     public HttpServletResponse getPathByName(@PathParam("service") Service service,
                                              @PathParam("name") String name,
-                                             @PathParam("path") String inPath) {
+                                             @PathParam("path") String inPath,
+                                             @QueryParam("theme") String theme) {
         Security.requirePriorAuthorization(request, name, service, null);
         String prefix = String.format("/render/%s", service);
-        return this.get(name, ResourceIdType.NAME, service, inPath, null, prefix, true, true);
+        return this.get(name, ResourceIdType.NAME, service, inPath, null, prefix, true, true, theme);
     }
 
     @GET
     @Path("{service}/{name}")
     @SecurityRequirement(name = "apiKey")
     public HttpServletResponse getIndexByName(@PathParam("service") Service service,
-                                              @PathParam("name") String name) {
+                                              @PathParam("name") String name,
+                                              @QueryParam("theme") String theme) {
         Security.requirePriorAuthorization(request, name, service, null);
         String prefix = String.format("/render/%s", service);
-        return this.get(name, ResourceIdType.NAME, service, "/", null, prefix, true, true);
+        return this.get(name, ResourceIdType.NAME, service, "/", null, prefix, true, true, theme);
     }
 
 
 
     private HttpServletResponse get(String resourceId, ResourceIdType resourceIdType, Service service, String inPath,
-                                    String secret58, String prefix, boolean usePrefix, boolean async) {
+                                    String secret58, String prefix, boolean usePrefix, boolean async, String theme) {
 
         ArbitraryDataRenderer renderer = new ArbitraryDataRenderer(resourceId, resourceIdType, service, inPath,
                 secret58, prefix, usePrefix, async, request, response, context);
+
+        if (theme != null) {
+            renderer.setTheme(theme);
+        }
         return renderer.render();
     }
 

src/main/java/org/qortal/api/resource/TransactionsResource.java

@@ -9,7 +9,10 @@ import io.swagger.v3.oas.annotations.parameters.RequestBody;
 import io.swagger.v3.oas.annotations.responses.ApiResponse;
 import io.swagger.v3.oas.annotations.tags.Tag;
 
+import java.lang.reflect.Constructor;
+import java.lang.reflect.InvocationTargetException;
 import java.util.ArrayList;
+import java.util.Comparator;
 import java.util.List;
 import java.util.concurrent.TimeUnit;
 import java.util.concurrent.locks.ReentrantLock;
@@ -30,6 +33,8 @@ import org.qortal.api.ApiException;
 import org.qortal.api.ApiExceptionFactory;
 import org.qortal.api.model.SimpleTransactionSignRequest;
 import org.qortal.controller.Controller;
+import org.qortal.controller.LiteNode;
+import org.qortal.crypto.Crypto;
 import org.qortal.data.transaction.TransactionData;
 import org.qortal.globalization.Translator;
 import org.qortal.repository.DataException;
@@ -44,6 +49,7 @@ import org.qortal.transform.transaction.TransactionTransformer;
 import org.qortal.utils.Base58;
 
 import com.google.common.primitives.Bytes;
+import org.qortal.utils.NTP;
 
 @Path("/transactions")
 @Tag(name = "Transactions")
@@ -247,14 +253,29 @@ public class TransactionsResource {
 		ApiError.REPOSITORY_ISSUE
 	})
 	public List<TransactionData> getUnconfirmedTransactions(@Parameter(
+			description = "A list of transaction types"
+	) @QueryParam("txType") List<TransactionType> txTypes, @Parameter(
+			description = "Transaction creator's base58 encoded public key"
+	) @QueryParam("creator") String creatorPublicKey58, @Parameter(
 		ref = "limit"
 	) @QueryParam("limit") Integer limit, @Parameter(
 		ref = "offset"
 	) @QueryParam("offset") Integer offset, @Parameter(
 		ref = "reverse"
 	) @QueryParam("reverse") Boolean reverse) {
+
+		// Decode public key if supplied
+		byte[] creatorPublicKey = null;
+		if (creatorPublicKey58 != null) {
+			try {
+				creatorPublicKey = Base58.decode(creatorPublicKey58);
+			} catch (NumberFormatException e) {
+				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_PUBLIC_KEY, e);
+			}
+		}
+
 		try (final Repository repository = RepositoryManager.getRepository()) {
-			return repository.getTransactionRepository().getUnconfirmedTransactions(limit, offset, reverse);
+			return repository.getTransactionRepository().getUnconfirmedTransactions(txTypes, creatorPublicKey, limit, offset, reverse);
 		} catch (ApiException e) {
 			throw e;
 		} catch (DataException e) {
@@ -363,6 +384,150 @@ public class TransactionsResource {
 		}
 	}
 
+	@GET
+	@Path("/address/{address}")
+	@Operation(
+			summary = "Returns transactions for given address",
+			responses = {
+					@ApiResponse(
+							description = "transactions",
+							content = @Content(
+									array = @ArraySchema(
+											schema = @Schema(
+													implementation = TransactionData.class
+											)
+									)
+							)
+					)
+			}
+	)
+	@ApiErrors({ApiError.INVALID_ADDRESS,  ApiError.REPOSITORY_ISSUE})
+	public List<TransactionData> getAddressTransactions(@PathParam("address") String address,
+												 		@Parameter(ref = "limit") @QueryParam("limit") Integer limit,
+												 		@Parameter(ref = "offset") @QueryParam("offset") Integer offset,
+														@Parameter(ref = "reverse") @QueryParam("reverse") Boolean reverse) {
+		if (!Crypto.isValidAddress(address)) {
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_ADDRESS);
+		}
+
+		if (limit == null) {
+			limit = 0;
+		}
+		if (offset == null) {
+			offset = 0;
+		}
+
+		List<TransactionData> transactions;
+
+		if (Settings.getInstance().isLite()) {
+			// Fetch from network
+			transactions = LiteNode.getInstance().fetchAccountTransactions(address, limit, offset);
+
+			// Sort the data, since we can't guarantee the order that a peer sent it in
+			if (reverse) {
+				transactions.sort(Comparator.comparingLong(TransactionData::getTimestamp).reversed());
+			} else {
+				transactions.sort(Comparator.comparingLong(TransactionData::getTimestamp));
+			}
+		}
+		else {
+			// Fetch from local db
+			try (final Repository repository = RepositoryManager.getRepository()) {
+				List<byte[]> signatures = repository.getTransactionRepository().getSignaturesMatchingCriteria(null, null, null,
+						null, null, null, address, TransactionsResource.ConfirmationStatus.CONFIRMED, limit, offset, reverse);
+
+				// Expand signatures to transactions
+				transactions = new ArrayList<>(signatures.size());
+				for (byte[] signature : signatures) {
+					transactions.add(repository.getTransactionRepository().fromSignature(signature));
+				}
+			} catch (ApiException e) {
+				throw e;
+			} catch (DataException e) {
+				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
+			}
+		}
+
+		return transactions;
+	}
+
+	@GET
+	@Path("/unitfee")
+	@Operation(
+			summary = "Get transaction unit fee",
+			responses = {
+					@ApiResponse(
+							content = @Content(
+									mediaType = MediaType.TEXT_PLAIN,
+									schema = @Schema(
+											type = "number"
+									)
+							)
+					)
+			}
+	)
+	@ApiErrors({
+			ApiError.INVALID_CRITERIA, ApiError.REPOSITORY_ISSUE
+	})
+	public long getTransactionUnitFee(@QueryParam("txType") TransactionType txType,
+                                      @QueryParam("timestamp") Long timestamp,
+									  @QueryParam("level") Integer accountLevel) {
+		try {
+			if (timestamp == null) {
+				timestamp = NTP.getTime();
+			}
+
+			Constructor<?> constructor = txType.constructor;
+			Transaction transaction = (Transaction) constructor.newInstance(null, null);
+			// FUTURE: add accountLevel parameter to transaction.getUnitFee() if needed
+			return transaction.getUnitFee(timestamp);
+
+		} catch (InstantiationException | IllegalAccessException | InvocationTargetException e) {
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_CRITERIA, e);
+		}
+	}
+
+	@POST
+	@Path("/fee")
+	@Operation(
+			summary = "Get recommended fee for supplied transaction data",
+			requestBody = @RequestBody(
+					required = true,
+					content = @Content(
+							mediaType = MediaType.TEXT_PLAIN,
+							schema = @Schema(
+									type = "string"
+							)
+					)
+			)
+	)
+	@ApiErrors({
+			ApiError.INVALID_CRITERIA, ApiError.REPOSITORY_ISSUE
+	})
+	public long getRecommendedTransactionFee(String rawInputBytes58) {
+		byte[] rawInputBytes = Base58.decode(rawInputBytes58);
+		if (rawInputBytes.length == 0)
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.JSON);
+
+		try (final Repository repository = RepositoryManager.getRepository()) {
+
+			// Append null signature on the end before transformation
+			byte[] rawBytes = Bytes.concat(rawInputBytes, new byte[TransactionTransformer.SIGNATURE_LENGTH]);
+
+			TransactionData transactionData = TransactionTransformer.fromBytes(rawBytes);
+			if (transactionData == null)
+				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_DATA);
+
+			Transaction transaction = Transaction.fromData(repository, transactionData);
+			return transaction.calcRecommendedFee();
+
+		} catch (DataException e) {
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.REPOSITORY_ISSUE, e);
+		}  catch (TransformationException e) {
+			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.TRANSFORMATION_ERROR, e);
+		}
+	}
+
 	@GET
 	@Path("/creator/{publickey}")
 	@Operation(
@@ -558,7 +723,10 @@ public class TransactionsResource {
 		ApiError.BLOCKCHAIN_NEEDS_SYNC, ApiError.INVALID_SIGNATURE, ApiError.INVALID_DATA, ApiError.TRANSFORMATION_ERROR, ApiError.REPOSITORY_ISSUE
 	})
 	public String processTransaction(String rawBytes58) {
-		if (!Controller.getInstance().isUpToDate())
+		// Only allow a transaction to be processed if our latest block is less than 60 minutes old
+		// If older than this, we should first wait until the blockchain is synced
+		final Long minLatestBlockTimestamp = NTP.getTime() - (60 * 60 * 1000L);
+		if (!Controller.getInstance().isUpToDate(minLatestBlockTimestamp))
 			throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.BLOCKCHAIN_NEEDS_SYNC);
 
 		byte[] rawBytes = Base58.decode(rawBytes58);
@@ -580,7 +748,7 @@ public class TransactionsResource {
 				throw ApiExceptionFactory.INSTANCE.createException(request, ApiError.INVALID_SIGNATURE);
 
 			ReentrantLock blockchainLock = Controller.getInstance().getBlockchainLock();
-			if (!blockchainLock.tryLock(30, TimeUnit.SECONDS))
+			if (!blockchainLock.tryLock(60, TimeUnit.SECONDS))
 				throw createTransactionInvalidException(request, ValidationResult.NO_BLOCKCHAIN_LOCK);
 
 			try {

src/main/java/org/qortal/api/websocket/PresenceWebSocket.java

@@ -20,6 +20,7 @@ import org.eclipse.jetty.websocket.api.annotations.OnWebSocketMessage;
 import org.eclipse.jetty.websocket.api.annotations.WebSocket;
 import org.eclipse.jetty.websocket.servlet.WebSocketServletFactory;
 import org.qortal.controller.Controller;
+import org.qortal.controller.Synchronizer;
 import org.qortal.crypto.Crypto;
 import org.qortal.data.transaction.PresenceTransactionData;
 import org.qortal.data.transaction.TransactionData;
@@ -99,13 +100,13 @@ public class PresenceWebSocket extends ApiWebSocket implements Listener {
 
 	@Override
 	public void listen(Event event) {
-		// We use NewBlockEvent as a proxy for 1-minute timer
-		if (!(event instanceof Controller.NewTransactionEvent) && !(event instanceof Controller.NewBlockEvent))
+		// We use Synchronizer.NewChainTipEvent as a proxy for 1-minute timer
+		if (!(event instanceof Controller.NewTransactionEvent) && !(event instanceof Synchronizer.NewChainTipEvent))
 			return;
 
 		removeOldEntries();
 
-		if (event instanceof Controller.NewBlockEvent)
+		if (event instanceof Synchronizer.NewChainTipEvent)
 			// We only wanted a chance to cull old entries
 			return;
 

src/main/java/org/qortal/api/websocket/TradeBotWebSocket.java

@@ -2,10 +2,7 @@ package org.qortal.api.websocket;
 
 import java.io.IOException;
 import java.io.StringWriter;
-import java.util.Collections;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
+import java.util.*;
 import java.util.stream.Collectors;
 
 import org.eclipse.jetty.websocket.api.Session;
@@ -85,6 +82,7 @@ public class TradeBotWebSocket extends ApiWebSocket implements Listener {
 	@Override
 	public void onWebSocketConnect(Session session) {
 		Map<String, List<String>> queryParams = session.getUpgradeRequest().getParameterMap();
+		final boolean excludeInitialData = queryParams.get("excludeInitialData") != null;
 
 		List<String> foreignBlockchains = queryParams.get("foreignBlockchain");
 		final String foreignBlockchain = foreignBlockchains == null ? null : foreignBlockchains.get(0);
@@ -98,15 +96,22 @@ public class TradeBotWebSocket extends ApiWebSocket implements Listener {
 		// save session's preferred blockchain (if any)
 		sessionBlockchain.put(session, foreignBlockchain);
 
-		// Send all known trade-bot entries
-		try (final Repository repository = RepositoryManager.getRepository()) {
-			List<TradeBotData> tradeBotEntries = repository.getCrossChainRepository().getAllTradeBotData();
 
-			// Optional filtering
-			if (foreignBlockchain != null)
-				tradeBotEntries = tradeBotEntries.stream()
-						.filter(tradeBotData -> tradeBotData.getForeignBlockchain().equals(foreignBlockchain))
-						.collect(Collectors.toList());
+
+		// Maybe send all known trade-bot entries
+		try (final Repository repository = RepositoryManager.getRepository()) {
+			List<TradeBotData> tradeBotEntries = new ArrayList<>();
+
+			// We might need to exclude the initial data from the response
+			if (!excludeInitialData) {
+				tradeBotEntries = repository.getCrossChainRepository().getAllTradeBotData();
+
+				// Optional filtering
+				if (foreignBlockchain != null)
+					tradeBotEntries = tradeBotEntries.stream()
+							.filter(tradeBotData -> tradeBotData.getForeignBlockchain().equals(foreignBlockchain))
+							.collect(Collectors.toList());
+			}
 
 			if (!sendEntries(session, tradeBotEntries)) {
 				session.close(4002, "websocket issue");

src/main/java/org/qortal/api/websocket/TradeOffersWebSocket.java

@@ -23,6 +23,7 @@ import org.eclipse.jetty.websocket.api.annotations.WebSocket;
 import org.eclipse.jetty.websocket.servlet.WebSocketServletFactory;
 import org.qortal.api.model.CrossChainOfferSummary;
 import org.qortal.controller.Controller;
+import org.qortal.controller.Synchronizer;
 import org.qortal.crosschain.SupportedBlockchain;
 import org.qortal.crosschain.ACCT;
 import org.qortal.crosschain.AcctMode;
@@ -80,10 +81,10 @@ public class TradeOffersWebSocket extends ApiWebSocket implements Listener {
 
 	@Override
 	public void listen(Event event) {
-		if (!(event instanceof Controller.NewBlockEvent))
+		if (!(event instanceof Synchronizer.NewChainTipEvent))
 			return;
 
-		BlockData blockData = ((Controller.NewBlockEvent) event).getBlockData();
+		BlockData blockData = ((Synchronizer.NewChainTipEvent) event).getNewChainTip();
 
 		// Process any new info
 
@@ -172,6 +173,7 @@ public class TradeOffersWebSocket extends ApiWebSocket implements Listener {
 	public void onWebSocketConnect(Session session) {
 		Map<String, List<String>> queryParams = session.getUpgradeRequest().getParameterMap();
 		final boolean includeHistoric = queryParams.get("includeHistoric") != null;
+		final boolean excludeInitialData = queryParams.get("excludeInitialData") != null;
 
 		List<String> foreignBlockchains = queryParams.get("foreignBlockchain");
 		final String foreignBlockchain = foreignBlockchains == null ? null : foreignBlockchains.get(0);
@@ -188,20 +190,23 @@ public class TradeOffersWebSocket extends ApiWebSocket implements Listener {
 
 		List<CrossChainOfferSummary> crossChainOfferSummaries = new ArrayList<>();
 
-		synchronized (cachedInfoByBlockchain) {
-			Collection<CachedOfferInfo> cachedInfos;
+		// We might need to exclude the initial data from the response
+		if (!excludeInitialData) {
+			synchronized (cachedInfoByBlockchain) {
+				Collection<CachedOfferInfo> cachedInfos;
 
-			if (foreignBlockchain == null)
-				// No preferred blockchain, so iterate through all of them
-				cachedInfos = cachedInfoByBlockchain.values();
-			else
-				cachedInfos = Collections.singleton(cachedInfoByBlockchain.computeIfAbsent(foreignBlockchain, k -> new CachedOfferInfo()));
+				if (foreignBlockchain == null)
+					// No preferred blockchain, so iterate through all of them
+					cachedInfos = cachedInfoByBlockchain.values();
+				else
+					cachedInfos = Collections.singleton(cachedInfoByBlockchain.computeIfAbsent(foreignBlockchain, k -> new CachedOfferInfo()));
 
-			for (CachedOfferInfo cachedInfo : cachedInfos) {
-				crossChainOfferSummaries.addAll(cachedInfo.currentSummaries.values());
+				for (CachedOfferInfo cachedInfo : cachedInfos) {
+					crossChainOfferSummaries.addAll(cachedInfo.currentSummaries.values());
 
-				if (includeHistoric)
-					crossChainOfferSummaries.addAll(cachedInfo.historicSummaries.values());
+					if (includeHistoric)
+						crossChainOfferSummaries.addAll(cachedInfo.historicSummaries.values());
+				}
 			}
 		}
 

src/main/java/org/qortal/api/websocket/TradePresenceWebSocket.java

@@ -0,0 +1,141 @@
+package org.qortal.api.websocket;
+
+import org.eclipse.jetty.websocket.api.Session;
+import org.eclipse.jetty.websocket.api.annotations.*;
+import org.eclipse.jetty.websocket.servlet.WebSocketServletFactory;
+import org.qortal.controller.Controller;
+import org.qortal.controller.tradebot.TradeBot;
+import org.qortal.data.network.TradePresenceData;
+import org.qortal.event.Event;
+import org.qortal.event.EventBus;
+import org.qortal.event.Listener;
+import org.qortal.utils.Base58;
+import org.qortal.utils.NTP;
+
+import java.io.IOException;
+import java.io.StringWriter;
+import java.util.*;
+
+@WebSocket
+@SuppressWarnings("serial")
+public class TradePresenceWebSocket extends ApiWebSocket implements Listener {
+
+	/** Map key is public key in base58, map value is trade presence */
+	private static final Map<String, TradePresenceData> currentEntries = Collections.synchronizedMap(new HashMap<>());
+
+	@Override
+	public void configure(WebSocketServletFactory factory) {
+		factory.register(TradePresenceWebSocket.class);
+
+		populateCurrentInfo();
+
+		EventBus.INSTANCE.addListener(this::listen);
+	}
+
+	@Override
+	public void listen(Event event) {
+		// XXX - Suggest we change this to something like Synchronizer.NewChainTipEvent?
+		// We use NewBlockEvent as a proxy for 1-minute timer
+		if (!(event instanceof TradeBot.TradePresenceEvent) && !(event instanceof Controller.NewBlockEvent))
+			return;
+
+		removeOldEntries();
+
+		if (event instanceof Controller.NewBlockEvent)
+			// We only wanted a chance to cull old entries
+			return;
+
+		TradePresenceData tradePresence = ((TradeBot.TradePresenceEvent) event).getTradePresenceData();
+
+		boolean somethingChanged = mergePresence(tradePresence);
+
+		if (!somethingChanged)
+			// nothing changed
+			return;
+
+		List<TradePresenceData> tradePresences = Collections.singletonList(tradePresence);
+
+		// Notify sessions
+		for (Session session : getSessions()) {
+			sendTradePresences(session, tradePresences);
+		}
+	}
+
+	@OnWebSocketConnect
+	@Override
+	public void onWebSocketConnect(Session session) {
+		Map<String, List<String>> queryParams = session.getUpgradeRequest().getParameterMap();
+		final boolean excludeInitialData = queryParams.get("excludeInitialData") != null;
+
+		List<TradePresenceData> tradePresences = new ArrayList<>();
+
+		// We might need to exclude the initial data from the response
+		if (!excludeInitialData) {
+			synchronized (currentEntries) {
+				tradePresences = List.copyOf(currentEntries.values());
+			}
+		}
+
+		if (!sendTradePresences(session, tradePresences)) {
+			session.close(4002, "websocket issue");
+			return;
+		}
+
+		super.onWebSocketConnect(session);
+	}
+
+	@OnWebSocketClose
+	@Override
+	public void onWebSocketClose(Session session, int statusCode, String reason) {
+		// clean up
+		super.onWebSocketClose(session, statusCode, reason);
+	}
+
+	@OnWebSocketError
+	public void onWebSocketError(Session session, Throwable throwable) {
+		/* ignored */
+	}
+
+	@OnWebSocketMessage
+	public void onWebSocketMessage(Session session, String message) {
+		/* ignored */
+	}
+
+	private boolean sendTradePresences(Session session, List<TradePresenceData> tradePresences) {
+		try {
+			StringWriter stringWriter = new StringWriter();
+			marshall(stringWriter, tradePresences);
+
+			String output = stringWriter.toString();
+			session.getRemote().sendStringByFuture(output);
+		} catch (IOException e) {
+			// No output this time?
+			return false;
+		}
+
+		return true;
+	}
+
+	private static void populateCurrentInfo() {
+		// We want ALL trade presences
+		TradeBot.getInstance().getAllTradePresences().stream()
+				.forEach(TradePresenceWebSocket::mergePresence);
+	}
+
+	/** Merge trade presence into cache of current entries, returns true if cache was updated. */
+	private static boolean mergePresence(TradePresenceData tradePresence) {
+		// Put/replace for this publickey making sure we keep newest timestamp
+		String pubKey58 = Base58.encode(tradePresence.getPublicKey());
+
+		TradePresenceData newEntry = currentEntries.compute(pubKey58, (k, v) -> v == null || v.getTimestamp() < tradePresence.getTimestamp() ? tradePresence : v);
+
+		return newEntry == tradePresence;
+	}
+
+	private static void removeOldEntries() {
+		long now = NTP.getTime();
+
+		currentEntries.values().removeIf(v -> v.getTimestamp() < now);
+	}
+
+}

src/main/java/org/qortal/arbitrary/ArbitraryDataBuildQueueItem.java

@@ -13,8 +13,11 @@ public class ArbitraryDataBuildQueueItem extends ArbitraryDataResource {
     private final Long creationTimestamp;
     private Long buildStartTimestamp = null;
     private Long buildEndTimestamp = null;
+    private Integer priority = 0;
     private boolean failed = false;
 
+    private static int HIGH_PRIORITY_THRESHOLD = 5;
+
     /* The maximum amount of time to spend on a single build */
     // TODO: interrupt an in-progress build
     public static long BUILD_TIMEOUT = 60*1000L; // 60 seconds
@@ -27,13 +30,20 @@ public class ArbitraryDataBuildQueueItem extends ArbitraryDataResource {
         this.creationTimestamp = NTP.getTime();
     }
 
+    public void prepareForBuild() {
+        this.buildStartTimestamp = NTP.getTime();
+    }
+
     public void build() throws IOException, DataException, MissingDataException {
         Long now = NTP.getTime();
         if (now == null) {
+            this.buildStartTimestamp = null;
             throw new DataException("NTP time hasn't synced yet");
         }
 
-        this.buildStartTimestamp = now;
+        if (this.buildStartTimestamp == null) {
+            this.buildStartTimestamp = now;
+        }
         ArbitraryDataReader arbitraryDataReader =
                 new ArbitraryDataReader(this.resourceId, this.resourceIdType, this.service, this.identifier);
 
@@ -70,6 +80,21 @@ public class ArbitraryDataBuildQueueItem extends ArbitraryDataResource {
         return this.buildStartTimestamp;
     }
 
+    public Integer getPriority() {
+        if (this.priority != null) {
+            return this.priority;
+        }
+        return 0;
+    }
+
+    public void setPriority(Integer priority) {
+        this.priority = priority;
+    }
+
+    public boolean isHighPriority() {
+        return this.priority >= HIGH_PRIORITY_THRESHOLD;
+    }
+
     public void setFailed(boolean failed) {
         this.failed = failed;
     }

src/main/java/org/qortal/arbitrary/ArbitraryDataCache.java

@@ -61,6 +61,9 @@ public class ArbitraryDataCache {
         }
 
         // No need to invalidate the cache
+        // Remember that it's up to date, so that we won't check again for a while
+        ArbitraryDataManager.getInstance().addResourceToCache(this.getArbitraryDataResource());
+
         return false;
     }
 
@@ -84,14 +87,7 @@ public class ArbitraryDataCache {
 
         // If the state's sig doesn't match the latest transaction's sig, we need to invalidate
         // This means that an updated layer is available
-        if (this.shouldInvalidateDueToSignatureMismatch()) {
-
-            // Add to the in-memory cache first, so that we won't check again for a while
-            ArbitraryDataManager.getInstance().addResourceToCache(this.getArbitraryDataResource());
-            return true;
-        }
-
-        return false;
+        return this.shouldInvalidateDueToSignatureMismatch();
     }
 
     /**

src/main/java/org/qortal/arbitrary/ArbitraryDataFile.java

@@ -53,7 +53,8 @@ public class ArbitraryDataFile {
     private static final Logger LOGGER = LogManager.getLogger(ArbitraryDataFile.class);
 
     public static final long MAX_FILE_SIZE = 500 * 1024 * 1024; // 500MiB
-    public static final int CHUNK_SIZE = 1 * 1024 * 1024; // 1MiB
+    protected static final int MAX_CHUNK_SIZE = 1 * 1024 * 1024; // 1MiB
+    public static final int CHUNK_SIZE = 512 * 1024; // 0.5MiB
     public static int SHORT_DIGEST_LENGTH = 8;
 
     protected Path filePath;
@@ -72,7 +73,6 @@ public class ArbitraryDataFile {
     }
 
     public ArbitraryDataFile(String hash58, byte[] signature) throws DataException {
-        this.createDataDirectory();
         this.filePath = ArbitraryDataFile.getOutputFilePath(hash58, signature, false);
         this.chunks = new ArrayList<>();
         this.hash58 = hash58;
@@ -94,14 +94,9 @@ public class ArbitraryDataFile {
         try (FileOutputStream outputStream = new FileOutputStream(outputFile)) {
             outputStream.write(fileContent);
             this.filePath = outputFilePath;
-            // Verify hash
-            if (!this.hash58.equals(this.digest58())) {
-                LOGGER.error("Hash {} does not match file digest {}", this.hash58, this.digest58());
-                this.delete();
-                throw new DataException("Data file digest validation failed");
-            }
         } catch (IOException e) {
-            throw new DataException("Unable to write data to file");
+            this.delete();
+            throw new DataException(String.format("Unable to write data with hash %s: %s", this.hash58, e.getMessage()));
         }
     }
 
@@ -110,6 +105,9 @@ public class ArbitraryDataFile {
     }
 
     public static ArbitraryDataFile fromHash(byte[] hash, byte[] signature) throws DataException {
+        if (hash == null) {
+            return null;
+        }
         return ArbitraryDataFile.fromHash58(Base58.encode(hash), signature);
     }
 
@@ -146,19 +144,6 @@ public class ArbitraryDataFile {
         return ArbitraryDataFile.fromPath(Paths.get(file.getPath()), signature);
     }
 
-    private boolean createDataDirectory() {
-        // Create the data directory if it doesn't exist
-        String dataPath = Settings.getInstance().getDataPath();
-        Path dataDirectory = Paths.get(dataPath);
-        try {
-            Files.createDirectories(dataDirectory);
-        } catch (IOException e) {
-            LOGGER.error("Unable to create data directory");
-            return false;
-        }
-        return true;
-    }
-
     private Path copyToDataDirectory(Path sourcePath, byte[] signature) throws DataException {
         if (this.hash58 == null || this.filePath == null) {
             return null;
@@ -366,6 +351,21 @@ public class ArbitraryDataFile {
         return false;
     }
 
+    public boolean delete(int attempts) {
+        // Keep trying to delete the data until it is deleted, or we reach 10 attempts
+        for (int i=0; i<attempts; i++) {
+            if (this.delete()) {
+                return true;
+            }
+            try {
+                Thread.sleep(1000L);
+            } catch (InterruptedException e) {
+                // Fall through to exit method
+            }
+        }
+        return false;
+    }
+
     public boolean deleteAllChunks() {
         boolean success = false;
 
@@ -439,6 +439,11 @@ public class ArbitraryDataFile {
                 return chunk.exists();
             }
         }
+        if (Arrays.equals(hash, this.metadataHash)) {
+            if (this.metadataFile != null) {
+                return this.metadataFile.exists();
+            }
+        }
         if (Arrays.equals(this.getHash(), hash)) {
             return this.exists();
         }
@@ -455,9 +460,6 @@ public class ArbitraryDataFile {
 
             if (this.metadataFile == null) {
                 this.metadataFile = ArbitraryDataFile.fromHash(this.metadataHash, this.signature);
-                if (!metadataFile.exists()) {
-                    return false;
-                }
             }
 
             // If the metadata file doesn't exist, we can't check if we have the chunks
@@ -471,6 +473,14 @@ public class ArbitraryDataFile {
 
             // Read the metadata
             List<byte[]> chunks = metadata.getChunks();
+
+            // If the chunks array is empty, then this resource has no chunks,
+            // so we must return false to avoid confusing the caller.
+            if (chunks.isEmpty()) {
+                return false;
+            }
+
+            // Otherwise, we need to check each chunk individually
             for (byte[] chunkHash : chunks) {
                 ArbitraryDataFileChunk chunk = ArbitraryDataFileChunk.fromHash(chunkHash, this.signature);
                 if (!chunk.exists()) {
@@ -496,9 +506,6 @@ public class ArbitraryDataFile {
 
             if (this.metadataFile == null) {
                 this.metadataFile = ArbitraryDataFile.fromHash(this.metadataHash, this.signature);
-                if (!metadataFile.exists()) {
-                    return false;
-                }
             }
 
             // If the metadata file doesn't exist, we can't check if we have any chunks
@@ -540,6 +547,50 @@ public class ArbitraryDataFile {
         return false;
     }
 
+    /**
+     * Retrieve a list of file hashes for this transaction that we do not hold locally
+     *
+     * @return a List of chunk hashes, or null if we are unable to determine what is missing
+     */
+    public List<byte[]> missingHashes() {
+        List<byte[]> missingHashes = new ArrayList<>();
+        try {
+            if (this.metadataHash == null) {
+                // We don't have any metadata so can't check if we have the chunks
+                // Even if this transaction has no chunks, we don't have the file either (already checked above)
+                return null;
+            }
+
+            if (this.metadataFile == null) {
+                this.metadataFile = ArbitraryDataFile.fromHash(this.metadataHash, this.signature);
+            }
+
+            // If the metadata file doesn't exist, we can't check if we have the chunks
+            if (!metadataFile.getFilePath().toFile().exists()) {
+                return null;
+            }
+
+            if (this.metadata == null) {
+                this.setMetadata(new ArbitraryDataTransactionMetadata(this.metadataFile.getFilePath()));
+            }
+
+            // Read the metadata
+            List<byte[]> chunks = metadata.getChunks();
+            for (byte[] chunkHash : chunks) {
+                ArbitraryDataFileChunk chunk = ArbitraryDataFileChunk.fromHash(chunkHash, this.signature);
+                if (!chunk.exists()) {
+                    missingHashes.add(chunkHash);
+                }
+            }
+
+            return missingHashes;
+
+        } catch (DataException e) {
+            // Something went wrong, so we can't make a sensible decision
+            return null;
+        }
+    }
+
     public boolean containsChunk(byte[] hash) {
         for (ArbitraryDataFileChunk chunk : this.chunks) {
             if (Arrays.equals(hash, chunk.getHash())) {
@@ -728,6 +779,10 @@ public class ArbitraryDataFile {
         this.loadMetadata();
     }
 
+    public ArbitraryDataTransactionMetadata getMetadata() {
+        return this.metadata;
+    }
+
     @Override
     public String toString() {
         return this.shortHash58();

src/main/java/org/qortal/arbitrary/ArbitraryDataFileChunk.java

@@ -40,8 +40,8 @@ public class ArbitraryDataFileChunk extends ArbitraryDataFile {
         try {
             // Validate the file size (chunks have stricter limits)
             long fileSize = Files.size(this.filePath);
-            if (fileSize > CHUNK_SIZE) {
-                LOGGER.error(String.format("DataFileChunk is too large: %d bytes (max chunk size: %d bytes)", fileSize, CHUNK_SIZE));
+            if (fileSize > MAX_CHUNK_SIZE) {
+                LOGGER.error(String.format("DataFileChunk is too large: %d bytes (max chunk size: %d bytes)", fileSize, MAX_CHUNK_SIZE));
                 return ValidationResult.FILE_TOO_LARGE;
             }
 

src/main/java/org/qortal/arbitrary/ArbitraryDataReader.java

@@ -122,10 +122,22 @@ public class ArbitraryDataReader {
      * This adds the build task to a queue, and the result will be cached when complete
      * To check the status of the build, periodically call isCachedDataAvailable()
      * Once it returns true, you can then use getFilePath() to access the data itself.
+     *
+     * @param overwrite - set to true to force rebuild an existing cache
      * @return true if added or already present in queue; false if not
      */
-    public boolean loadAsynchronously() {
-        return ArbitraryDataBuildManager.getInstance().addToBuildQueue(this.createQueueItem());
+    public boolean loadAsynchronously(boolean overwrite, int priority) {
+        ArbitraryDataCache cache = new ArbitraryDataCache(this.uncompressedPath, overwrite,
+                this.resourceId, this.resourceIdType, this.service, this.identifier);
+        if (cache.isCachedDataAvailable()) {
+            // Use cached data
+            this.filePath = this.uncompressedPath;
+            return true;
+        }
+
+        ArbitraryDataBuildQueueItem item = this.createQueueItem();
+        item.setPriority(priority);
+        return ArbitraryDataBuildManager.getInstance().addToBuildQueue(item);
     }
 
     /**
@@ -158,6 +170,7 @@ public class ArbitraryDataReader {
             this.validate();
 
         } catch (DataException e) {
+            LOGGER.info("DataException when trying to load QDN resource", e);
             this.deleteWorkingDirectory();
             throw new DataException(e.getMessage());
 
@@ -196,8 +209,13 @@ public class ArbitraryDataReader {
      * serve a cached version of the resource for subsequent requests.
      * @throws IOException
      */
-    private void deleteWorkingDirectory() throws IOException {
-        FilesystemUtils.safeDeleteDirectory(this.workingPath, true);
+    private void deleteWorkingDirectory() {
+        try {
+            FilesystemUtils.safeDeleteDirectory(this.workingPath, true);
+        } catch (IOException e) {
+            // Ignore failures as this isn't an essential step
+            LOGGER.info("Unable to delete working path {}: {}", this.workingPath, e.getMessage());
+        }
     }
 
     private void createUncompressedDirectory() throws DataException {
@@ -363,7 +381,7 @@ public class ArbitraryDataReader {
                 }
 
                 // Throw a missing data exception, which allows subsequent layers to fetch data
-                LOGGER.debug(message);
+                LOGGER.trace(message);
                 throw new MissingDataException(message);
             }
         }
@@ -391,13 +409,28 @@ public class ArbitraryDataReader {
     }
 
     private void decrypt() throws DataException {
+        try {
+            // First try with explicit parameters (CBC mode with PKCS5 padding)
+            this.decryptUsingAlgo("AES/CBC/PKCS5Padding");
+
+        } catch (DataException e) {
+            LOGGER.info("Unable to decrypt using specific parameters: {}", e.getMessage());
+            // Something went wrong, so fall back to default AES params (necessary for legacy resource support)
+            this.decryptUsingAlgo("AES");
+
+            // TODO: delete files and block this resource if privateDataEnabled is false and the second attempt fails too
+        }
+    }
+
+    private void decryptUsingAlgo(String algorithm) throws DataException {
         // Decrypt if we have the secret key.
         byte[] secret = this.secret58 != null ? Base58.decode(this.secret58) : null;
         if (secret != null && secret.length == Transformer.AES256_LENGTH) {
             try {
+                LOGGER.info("Decrypting using algorithm {}...", algorithm);
                 Path unencryptedPath = Paths.get(this.workingPath.toString(), "zipped.zip");
                 SecretKey aesKey = new SecretKeySpec(secret, 0, secret.length, "AES");
-                AES.decryptFile("AES", aesKey, this.filePath.toString(), unencryptedPath.toString());
+                AES.decryptFile(algorithm, aesKey, this.filePath.toString(), unencryptedPath.toString());
 
                 // Replace filePath pointer with the encrypted file path
                 // Don't delete the original ArbitraryDataFile, as this is handled in the cleanup phase
@@ -405,8 +438,8 @@ public class ArbitraryDataReader {
 
             } catch (NoSuchAlgorithmException | InvalidAlgorithmParameterException | NoSuchPaddingException
                     | BadPaddingException | IllegalBlockSizeException | IOException | InvalidKeyException e) {
-                // TODO: delete files and block this resource if privateDataEnabled is false
-                throw new DataException(String.format("Unable to decrypt file at path %s: %s", this.filePath, e.getMessage()));
+                LOGGER.info(String.format("Exception when decrypting using algorithm %s", algorithm), e);
+                throw new DataException(String.format("Unable to decrypt file at path %s using algorithm %s: %s", this.filePath, algorithm, e.getMessage()));
             }
         } else {
             // Assume it is unencrypted. This will be the case when we have built a custom path by combining
@@ -446,12 +479,23 @@ public class ArbitraryDataReader {
             throw new DataException(String.format("Unable to unzip file: %s", e.getMessage()));
         }
 
-        // Replace filePath pointer with the uncompressed file path
+        if (!this.uncompressedPath.toFile().exists()) {
+            throw new DataException(String.format("Unable to unzip file: %s", this.filePath));
+        }
+
+        // Delete original compressed file
         if (FilesystemUtils.pathInsideDataOrTempPath(this.filePath)) {
             if (Files.exists(this.filePath)) {
-                Files.delete(this.filePath);
+                try {
+                    Files.delete(this.filePath);
+                } catch (IOException e) {
+                    // Ignore failures as this isn't an essential step
+                    LOGGER.info("Unable to delete file at path {}", this.filePath);
+                }
             }
         }
+
+        // Replace filePath pointer with the uncompressed file path
         this.filePath = this.uncompressedPath;
     }
 

src/main/java/org/qortal/arbitrary/ArbitraryDataRenderer.java

@@ -9,7 +9,6 @@ import org.qortal.arbitrary.ArbitraryDataFile.*;
 import org.qortal.arbitrary.exception.MissingDataException;
 import org.qortal.arbitrary.misc.Service;
 import org.qortal.controller.Controller;
-import org.qortal.repository.DataException;
 import org.qortal.settings.Settings;
 
 import javax.servlet.ServletContext;
@@ -35,6 +34,7 @@ public class ArbitraryDataRenderer {
     private final String resourceId;
     private final ResourceIdType resourceIdType;
     private final Service service;
+    private String theme = "light";
     private String inPath;
     private final String secret58;
     private final String prefix;
@@ -77,8 +77,8 @@ public class ArbitraryDataRenderer {
             if (!arbitraryDataReader.isCachedDataAvailable()) {
                 // If async is requested, show a loading screen whilst build is in progress
                 if (async) {
-                    arbitraryDataReader.loadAsynchronously();
-                    return this.getLoadingResponse(service, resourceId);
+                    arbitraryDataReader.loadAsynchronously(false, 10);
+                    return this.getLoadingResponse(service, resourceId, theme);
                 }
 
                 // Otherwise, loop until we have data
@@ -119,7 +119,8 @@ public class ArbitraryDataRenderer {
                 // HTML file - needs to be parsed
                 byte[] data = Files.readAllBytes(Paths.get(filePath)); // TODO: limit file size that can be read into memory
                 HTMLParser htmlParser = new HTMLParser(resourceId, inPath, prefix, usePrefix, data);
-                htmlParser.setDocumentBaseUrl();
+                htmlParser.addAdditionalHeaderTags();
+                response.addHeader("Content-Security-Policy", "default-src 'self' 'unsafe-inline' 'unsafe-eval'; media-src 'self' blob:; img-src 'self' data: blob:;");
                 response.setContentType(context.getMimeType(filename));
                 response.setContentLength(htmlParser.getData().length);
                 response.getOutputStream().write(htmlParser.getData());
@@ -128,6 +129,7 @@ public class ArbitraryDataRenderer {
                 // Regular file - can be streamed directly
                 File file = new File(filePath);
                 FileInputStream inputStream = new FileInputStream(file);
+                response.addHeader("Content-Security-Policy", "default-src 'self'");
                 response.setContentType(context.getMimeType(filename));
                 int bytesRead, length = 0;
                 byte[] buffer = new byte[10240];
@@ -170,7 +172,7 @@ public class ArbitraryDataRenderer {
         return userPath;
     }
 
-    private HttpServletResponse getLoadingResponse(Service service, String name) {
+    private HttpServletResponse getLoadingResponse(Service service, String name, String theme) {
         String responseString = "";
         URL url = Resources.getResource("loading/index.html");
         try {
@@ -179,6 +181,7 @@ public class ArbitraryDataRenderer {
             // Replace vars
             responseString = responseString.replace("%%SERVICE%%", service.toString());
             responseString = responseString.replace("%%NAME%%", name);
+            responseString = responseString.replace("%%THEME%%", theme);
 
         } catch (IOException e) {
             LOGGER.info("Unable to show loading screen: {}", e.getMessage());
@@ -209,4 +212,8 @@ public class ArbitraryDataRenderer {
         return indexFiles;
     }
 
+    public void setTheme(String theme) {
+        this.theme = theme;
+    }
+
 }

src/main/java/org/qortal/arbitrary/ArbitraryDataResource.java

@@ -1,9 +1,13 @@
 package org.qortal.arbitrary;
 
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
 import org.qortal.arbitrary.ArbitraryDataFile.ResourceIdType;
+import org.qortal.arbitrary.metadata.ArbitraryDataTransactionMetadata;
 import org.qortal.arbitrary.misc.Service;
 import org.qortal.controller.arbitrary.ArbitraryDataBuildManager;
 import org.qortal.controller.arbitrary.ArbitraryDataManager;
+import org.qortal.controller.arbitrary.ArbitraryDataStorageManager;
 import org.qortal.data.arbitrary.ArbitraryResourceStatus;
 import org.qortal.data.transaction.ArbitraryTransactionData;
 import org.qortal.list.ResourceListManager;
@@ -25,6 +29,8 @@ import static org.qortal.data.arbitrary.ArbitraryResourceStatus.Status;
 
 public class ArbitraryDataResource {
 
+    private static final Logger LOGGER = LogManager.getLogger(ArbitraryDataResource.class);
+
     protected final String resourceId;
     protected final ResourceIdType resourceIdType;
     protected final Service service;
@@ -32,7 +38,10 @@ public class ArbitraryDataResource {
 
     private List<ArbitraryTransactionData> transactions;
     private ArbitraryTransactionData latestPutTransaction;
+    private ArbitraryTransactionData latestTransaction;
     private int layerCount;
+    private Integer localChunkCount = null;
+    private Integer totalChunkCount = null;
 
     public ArbitraryDataResource(String resourceId, ResourceIdType resourceIdType, Service service, String identifier) {
         this.resourceId = resourceId.toLowerCase();
@@ -46,50 +55,83 @@ public class ArbitraryDataResource {
         this.identifier = identifier;
     }
 
-    public ArbitraryResourceStatus getStatus() {
+    public ArbitraryResourceStatus getStatus(boolean quick) {
+        // Calculate the chunk counts
+        // Avoid this for "quick" statuses, to speed things up
+        if (!quick) {
+            this.calculateChunkCounts();
+        }
+
         if (resourceIdType != ResourceIdType.NAME) {
             // We only support statuses for resources with a name
-            return new ArbitraryResourceStatus(Status.UNSUPPORTED);
+            return new ArbitraryResourceStatus(Status.UNSUPPORTED, this.localChunkCount, this.totalChunkCount);
         }
 
         // Check if the name is blocked
         if (ResourceListManager.getInstance()
                 .listContains("blockedNames", this.resourceId, false)) {
-            return new ArbitraryResourceStatus(Status.BLOCKED);
+            return new ArbitraryResourceStatus(Status.BLOCKED, this.localChunkCount, this.totalChunkCount);
+        }
+
+        // Check if a build has failed
+        ArbitraryDataBuildQueueItem queueItem =
+                new ArbitraryDataBuildQueueItem(resourceId, resourceIdType, service, identifier);
+        if (ArbitraryDataBuildManager.getInstance().isInFailedBuildsList(queueItem)) {
+            return new ArbitraryResourceStatus(Status.BUILD_FAILED, this.localChunkCount, this.totalChunkCount);
         }
 
         // Firstly check the cache to see if it's already built
         ArbitraryDataReader arbitraryDataReader = new ArbitraryDataReader(
                 resourceId, resourceIdType, service, identifier);
         if (arbitraryDataReader.isCachedDataAvailable()) {
-            return new ArbitraryResourceStatus(Status.READY);
-        }
-
-        // Next check if there's a build in progress
-        ArbitraryDataBuildQueueItem queueItem =
-                new ArbitraryDataBuildQueueItem(resourceId, resourceIdType, service, identifier);
-        if (ArbitraryDataBuildManager.getInstance().isInBuildQueue(queueItem)) {
-            return new ArbitraryResourceStatus(Status.BUILDING);
-        }
-
-        // Check if a build has failed
-        if (ArbitraryDataBuildManager.getInstance().isInFailedBuildsList(queueItem)) {
-            return new ArbitraryResourceStatus(Status.BUILD_FAILED);
+            return new ArbitraryResourceStatus(Status.READY, this.localChunkCount, this.totalChunkCount);
         }
 
         // Check if we have all data locally for this resource
         if (!this.allFilesDownloaded()) {
             if (this.isDownloading()) {
-                return new ArbitraryResourceStatus(Status.DOWNLOADING);
+                return new ArbitraryResourceStatus(Status.DOWNLOADING, this.localChunkCount, this.totalChunkCount);
             }
             else if (this.isDataPotentiallyAvailable()) {
-                return new ArbitraryResourceStatus(Status.NOT_STARTED);
+                return new ArbitraryResourceStatus(Status.PUBLISHED, this.localChunkCount, this.totalChunkCount);
             }
-            return new ArbitraryResourceStatus(Status.MISSING_DATA);
+            return new ArbitraryResourceStatus(Status.MISSING_DATA, this.localChunkCount, this.totalChunkCount);
+        }
+
+        // Check if there's a build in progress
+        if (ArbitraryDataBuildManager.getInstance().isInBuildQueue(queueItem)) {
+            return new ArbitraryResourceStatus(Status.BUILDING, this.localChunkCount, this.totalChunkCount);
         }
 
         // We have all data locally
-        return new ArbitraryResourceStatus(Status.DOWNLOADED);
+        return new ArbitraryResourceStatus(Status.DOWNLOADED, this.localChunkCount, this.totalChunkCount);
+    }
+
+    public ArbitraryDataTransactionMetadata getLatestTransactionMetadata() {
+        this.fetchLatestTransaction();
+
+        if (latestTransaction != null) {
+            byte[] signature = latestTransaction.getSignature();
+            byte[] metadataHash = latestTransaction.getMetadataHash();
+            if (metadataHash == null) {
+                // This resource doesn't have metadata
+                return null;
+            }
+
+            try {
+                ArbitraryDataFile metadataFile = ArbitraryDataFile.fromHash(metadataHash, signature);
+                if (metadataFile.exists()) {
+                    ArbitraryDataTransactionMetadata transactionMetadata = new ArbitraryDataTransactionMetadata(metadataFile.getFilePath());
+                    transactionMetadata.read();
+                    return transactionMetadata;
+                }
+
+            } catch (DataException | IOException e) {
+                // Do nothing
+            }
+        }
+
+        return null;
     }
 
     public boolean delete() {
@@ -112,6 +154,9 @@ public class ArbitraryDataResource {
             // Also delete cached data for the entire resource
             this.deleteCache();
 
+            // Invalidate the hosted transactions cache as we have removed an item
+            ArbitraryDataStorageManager.getInstance().invalidateHostedTransactionsCache();
+
             return true;
 
         } catch (DataException | IOException e) {
@@ -120,15 +165,31 @@ public class ArbitraryDataResource {
     }
 
     public void deleteCache() throws IOException {
+        // Don't delete anything if there's a build in progress
+        ArbitraryDataBuildQueueItem queueItem =
+                new ArbitraryDataBuildQueueItem(resourceId, resourceIdType, service, identifier);
+        if (ArbitraryDataBuildManager.getInstance().isInBuildQueue(queueItem)) {
+            return;
+        }
+
         String baseDir = Settings.getInstance().getTempDataPath();
         String identifier = this.identifier != null ?  this.identifier : "default";
         Path cachePath = Paths.get(baseDir, "reader", this.resourceIdType.toString(), this.resourceId, this.service.toString(), identifier);
         if (cachePath.toFile().exists()) {
-            FilesystemUtils.safeDeleteDirectory(cachePath, true);
+            boolean success = FilesystemUtils.safeDeleteDirectory(cachePath, true);
+            if (success) {
+                LOGGER.info("Cleared cache for resource {}", this.toString());
+            }
         }
     }
 
     private boolean allFilesDownloaded() {
+        // Use chunk counts to speed things up if we can
+        if (this.localChunkCount != null && this.totalChunkCount != null &&
+                this.localChunkCount >= this.totalChunkCount) {
+            return true;
+        }
+
         try {
             this.fetchTransactions();
 
@@ -147,6 +208,25 @@ public class ArbitraryDataResource {
         }
     }
 
+    private void calculateChunkCounts() {
+        try {
+            this.fetchTransactions();
+
+            List<ArbitraryTransactionData> transactionDataList = new ArrayList<>(this.transactions);
+            int localChunkCount = 0;
+            int totalChunkCount = 0;
+
+            for (ArbitraryTransactionData transactionData : transactionDataList) {
+                localChunkCount += ArbitraryTransactionUtils.ourChunkCount(transactionData);
+                totalChunkCount += ArbitraryTransactionUtils.totalChunkCount(transactionData);
+            }
+
+            this.localChunkCount = localChunkCount;
+            this.totalChunkCount = totalChunkCount;
+
+        } catch (DataException e) {}
+    }
+
     private boolean isRateLimited() {
         try {
             this.fetchTransactions();
@@ -255,6 +335,32 @@ public class ArbitraryDataResource {
 
             this.transactions = transactionDataList;
             this.layerCount = transactionDataList.size();
+
+        } catch (DataException e) {
+            LOGGER.info(String.format("Repository error when fetching transactions for resource %s: %s", this, e.getMessage()));
+        }
+    }
+
+    private void fetchLatestTransaction() {
+        if (this.latestTransaction != null) {
+            // Already fetched
+            return;
+        }
+
+        try (final Repository repository = RepositoryManager.getRepository()) {
+
+            // Get the most recent transaction
+            ArbitraryTransactionData latestTransaction = repository.getArbitraryRepository()
+                    .getLatestTransaction(this.resourceId, this.service, null, this.identifier);
+            if (latestTransaction == null) {
+                String message = String.format("Couldn't find transaction for name %s, service %s and identifier %s",
+                        this.resourceId, this.service, this.identifierString());
+                throw new DataException(message);
+            }
+            this.latestTransaction = latestTransaction;
+
+        } catch (DataException e) {
+            LOGGER.info(String.format("Repository error when fetching latest transaction for resource %s: %s", this, e.getMessage()));
         }
     }
 

src/main/java/org/qortal/arbitrary/ArbitraryDataTransactionBuilder.java

@@ -6,8 +6,9 @@ import org.qortal.arbitrary.exception.MissingDataException;
 import org.qortal.arbitrary.ArbitraryDataFile.ResourceIdType;
 import org.qortal.arbitrary.ArbitraryDataDiff.*;
 import org.qortal.arbitrary.metadata.ArbitraryDataMetadataPatch;
+import org.qortal.arbitrary.metadata.ArbitraryDataTransactionMetadata;
+import org.qortal.arbitrary.misc.Category;
 import org.qortal.arbitrary.misc.Service;
-import org.qortal.block.BlockChain;
 import org.qortal.crypto.Crypto;
 import org.qortal.data.PaymentData;
 import org.qortal.data.transaction.ArbitraryTransactionData;
@@ -27,6 +28,7 @@ import java.io.IOException;
 import java.nio.file.Path;
 import java.util.ArrayList;
 import java.util.List;
+import java.util.Objects;
 import java.util.Random;
 
 public class ArbitraryDataTransactionBuilder {
@@ -51,13 +53,20 @@ public class ArbitraryDataTransactionBuilder {
     private final String identifier;
     private final Repository repository;
 
+    // Metadata
+    private final String title;
+    private final String description;
+    private final List<String> tags;
+    private final Category category;
+
     private int chunkSize = ArbitraryDataFile.CHUNK_SIZE;
 
     private ArbitraryTransactionData arbitraryTransactionData;
     private ArbitraryDataFile arbitraryDataFile;
 
     public ArbitraryDataTransactionBuilder(Repository repository, String publicKey58, Path path, String name,
-                                           Method method, Service service, String identifier) {
+                                           Method method, Service service, String identifier,
+                                           String title, String description, List<String> tags, Category category) {
         this.repository = repository;
         this.publicKey58 = publicKey58;
         this.path = path;
@@ -70,6 +79,12 @@ public class ArbitraryDataTransactionBuilder {
             identifier = null;
         }
         this.identifier = identifier;
+
+        // Metadata (optional)
+        this.title = ArbitraryDataTransactionMetadata.limitTitle(title);
+        this.description = ArbitraryDataTransactionMetadata.limitDescription(description);
+        this.tags = ArbitraryDataTransactionMetadata.limitTags(tags);
+        this.category = category;
     }
 
     public void build() throws DataException {
@@ -108,6 +123,10 @@ public class ArbitraryDataTransactionBuilder {
             return Method.PUT;
         }
 
+        // Get existing metadata and see if it matches the new metadata
+        ArbitraryDataResource resource = new ArbitraryDataResource(this.name, ResourceIdType.NAME, this.service, this.identifier);
+        ArbitraryDataTransactionMetadata existingMetadata = resource.getLatestTransactionMetadata();
+
         try {
             // Check layer count
             int layerCount = reader.getLayerCount();
@@ -118,7 +137,23 @@ public class ArbitraryDataTransactionBuilder {
 
             // Check size of differences between this layer and previous layer
             ArbitraryDataCreatePatch patch = new ArbitraryDataCreatePatch(reader.getFilePath(), this.path, reader.getLatestSignature());
-            patch.create();
+            try {
+                patch.create();
+            }
+            catch (DataException | IOException e) {
+                // Handle matching states separately, as it's best to block transactions with duplicate states
+                if (e.getMessage().equals("Current state matches previous state. Nothing to do.")) {
+                    // Only throw an exception if the metadata is also identical, as well as the data
+                    if (this.isMetadataEqual(existingMetadata)) {
+                        throw new DataException(e.getMessage());
+                    }
+                }
+
+                LOGGER.info("Caught exception when creating patch: {}", e.getMessage());
+                LOGGER.info("Unable to load existing resource - using PUT to overwrite it.");
+                return Method.PUT;
+            }
+
             long diffSize = FilesystemUtils.getDirectorySize(patch.getFinalPath());
             long existingStateSize = FilesystemUtils.getDirectorySize(reader.getFilePath());
             double difference = (double) diffSize / (double) existingStateSize;
@@ -155,11 +190,8 @@ public class ArbitraryDataTransactionBuilder {
             // State is appropriate for a PATCH transaction
             return Method.PATCH;
         }
-        catch (IOException | DataException e) {
-            // Handle matching states separately, as it's best to block transactions with duplicate states
-            if (e.getMessage().equals("Current state matches previous state. Nothing to do.")) {
-                throw new DataException(e.getMessage());
-            }
+        catch (IOException e) {
+            // IMPORTANT: Don't catch DataException here, as they must be passed to the caller
             LOGGER.info("Caught exception: {}", e.getMessage());
             LOGGER.info("Unable to load existing resource - using PUT to overwrite it.");
             return Method.PUT;
@@ -200,7 +232,8 @@ public class ArbitraryDataTransactionBuilder {
             // FUTURE? Use zip compression for directories, or no compression for single files
             // Compression compression = (path.toFile().isDirectory()) ? Compression.ZIP : Compression.NONE;
 
-            ArbitraryDataWriter arbitraryDataWriter = new ArbitraryDataWriter(path, name, service, identifier, method, compression);
+            ArbitraryDataWriter arbitraryDataWriter = new ArbitraryDataWriter(path, name, service, identifier, method,
+                    compression, title, description, tags, category);
             try {
                 arbitraryDataWriter.setChunkSize(this.chunkSize);
                 arbitraryDataWriter.save();
@@ -253,6 +286,22 @@ public class ArbitraryDataTransactionBuilder {
 
     }
 
+    private boolean isMetadataEqual(ArbitraryDataTransactionMetadata existingMetadata) {
+        if (!Objects.equals(existingMetadata.getTitle(), this.title)) {
+            return false;
+        }
+        if (!Objects.equals(existingMetadata.getDescription(), this.description)) {
+            return false;
+        }
+        if (!Objects.equals(existingMetadata.getCategory(), this.category)) {
+            return false;
+        }
+        if (!Objects.equals(existingMetadata.getTags(), this.tags)) {
+            return false;
+        }
+        return true;
+    }
+
     public void computeNonce() throws DataException {
         if (this.arbitraryTransactionData == null) {
             throw new DataException("Arbitrary transaction data is required to compute nonce");

src/main/java/org/qortal/arbitrary/ArbitraryDataWriter.java

@@ -5,6 +5,7 @@ import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
 import org.qortal.arbitrary.exception.MissingDataException;
 import org.qortal.arbitrary.metadata.ArbitraryDataTransactionMetadata;
+import org.qortal.arbitrary.misc.Category;
 import org.qortal.arbitrary.misc.Service;
 import org.qortal.crypto.Crypto;
 import org.qortal.data.transaction.ArbitraryTransactionData.*;
@@ -28,6 +29,10 @@ import java.nio.file.Paths;
 import java.security.InvalidAlgorithmParameterException;
 import java.security.InvalidKeyException;
 import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Objects;
 
 public class ArbitraryDataWriter {
 
@@ -40,6 +45,12 @@ public class ArbitraryDataWriter {
     private final Method method;
     private final Compression compression;
 
+    // Metadata
+    private final String title;
+    private final String description;
+    private final List<String> tags;
+    private final Category category;
+
     private int chunkSize = ArbitraryDataFile.CHUNK_SIZE;
 
     private SecretKey aesKey;
@@ -50,7 +61,8 @@ public class ArbitraryDataWriter {
     private Path compressedPath;
     private Path encryptedPath;
 
-    public ArbitraryDataWriter(Path filePath, String name, Service service, String identifier, Method method, Compression compression) {
+    public ArbitraryDataWriter(Path filePath, String name, Service service, String identifier, Method method, Compression compression,
+                               String title, String description, List<String> tags, Category category) {
         this.filePath = filePath;
         this.name = name;
         this.service = service;
@@ -62,6 +74,12 @@ public class ArbitraryDataWriter {
             identifier = null;
         }
         this.identifier = identifier;
+
+        // Metadata (optional)
+        this.title = ArbitraryDataTransactionMetadata.limitTitle(title);
+        this.description = ArbitraryDataTransactionMetadata.limitDescription(description);
+        this.tags = ArbitraryDataTransactionMetadata.limitTags(tags);
+        this.category = category;
     }
 
     public void save() throws IOException, DataException, InterruptedException, MissingDataException {
@@ -225,7 +243,7 @@ public class ArbitraryDataWriter {
             // Encrypt the file with AES
             LOGGER.info("Encrypting...");
             this.aesKey = AES.generateKey(256);
-            AES.encryptFile("AES", this.aesKey, this.filePath.toString(), this.encryptedPath.toString());
+            AES.encryptFile("AES/CBC/PKCS5Padding", this.aesKey, this.filePath.toString(), this.encryptedPath.toString());
 
             // Delete the input file
             if (FilesystemUtils.pathInsideDataOrTempPath(this.filePath)) {
@@ -258,12 +276,16 @@ public class ArbitraryDataWriter {
 
     private void createMetadataFile() throws IOException, DataException {
         // If we have at least one chunk, we need to create an index file containing their hashes
-        if (this.arbitraryDataFile.chunkCount() > 1) {
+        if (this.needsMetadataFile()) {
             // Create the JSON file
             Path chunkFilePath = Paths.get(this.workingPath.toString(), "metadata.json");
-            ArbitraryDataTransactionMetadata chunkMetadata = new ArbitraryDataTransactionMetadata(chunkFilePath);
-            chunkMetadata.setChunks(this.arbitraryDataFile.chunkHashList());
-            chunkMetadata.write();
+            ArbitraryDataTransactionMetadata metadata = new ArbitraryDataTransactionMetadata(chunkFilePath);
+            metadata.setTitle(this.title);
+            metadata.setDescription(this.description);
+            metadata.setTags(this.tags);
+            metadata.setCategory(this.category);
+            metadata.setChunks(this.arbitraryDataFile.chunkHashList());
+            metadata.write();
 
             // Create an ArbitraryDataFile from the JSON file (we don't have a signature yet)
             ArbitraryDataFile metadataFile = ArbitraryDataFile.fromPath(chunkFilePath, null);
@@ -308,6 +330,20 @@ public class ArbitraryDataWriter {
                     throw new DataException(String.format("Missing chunk %s in metadata file", Base58.encode(chunk)));
                 }
             }
+
+            // Check that the metadata is correct
+            if (!Objects.equals(metadata.getTitle(), this.title)) {
+                throw new DataException("Metadata mismatch: title");
+            }
+            if (!Objects.equals(metadata.getDescription(), this.description)) {
+                throw new DataException("Metadata mismatch: description");
+            }
+            if (!Objects.equals(metadata.getTags(), this.tags)) {
+                throw new DataException("Metadata mismatch: tags");
+            }
+            if (!Objects.equals(metadata.getCategory(), this.category)) {
+                throw new DataException("Metadata mismatch: category");
+            }
         }
     }
 
@@ -330,6 +366,16 @@ public class ArbitraryDataWriter {
         }
     }
 
+    private boolean needsMetadataFile() {
+        if (this.arbitraryDataFile.chunkCount() > 1) {
+            return true;
+        }
+        if (this.title != null || this.description != null || this.tags != null || this.category != null) {
+            return true;
+        }
+        return false;
+    }
+
 
     public ArbitraryDataFile getArbitraryDataFile() {
         return this.arbitraryDataFile;

src/main/java/org/qortal/arbitrary/metadata/ArbitraryDataTransactionMetadata.java

@@ -2,17 +2,28 @@ package org.qortal.arbitrary.metadata;
 
 import org.json.JSONArray;
 import org.json.JSONObject;
+import org.qortal.arbitrary.misc.Category;
 import org.qortal.repository.DataException;
 import org.qortal.utils.Base58;
 
 import java.nio.file.Path;
 import java.util.ArrayList;
 import java.util.Arrays;
+import java.util.Iterator;
 import java.util.List;
 
 public class ArbitraryDataTransactionMetadata extends ArbitraryDataMetadata {
 
     private List<byte[]> chunks;
+    private String title;
+    private String description;
+    private List<String> tags;
+    private Category category;
+
+    private static int MAX_TITLE_LENGTH = 80;
+    private static int MAX_DESCRIPTION_LENGTH = 500;
+    private static int MAX_TAG_LENGTH = 20;
+    private static int MAX_TAGS_COUNT = 5;
 
     public ArbitraryDataTransactionMetadata(Path filePath) {
         super(filePath);
@@ -25,10 +36,37 @@ public class ArbitraryDataTransactionMetadata extends ArbitraryDataMetadata {
             throw new DataException("Transaction metadata JSON string is null");
         }
 
+        JSONObject metadata = new JSONObject(this.jsonString);
+
+        if (metadata.has("title")) {
+            this.title = metadata.getString("title");
+        }
+
+        if (metadata.has("description")) {
+            this.description = metadata.getString("description");
+        }
+
+        List<String> tagsList = new ArrayList<>();
+        if (metadata.has("tags")) {
+            JSONArray tags = metadata.getJSONArray("tags");
+            if (tags != null) {
+                for (int i=0; i<tags.length(); i++) {
+                    String tag = tags.getString(i);
+                    if (tag != null) {
+                        tagsList.add(tag);
+                    }
+                }
+            }
+            this.tags = tagsList;
+        }
+
+        if (metadata.has("category")) {
+            this.category = Category.uncategorizedValueOf(metadata.getString("category"));
+        }
+
         List<byte[]> chunksList = new ArrayList<>();
-        JSONObject cache = new JSONObject(this.jsonString);
-        if (cache.has("chunks")) {
-            JSONArray chunks = cache.getJSONArray("chunks");
+        if (metadata.has("chunks")) {
+            JSONArray chunks = metadata.getJSONArray("chunks");
             if (chunks != null) {
                 for (int i=0; i<chunks.length(); i++) {
                     String chunk = chunks.getString(i);
@@ -45,6 +83,26 @@ public class ArbitraryDataTransactionMetadata extends ArbitraryDataMetadata {
     protected void buildJson() {
         JSONObject outer = new JSONObject();
 
+        if (this.title != null && !this.title.isEmpty()) {
+            outer.put("title", this.title);
+        }
+
+        if (this.description != null && !this.description.isEmpty()) {
+            outer.put("description", this.description);
+        }
+
+        JSONArray tags = new JSONArray();
+        if (this.tags != null) {
+            for (String tag : this.tags) {
+                tags.put(tag);
+            }
+            outer.put("tags", tags);
+        }
+
+        if (this.category != null) {
+            outer.put("category", this.category.toString());
+        }
+
         JSONArray chunks = new JSONArray();
         if (this.chunks != null) {
             for (byte[] chunk : this.chunks) {
@@ -66,6 +124,38 @@ public class ArbitraryDataTransactionMetadata extends ArbitraryDataMetadata {
         return this.chunks;
     }
 
+    public void setTitle(String title) {
+        this.title = title;
+    }
+
+    public String getTitle() {
+        return this.title;
+    }
+
+    public void setDescription(String description) {
+        this.description = description;
+    }
+
+    public String getDescription() {
+        return this.description;
+    }
+
+    public void setTags(List<String> tags) {
+        this.tags = tags;
+    }
+
+    public List<String> getTags() {
+        return this.tags;
+    }
+
+    public void setCategory(Category category) {
+        this.category = category;
+    }
+
+    public Category getCategory() {
+        return this.category;
+    }
+
     public boolean containsChunk(byte[] chunk) {
         for (byte[] c : this.chunks) {
             if (Arrays.equals(c, chunk)) {
@@ -75,4 +165,61 @@ public class ArbitraryDataTransactionMetadata extends ArbitraryDataMetadata {
         return false;
     }
 
+
+    // Static helper methods
+
+    public static String limitTitle(String title) {
+        if (title == null) {
+            return null;
+        }
+        if (title.isEmpty()) {
+            return null;
+        }
+
+        return title.substring(0, Math.min(title.length(), MAX_TITLE_LENGTH));
+    }
+
+    public static String limitDescription(String description) {
+        if (description == null) {
+            return null;
+        }
+        if (description.isEmpty()) {
+            return null;
+        }
+
+        return description.substring(0, Math.min(description.length(), MAX_DESCRIPTION_LENGTH));
+    }
+
+    public static List<String> limitTags(List<String> tags) {
+        if (tags == null) {
+            return null;
+        }
+
+        // Ensure tags list is mutable
+        List<String> mutableTags = new ArrayList<>(tags);
+
+        int tagCount = mutableTags.size();
+        if (tagCount == 0) {
+            return null;
+        }
+
+        // Remove tags over the limit
+        // This is cleaner than truncating, which results in malformed tags
+        // Also remove tags that are empty
+        Iterator iterator = mutableTags.iterator();
+        while (iterator.hasNext()) {
+            String tag = (String) iterator.next();
+            if (tag == null || tag.length() > MAX_TAG_LENGTH || tag.isEmpty()) {
+                iterator.remove();
+            }
+        }
+
+        // Limit the total number of tags
+        if (tagCount > MAX_TAGS_COUNT) {
+            mutableTags = mutableTags.subList(0, MAX_TAGS_COUNT);
+        }
+
+        return mutableTags;
+    }
+
 }

src/main/java/org/qortal/arbitrary/misc/Category.java

@@ -0,0 +1,81 @@
+package org.qortal.arbitrary.misc;
+
+public enum Category {
+    ART("Art and Design"),
+    AUTOMOTIVE("Automotive"),
+    BEAUTY("Beauty"),
+    BOOKS("Books and Reference"),
+    BUSINESS("Business"),
+    COMMUNICATIONS("Communications"),
+    CRYPTOCURRENCY("Cryptocurrency and Blockchain"),
+    CULTURE("Culture"),
+    DATING("Dating"),
+    DESIGN("Design"),
+    ENTERTAINMENT("Entertainment"),
+    EVENTS("Events"),
+    FAITH("Faith and Religion"),
+    FASHION("Fashion"),
+    FINANCE("Finance"),
+    FOOD("Food and Drink"),
+    GAMING("Gaming"),
+    GEOGRAPHY("Geography"),
+    HEALTH("Health"),
+    HISTORY("History"),
+    HOME("Home"),
+    KNOWLEDGE("Knowledge Share"),
+    LANGUAGE("Language"),
+    LIFESTYLE("Lifestyle"),
+    MANUFACTURING("Manufacturing"),
+    MAPS("Maps and Navigation"),
+    MUSIC("Music"),
+    NEWS("News"),
+    OTHER("Other"),
+    PETS("Pets"),
+    PHILOSOPHY("Philosophy"),
+    PHOTOGRAPHY("Photography"),
+    POLITICS("Politics"),
+    PRODUCE("Products and Services"),
+    PRODUCTIVITY("Productivity"),
+    PSYCHOLOGY("Psychology"),
+    QORTAL("Qortal"),
+    SCIENCE("Science"),
+    SELF_CARE("Self Care"),
+    SELF_SUFFICIENCY("Self-Sufficiency and Homesteading"),
+    SHOPPING("Shopping"),
+    SOCIAL("Social"),
+    SOFTWARE("Software"),
+    SPIRITUALITY("Spirituality"),
+    SPORTS("Sports"),
+    STORYTELLING("Storytelling"),
+    TECHNOLOGY("Technology"),
+    TOOLS("Tools"),
+    TRAVEL("Travel"),
+    UNCATEGORIZED("Uncategorized"),
+    VIDEO("Video"),
+    WEATHER("Weather");
+
+    private final String name;
+
+    Category(String name) {
+        this.name = name;
+    }
+
+    public String getName() {
+        return this.name;
+    }
+
+    /**
+     * Same as valueOf() but with fallback to UNCATEGORIZED if there's no match
+     * @param name
+     * @return a Category (using UNCATEGORIZED if no match found)
+     */
+    public static Category uncategorizedValueOf(String name) {
+        try {
+            return Category.valueOf(name);
+        }
+        catch (IllegalArgumentException e) {
+            return Category.UNCATEGORIZED;
+        }
+    }
+
+}

src/main/java/org/qortal/at/QortalATAPI.java

@@ -551,7 +551,7 @@ public class QortalATAPI extends API {
 	 * <p>
 	 * Otherwise, assume B is a public key.
 	 */
-	private Account getAccountFromB(MachineState state) {
+	/*package*/ Account getAccountFromB(MachineState state) {
 		byte[] bBytes = this.getB(state);
 
 		if ((bBytes[0] == Crypto.ADDRESS_VERSION || bBytes[0] == Crypto.AT_ADDRESS_VERSION)

src/main/java/org/qortal/at/QortalFunctionCode.java

@@ -10,9 +10,11 @@ import org.ciyam.at.ExecutionException;
 import org.ciyam.at.FunctionData;
 import org.ciyam.at.IllegalFunctionCodeException;
 import org.ciyam.at.MachineState;
+import org.qortal.account.Account;
 import org.qortal.crosschain.Bitcoin;
 import org.qortal.crypto.Crypto;
 import org.qortal.data.transaction.TransactionData;
+import org.qortal.repository.DataException;
 import org.qortal.settings.Settings;
 
 /**
@@ -160,6 +162,68 @@ public enum QortalFunctionCode {
 		protected void postCheckExecute(FunctionData functionData, MachineState state, short rawFunctionCode) throws ExecutionException {
 			convertAddressInB(Crypto.ADDRESS_VERSION, state);
 		}
+	},
+	/**
+	 * Returns account level of account in B.<br>
+	 * <tt>0x0520</tt><br>
+	 * B should contain either Qortal address or public key,<br>
+	 * e.g. as a result of calling function {@link org.ciyam.at.FunctionCode#PUT_ADDRESS_FROM_TX_IN_A_INTO_B}</code>.
+	 * <p></p>
+	 * Returns account level, or -1 if account unknown.
+	 * <p></p>
+	 * @see QortalATAPI#getAccountFromB(MachineState)
+	 */
+	GET_ACCOUNT_LEVEL_FROM_ACCOUNT_IN_B(0x0520, 0, true) {
+		@Override
+		protected void postCheckExecute(FunctionData functionData, MachineState state, short rawFunctionCode) throws ExecutionException {
+			QortalATAPI api = (QortalATAPI) state.getAPI();
+			Account account = api.getAccountFromB(state);
+
+			Integer accountLevel = null;
+
+			if (account != null) {
+				try {
+					accountLevel = account.getLevel();
+				} catch (DataException e) {
+					throw new RuntimeException("AT API unable to fetch account level?", e);
+				}
+			}
+
+			functionData.returnValue = accountLevel != null
+					? accountLevel.longValue()
+					: -1;
+		}
+	},
+	/**
+	 * Returns account's minted block count of account in B.<br>
+	 * <tt>0x0521</tt><br>
+	 * B should contain either Qortal address or public key,<br>
+	 * e.g. as a result of calling function {@link org.ciyam.at.FunctionCode#PUT_ADDRESS_FROM_TX_IN_A_INTO_B}</code>.
+	 * <p></p>
+	 * Returns account level, or -1 if account unknown.
+	 * <p></p>
+	 * @see QortalATAPI#getAccountFromB(MachineState)
+	 */
+	GET_BLOCKS_MINTED_FROM_ACCOUNT_IN_B(0x0521, 0, true) {
+		@Override
+		protected void postCheckExecute(FunctionData functionData, MachineState state, short rawFunctionCode) throws ExecutionException {
+			QortalATAPI api = (QortalATAPI) state.getAPI();
+			Account account = api.getAccountFromB(state);
+
+			Integer blocksMinted = null;
+
+			if (account != null) {
+				try {
+					blocksMinted = account.getBlocksMinted();
+				} catch (DataException e) {
+					throw new RuntimeException("AT API unable to fetch account's minted block count?", e);
+				}
+			}
+
+			functionData.returnValue = blocksMinted != null
+					? blocksMinted.longValue()
+					: -1;
+		}
 	};
 
 	public final short value;

src/main/java/org/qortal/block/Block.java

@@ -3,18 +3,15 @@ package org.qortal.block;
 import static java.util.Arrays.stream;
 import static java.util.stream.Collectors.toMap;
 
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
 import java.math.BigDecimal;
 import java.math.BigInteger;
 import java.math.RoundingMode;
+import java.nio.charset.StandardCharsets;
 import java.text.DecimalFormat;
 import java.text.NumberFormat;
-import java.util.ArrayList;
-import java.util.Arrays;
-import java.util.HashMap;
-import java.util.HashSet;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
+import java.util.*;
 import java.util.stream.Collectors;
 
 import org.apache.logging.log4j.Level;
@@ -28,8 +25,9 @@ import org.qortal.asset.Asset;
 import org.qortal.at.AT;
 import org.qortal.block.BlockChain.BlockTimingByHeight;
 import org.qortal.block.BlockChain.AccountLevelShareBin;
-import org.qortal.controller.Controller;
+import org.qortal.controller.OnlineAccountsManager;
 import org.qortal.crypto.Crypto;
+import org.qortal.crypto.Qortal25519Extras;
 import org.qortal.data.account.AccountBalanceData;
 import org.qortal.data.account.AccountData;
 import org.qortal.data.account.EligibleQoraHolderData;
@@ -91,7 +89,8 @@ public class Block {
 		ONLINE_ACCOUNT_UNKNOWN(71),
 		ONLINE_ACCOUNT_SIGNATURES_MISSING(72),
 		ONLINE_ACCOUNT_SIGNATURES_MALFORMED(73),
-		ONLINE_ACCOUNT_SIGNATURE_INCORRECT(74);
+		ONLINE_ACCOUNT_SIGNATURE_INCORRECT(74),
+		ONLINE_ACCOUNT_NONCE_INCORRECT(75);
 
 		public final int value;
 
@@ -124,6 +123,8 @@ public class Block {
 
 	/** Remote/imported/loaded AT states */
 	protected List<ATStateData> atStates;
+	/** Remote hash of AT states - in lieu of full AT state data in {@code atStates} */
+	protected byte[] atStatesHash;
 	/** Locally-generated AT states */
 	protected List<ATStateData> ourAtStates;
 	/** Locally-generated AT fees */
@@ -184,8 +185,11 @@ public class Block {
 			if (accountLevel <= 0)
 				return null; // level 0 isn't included in any share bins
 
+			// Select the correct set of share bins based on block height
 			final BlockChain blockChain = BlockChain.getInstance();
-			final AccountLevelShareBin[] shareBinsByLevel = blockChain.getShareBinsByAccountLevel();
+			final AccountLevelShareBin[] shareBinsByLevel = (blockHeight >= blockChain.getSharesByLevelV2Height()) ?
+					blockChain.getShareBinsByAccountLevelV2() : blockChain.getShareBinsByAccountLevelV1();
+
 			if (accountLevel > shareBinsByLevel.length)
 				return null;
 
@@ -198,6 +202,11 @@ public class Block {
 
 		}
 
+		public boolean hasShareBin(AccountLevelShareBin shareBin, int blockHeight) {
+			AccountLevelShareBin ourShareBin = this.getShareBin(blockHeight);
+			return ourShareBin != null && shareBin.id == ourShareBin.id;
+		}
+
 		public long distribute(long accountAmount, Map<String, Long> balanceChanges) {
 			if (this.isRecipientAlsoMinter) {
 				// minter & recipient the same - simpler case
@@ -222,11 +231,10 @@ public class Block {
 			return accountAmount;
 		}
 	}
+
 	/** Always use getExpandedAccounts() to access this, as it's lazy-instantiated. */
 	private List<ExpandedAccount> cachedExpandedAccounts = null;
 
-	/** Opportunistic cache of this block's valid online accounts. Only created by call to isValid(). */
-	private List<OnlineAccountData> cachedValidOnlineAccounts = null;
 	/** Opportunistic cache of this block's valid online reward-shares. Only created by call to isValid(). */
 	private List<RewardShareData> cachedOnlineRewardShares = null;
 
@@ -261,7 +269,7 @@ public class Block {
 	 * Constructs new Block using passed transaction and AT states.
 	 * <p>
 	 * This constructor typically used when receiving a serialized block over the network.
-	 * 
+	 *
 	 * @param repository
 	 * @param blockData
 	 * @param transactions
@@ -287,6 +295,35 @@ public class Block {
 		this.blockData.setTotalFees(totalFees);
 	}
 
+	/**
+	 * Constructs new Block using passed transaction and minimal AT state info.
+	 * <p>
+	 * This constructor typically used when receiving a serialized block over the network.
+	 *
+	 * @param repository
+	 * @param blockData
+	 * @param transactions
+	 * @param atStatesHash
+	 */
+	public Block(Repository repository, BlockData blockData, List<TransactionData> transactions, byte[] atStatesHash) {
+		this(repository, blockData);
+
+		this.transactions = new ArrayList<>();
+
+		long totalFees = 0;
+
+		// We have to sum fees too
+		for (TransactionData transactionData : transactions) {
+			this.transactions.add(Transaction.fromData(repository, transactionData));
+			totalFees += transactionData.getFee();
+		}
+
+		this.atStatesHash = atStatesHash;
+		totalFees += this.blockData.getATFees();
+
+		this.blockData.setTotalFees(totalFees);
+	}
+
 	/**
 	 * Constructs new Block with empty transaction list, using passed minter account.
 	 * 
@@ -319,28 +356,38 @@ public class Block {
 		int version = parentBlock.getNextBlockVersion();
 		byte[] reference = parentBlockData.getSignature();
 
+		// Qortal: minter is always a reward-share, so find actual minter and get their effective minting level
+		int minterLevel = Account.getRewardShareEffectiveMintingLevel(repository, minter.getPublicKey());
+		if (minterLevel == 0) {
+			LOGGER.error("Minter effective level returned zero?");
+			return null;
+		}
+
+		long timestamp = calcTimestamp(parentBlockData, minter.getPublicKey(), minterLevel);
+		long onlineAccountsTimestamp = OnlineAccountsManager.getCurrentOnlineAccountTimestamp();
+
 		// Fetch our list of online accounts
-		List<OnlineAccountData> onlineAccounts = Controller.getInstance().getOnlineAccounts();
+		List<OnlineAccountData> onlineAccounts = OnlineAccountsManager.getInstance().getOnlineAccounts(onlineAccountsTimestamp);
+
+		// If mempow is active, remove any legacy accounts that are missing a nonce
+		if (timestamp >= BlockChain.getInstance().getOnlineAccountsMemoryPoWTimestamp()) {
+			onlineAccounts.removeIf(a -> a.getNonce() == null || a.getNonce() < 0);
+		}
+
 		if (onlineAccounts.isEmpty()) {
 			LOGGER.error("No online accounts - not even our own?");
 			return null;
 		}
 
-		// Find newest online accounts timestamp
-		long onlineAccountsTimestamp = 0;
-		for (OnlineAccountData onlineAccountData : onlineAccounts) {
-			if (onlineAccountData.getTimestamp() > onlineAccountsTimestamp)
-				onlineAccountsTimestamp = onlineAccountData.getTimestamp();
-		}
+		// Load sorted list of reward share public keys into memory, so that the indexes can be obtained.
+		// This is up to 100x faster than querying each index separately. For 4150 reward share keys, it
+		// was taking around 5000ms to query individually, vs 50ms using this approach.
+		List<byte[]> allRewardSharePublicKeys = repository.getAccountRepository().getRewardSharePublicKeys();
 
 		// Map using index into sorted list of reward-shares as key
 		Map<Integer, OnlineAccountData> indexedOnlineAccounts = new HashMap<>();
 		for (OnlineAccountData onlineAccountData : onlineAccounts) {
-			// Disregard online accounts with different timestamps
-			if (onlineAccountData.getTimestamp() != onlineAccountsTimestamp)
-				continue;
-
-			Integer accountIndex = repository.getAccountRepository().getRewardShareIndex(onlineAccountData.getPublicKey());
+			Integer accountIndex = getRewardShareIndex(onlineAccountData.getPublicKey(), allRewardSharePublicKeys);
 			if (accountIndex == null)
 				// Online account (reward-share) with current timestamp but reward-share cancelled
 				continue;
@@ -356,26 +403,43 @@ public class Block {
 		byte[] encodedOnlineAccounts = BlockTransformer.encodeOnlineAccounts(onlineAccountsSet);
 		int onlineAccountsCount = onlineAccountsSet.size();
 
-		// Concatenate online account timestamp signatures (in correct order)
-		byte[] onlineAccountsSignatures = new byte[onlineAccountsCount * Transformer.SIGNATURE_LENGTH];
-		for (int i = 0; i < onlineAccountsCount; ++i) {
-			Integer accountIndex = accountIndexes.get(i);
-			OnlineAccountData onlineAccountData = indexedOnlineAccounts.get(accountIndex);
-			System.arraycopy(onlineAccountData.getSignature(), 0, onlineAccountsSignatures, i * Transformer.SIGNATURE_LENGTH, Transformer.SIGNATURE_LENGTH);
+		// Collate all signatures
+		Collection<byte[]> signaturesToAggregate = indexedOnlineAccounts.values()
+				.stream()
+				.map(OnlineAccountData::getSignature)
+				.collect(Collectors.toList());
+
+		// Aggregated, single signature
+		byte[] onlineAccountsSignatures = Qortal25519Extras.aggregateSignatures(signaturesToAggregate);
+
+		// Add nonces to the end of the online accounts signatures if mempow is active
+		if (timestamp >= BlockChain.getInstance().getOnlineAccountsMemoryPoWTimestamp()) {
+			try {
+				// Create ordered list of nonce values
+				List<Integer> nonces = new ArrayList<>();
+				for (int i = 0; i < onlineAccountsCount; ++i) {
+					Integer accountIndex = accountIndexes.get(i);
+					OnlineAccountData onlineAccountData = indexedOnlineAccounts.get(accountIndex);
+					nonces.add(onlineAccountData.getNonce());
+				}
+
+				// Encode the nonces to a byte array
+				byte[] encodedNonces = BlockTransformer.encodeOnlineAccountNonces(nonces);
+
+				// Append the encoded nonces to the encoded online account signatures
+				ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
+				outputStream.write(onlineAccountsSignatures);
+				outputStream.write(encodedNonces);
+				onlineAccountsSignatures = outputStream.toByteArray();
+			}
+			catch (TransformationException | IOException e) {
+				return null;
+			}
 		}
 
 		byte[] minterSignature = minter.sign(BlockTransformer.getBytesForMinterSignature(parentBlockData,
 				minter.getPublicKey(), encodedOnlineAccounts));
 
-		// Qortal: minter is always a reward-share, so find actual minter and get their effective minting level
-		int minterLevel = Account.getRewardShareEffectiveMintingLevel(repository, minter.getPublicKey());
-		if (minterLevel == 0) {
-			LOGGER.error("Minter effective level returned zero?");
-			return null;
-		}
-
-		long timestamp = calcTimestamp(parentBlockData, minter.getPublicKey(), minterLevel);
-
 		int transactionCount = 0;
 		byte[] transactionsSignature = null;
 		int height = parentBlockData.getHeight() + 1;
@@ -980,49 +1044,68 @@ public class Block {
 		if (this.blockData.getOnlineAccountsSignatures() == null || this.blockData.getOnlineAccountsSignatures().length == 0)
 			return ValidationResult.ONLINE_ACCOUNT_SIGNATURES_MISSING;
 
-		if (this.blockData.getOnlineAccountsSignatures().length != onlineRewardShares.size() * Transformer.SIGNATURE_LENGTH)
-			return ValidationResult.ONLINE_ACCOUNT_SIGNATURES_MALFORMED;
+		final int signaturesLength = Transformer.SIGNATURE_LENGTH;
+		final int noncesLength = onlineRewardShares.size() * Transformer.INT_LENGTH;
+
+		if (this.blockData.getTimestamp() >= BlockChain.getInstance().getOnlineAccountsMemoryPoWTimestamp()) {
+			// We expect nonces to be appended to the online accounts signatures
+			if (this.blockData.getOnlineAccountsSignatures().length != signaturesLength + noncesLength)
+				return ValidationResult.ONLINE_ACCOUNT_SIGNATURES_MALFORMED;
+		} else {
+			if (this.blockData.getOnlineAccountsSignatures().length != signaturesLength)
+				return ValidationResult.ONLINE_ACCOUNT_SIGNATURES_MALFORMED;
+		}
 
 		// Check signatures
 		long onlineTimestamp = this.blockData.getOnlineAccountsTimestamp();
 		byte[] onlineTimestampBytes = Longs.toByteArray(onlineTimestamp);
 
-		// If this block is much older than current online timestamp, then there's no point checking current online accounts
-		List<OnlineAccountData> currentOnlineAccounts = onlineTimestamp < NTP.getTime() - Controller.ONLINE_TIMESTAMP_MODULUS
-				? null
-				: Controller.getInstance().getOnlineAccounts();
-		List<OnlineAccountData> latestBlocksOnlineAccounts = Controller.getInstance().getLatestBlocksOnlineAccounts();
+		byte[] encodedOnlineAccountSignatures = this.blockData.getOnlineAccountsSignatures();
 
-		// Extract online accounts' timestamp signatures from block data
-		List<byte[]> onlineAccountsSignatures = BlockTransformer.decodeTimestampSignatures(this.blockData.getOnlineAccountsSignatures());
+		// Split online account signatures into signature(s) + nonces, then validate the nonces
+		if (this.blockData.getTimestamp() >= BlockChain.getInstance().getOnlineAccountsMemoryPoWTimestamp()) {
+			byte[] extractedSignatures = BlockTransformer.extract(encodedOnlineAccountSignatures, 0, signaturesLength);
+			byte[] extractedNonces = BlockTransformer.extract(encodedOnlineAccountSignatures, signaturesLength, onlineRewardShares.size() * Transformer.INT_LENGTH);
+			encodedOnlineAccountSignatures = extractedSignatures;
 
-		// We'll build up a list of online accounts to hand over to Controller if block is added to chain
-		// and this will become latestBlocksOnlineAccounts (above) to reduce CPU load when we process next block...
-		List<OnlineAccountData> ourOnlineAccounts = new ArrayList<>();
+			List<Integer> nonces = BlockTransformer.decodeOnlineAccountNonces(extractedNonces);
 
-		for (int i = 0; i < onlineAccountsSignatures.size(); ++i) {
-			byte[] signature = onlineAccountsSignatures.get(i);
-			byte[] publicKey = onlineRewardShares.get(i).getRewardSharePublicKey();
+			// Build block's view of online accounts (without signatures, as we don't need them here)
+			Set<OnlineAccountData> onlineAccounts = new HashSet<>();
+			for (int i = 0; i < onlineRewardShares.size(); ++i) {
+				Integer nonce = nonces.get(i);
+				byte[] publicKey = onlineRewardShares.get(i).getRewardSharePublicKey();
 
-			OnlineAccountData onlineAccountData = new OnlineAccountData(onlineTimestamp, signature, publicKey);
-			ourOnlineAccounts.add(onlineAccountData);
+				OnlineAccountData onlineAccountData = new OnlineAccountData(onlineTimestamp, null, publicKey, nonce);
+				onlineAccounts.add(onlineAccountData);
+			}
 
-			// If signature is still current then no need to perform Ed25519 verify
-			if (currentOnlineAccounts != null && currentOnlineAccounts.remove(onlineAccountData))
-				// remove() returned true, so online account still current
-				// and one less entry in currentOnlineAccounts to check next time
-				continue;
+			// Remove those already validated & cached by online accounts manager - no need to re-validate them
+			OnlineAccountsManager.getInstance().removeKnown(onlineAccounts, onlineTimestamp);
 
-			// If signature was okay in latest block then no need to perform Ed25519 verify
-			if (latestBlocksOnlineAccounts != null && latestBlocksOnlineAccounts.contains(onlineAccountData))
-				continue;
-
-			if (!Crypto.verify(publicKey, signature, onlineTimestampBytes))
-				return ValidationResult.ONLINE_ACCOUNT_SIGNATURE_INCORRECT;
+			// Validate the rest
+			for (OnlineAccountData onlineAccount : onlineAccounts)
+				if (!OnlineAccountsManager.getInstance().verifyMemoryPoW(onlineAccount, this.blockData.getTimestamp()))
+					return ValidationResult.ONLINE_ACCOUNT_NONCE_INCORRECT;
 		}
 
+		// Extract online accounts' timestamp signatures from block data. Only one signature if aggregated.
+		List<byte[]> onlineAccountsSignatures = BlockTransformer.decodeTimestampSignatures(encodedOnlineAccountSignatures);
+
+		// Aggregate all public keys
+		Collection<byte[]> publicKeys = onlineRewardShares.stream()
+				.map(RewardShareData::getRewardSharePublicKey)
+				.collect(Collectors.toList());
+
+		byte[] aggregatePublicKey = Qortal25519Extras.aggregatePublicKeys(publicKeys);
+
+		byte[] aggregateSignature = onlineAccountsSignatures.get(0);
+
+		// One-step verification of aggregate signature using aggregate public key
+		if (!Qortal25519Extras.verifyAggregated(aggregatePublicKey, aggregateSignature, onlineTimestampBytes))
+			return ValidationResult.ONLINE_ACCOUNT_SIGNATURE_INCORRECT;
+
 		// All online accounts valid, so save our list of online accounts for potential later use
-		this.cachedValidOnlineAccounts = ourOnlineAccounts;
 		this.cachedOnlineRewardShares = onlineRewardShares;
 
 		return ValidationResult.OK;
@@ -1169,6 +1252,7 @@ public class Block {
 				}
 			}
 		} catch (DataException e) {
+			LOGGER.info("DataException during transaction validation", e);
 			return ValidationResult.TRANSACTION_INVALID;
 		} finally {
 			// Rollback repository changes made by test-processing transactions above
@@ -1195,7 +1279,7 @@ public class Block {
 	 */
 	private ValidationResult areAtsValid() throws DataException {
 		// Locally generated AT states should be valid so no need to re-execute them
-		if (this.ourAtStates == this.getATStates()) // Note object reference compare
+		if (this.ourAtStates != null && this.ourAtStates == this.atStates) // Note object reference compare
 			return ValidationResult.OK;
 
 		// Generate local AT states for comparison
@@ -1209,8 +1293,33 @@ public class Block {
 		if (this.ourAtFees != this.blockData.getATFees())
 			return ValidationResult.AT_STATES_MISMATCH;
 
-		// Note: this.atStates fully loaded thanks to this.getATStates() call above
-		for (int s = 0; s < this.atStates.size(); ++s) {
+		// If we have a single AT states hash then compare that in preference
+		if (this.atStatesHash != null) {
+			int atBytesLength = blockData.getATCount() * BlockTransformer.AT_ENTRY_LENGTH;
+			ByteArrayOutputStream atHashBytes = new ByteArrayOutputStream(atBytesLength);
+
+			try {
+				for (ATStateData atStateData : this.ourAtStates) {
+					atHashBytes.write(atStateData.getATAddress().getBytes(StandardCharsets.UTF_8));
+					atHashBytes.write(atStateData.getStateHash());
+					atHashBytes.write(Longs.toByteArray(atStateData.getFees()));
+				}
+			} catch (IOException e) {
+				throw new DataException("Couldn't validate AT states hash due to serialization issue?", e);
+			}
+
+			byte[] ourAtStatesHash = Crypto.digest(atHashBytes.toByteArray());
+			if (!Arrays.equals(ourAtStatesHash, this.atStatesHash))
+				return ValidationResult.AT_STATES_MISMATCH;
+
+			// Use our AT state data from now on
+			this.atStates = this.ourAtStates;
+			return ValidationResult.OK;
+		}
+
+		// Note: this.atStates fully loaded thanks to this.getATStates() call:
+		this.getATStates();
+		for (int s = 0; s < this.ourAtStates.size(); ++s) {
 			ATStateData ourAtState = this.ourAtStates.get(s);
 			ATStateData theirAtState = this.atStates.get(s);
 
@@ -1368,9 +1477,6 @@ public class Block {
 
 		postBlockTidy();
 
-		// Give Controller our cached, valid online accounts data (if any) to help reduce CPU load for next block
-		Controller.getInstance().pushLatestBlocksOnlineAccounts(this.cachedValidOnlineAccounts);
-
 		// Log some debugging info relating to the block weight calculation
 		this.logDebugInfo();
 	}
@@ -1586,9 +1692,6 @@ public class Block {
 		this.blockData.setHeight(null);
 
 		postBlockTidy();
-
-		// Remove any cached, valid online accounts data from Controller
-		Controller.getInstance().popLatestBlocksOnlineAccounts();
 	}
 
 	protected void orphanTransactionsFromBlock() throws DataException {
@@ -1825,13 +1928,72 @@ public class Block {
 		final List<ExpandedAccount> onlineFounderAccounts = expandedAccounts.stream().filter(expandedAccount -> expandedAccount.isMinterFounder).collect(Collectors.toList());
 		final boolean haveFounders = !onlineFounderAccounts.isEmpty();
 
+		// Select the correct set of share bins based on block height
+		List<AccountLevelShareBin> accountLevelShareBinsForBlock = (this.blockData.getHeight() >= BlockChain.getInstance().getSharesByLevelV2Height()) ?
+				BlockChain.getInstance().getAccountLevelShareBinsV2() : BlockChain.getInstance().getAccountLevelShareBinsV1();
+
 		// Determine reward candidates based on account level
-		List<AccountLevelShareBin> accountLevelShareBins = BlockChain.getInstance().getAccountLevelShareBins();
-		for (int binIndex = 0; binIndex < accountLevelShareBins.size(); ++binIndex) {
-			// Find all accounts in share bin. getShareBin() returns null for minter accounts that are also founders, so they are effectively filtered out.
+		// This needs a deep copy, so the shares can be modified when tiers aren't activated yet
+		List<AccountLevelShareBin> accountLevelShareBins = new ArrayList<>();
+		for (AccountLevelShareBin accountLevelShareBin : accountLevelShareBinsForBlock) {
+			accountLevelShareBins.add((AccountLevelShareBin) accountLevelShareBin.clone());
+		}
+
+		Map<Integer, List<ExpandedAccount>> accountsForShareBin = new HashMap<>();
+
+		// We might need to combine some share bins if they haven't reached the minimum number of minters yet
+		for (int binIndex = accountLevelShareBins.size()-1; binIndex >= 0; --binIndex) {
 			AccountLevelShareBin accountLevelShareBin = accountLevelShareBins.get(binIndex);
-			// Object reference compare is OK as all references are read-only from blockchain config.
-			List<ExpandedAccount> binnedAccounts = expandedAccounts.stream().filter(accountInfo -> accountInfo.getShareBin(this.blockData.getHeight()) == accountLevelShareBin).collect(Collectors.toList());
+
+			// Find all accounts in share bin. getShareBin() returns null for minter accounts that are also founders, so they are effectively filtered out.
+			List<ExpandedAccount> binnedAccounts = expandedAccounts.stream().filter(accountInfo -> accountInfo.hasShareBin(accountLevelShareBin, this.blockData.getHeight())).collect(Collectors.toList());
+			// Add any accounts that have been moved down from a higher tier
+			List<ExpandedAccount> existingBinnedAccounts = accountsForShareBin.get(binIndex);
+			if (existingBinnedAccounts != null)
+				binnedAccounts.addAll(existingBinnedAccounts);
+
+			// Logic below may only apply to higher levels, and only for share bins with a specific range of online accounts
+			if (accountLevelShareBin.levels.get(0) < BlockChain.getInstance().getShareBinActivationMinLevel() ||
+					binnedAccounts.isEmpty() || binnedAccounts.size() >= BlockChain.getInstance().getMinAccountsToActivateShareBin()) {
+				// Add all accounts for this share bin to the accountsForShareBin list
+				accountsForShareBin.put(binIndex, binnedAccounts);
+				continue;
+			}
+
+			// Share bin contains more than one, but less than the minimum number of minters. We treat this share bin
+			// as not activated yet. In these cases, the rewards and minters are combined and paid out to the previous
+			// share bin, to prevent a single or handful of accounts receiving the entire rewards for a share bin.
+			//
+			// Example:
+			//
+			// - Share bin for levels 5 and 6 has 100 minters
+			// - Share bin for levels 7 and 8 has 10 minters
+			//
+			// This is below the minimum of 30, so share bins are reconstructed as follows:
+			//
+			// - Share bin for levels 5 and 6 now contains 110 minters
+			// - Share bin for levels 7 and 8 now contains 0 minters
+			// - Share bin for levels 5 and 6 now pays out rewards for levels 5, 6, 7, and 8
+			// - Share bin for levels 7 and 8 pays zero rewards
+			//
+			// This process is iterative, so will combine several tiers if needed.
+
+			// Designate this share bin as empty
+			accountsForShareBin.put(binIndex, new ArrayList<>());
+
+			// Move the accounts originally intended for this share bin to the previous one
+			accountsForShareBin.put(binIndex - 1, binnedAccounts);
+
+			// Move the block reward from this share bin to the previous one
+			AccountLevelShareBin previousShareBin = accountLevelShareBins.get(binIndex - 1);
+			previousShareBin.share += accountLevelShareBin.share;
+			accountLevelShareBin.share = 0L;
+		}
+
+		// Now loop through (potentially modified) share bins and determine the reward candidates
+		for (int binIndex = 0; binIndex < accountLevelShareBins.size(); ++binIndex) {
+			AccountLevelShareBin accountLevelShareBin = accountLevelShareBins.get(binIndex);
+			List<ExpandedAccount> binnedAccounts = accountsForShareBin.get(binIndex);
 
 			// No online accounts in this bin? Skip to next one
 			if (binnedAccounts.isEmpty())
@@ -1849,7 +2011,7 @@ public class Block {
 		// Fetch list of legacy QORA holders who haven't reached their cap of QORT reward.
 		List<EligibleQoraHolderData> qoraHolders = this.repository.getAccountRepository().getEligibleLegacyQoraHolders(isProcessingNotOrphaning ? null : this.blockData.getHeight());
 		final boolean haveQoraHolders = !qoraHolders.isEmpty();
-		final long qoraHoldersShare = BlockChain.getInstance().getQoraHoldersShare();
+		final long qoraHoldersShare = BlockChain.getInstance().getQoraHoldersShareAtHeight(this.blockData.getHeight());
 
 		// Perform account-level-based reward scaling if appropriate
 		if (!haveFounders) {
@@ -2029,6 +2191,26 @@ public class Block {
 		this.repository.getAccountRepository().tidy();
 	}
 
+	// Utils
+
+	/**
+	 * Find index of rewardSharePublicKey in list of rewardSharePublicKeys
+	 *
+	 * @param rewardSharePublicKey - the key to query
+	 * @param rewardSharePublicKeys - a sorted list of keys
+	 * @return - the index of the key, or null if not found
+	 */
+	private static Integer getRewardShareIndex(byte[] rewardSharePublicKey, List<byte[]> rewardSharePublicKeys) {
+		int index = 0;
+		for (byte[] publicKey : rewardSharePublicKeys) {
+			if (Arrays.equals(rewardSharePublicKey, publicKey)) {
+				return index;
+			}
+			index++;
+		}
+		return null;
+	}
+
 	private void logDebugInfo() {
 		try {
 			// Avoid calculations if possible. We have to check against INFO here, since Level.isMoreSpecificThan() confusingly uses <= rather than just <

src/main/java/org/qortal/block/BlockChain.java

@@ -68,9 +68,23 @@ public class BlockChain {
 		atFindNextTransactionFix,
 		newBlockSigHeight,
 		shareBinFix,
-		calcChainWeightTimestamp;
+		sharesByLevelV2Height,
+		rewardShareLimitTimestamp,
+		calcChainWeightTimestamp,
+		transactionV5Timestamp,
+		transactionV6Timestamp,
+		disableReferenceTimestamp;
 	}
 
+	// Custom transaction fees
+	/** Unit fees by transaction timestamp */
+	public static class UnitFeesByTimestamp {
+		public long timestamp;
+		@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
+		public long fee;
+	}
+	private List<UnitFeesByTimestamp> nameRegistrationUnitFees;
+
 	/** Map of which blockchain features are enabled when (height/timestamp) */
 	@XmlJavaTypeAdapter(StringLongMapXmlAdapter.class)
 	private Map<String, Long> featureTriggers;
@@ -90,23 +104,48 @@ public class BlockChain {
 	private List<RewardByHeight> rewardsByHeight;
 
 	/** Share of block reward/fees by account level */
-	public static class AccountLevelShareBin {
+	public static class AccountLevelShareBin implements Cloneable {
+		public int id;
 		public List<Integer> levels;
 		@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
 		public long share;
-	}
-	private List<AccountLevelShareBin> sharesByLevel;
-	/** Generated lookup of share-bin by account level */
-	private AccountLevelShareBin[] shareBinsByLevel;
 
-	/** Share of block reward/fees to legacy QORA coin holders */
-	@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
-	private Long qoraHoldersShare;
+		public Object clone() {
+			AccountLevelShareBin shareBinCopy = new AccountLevelShareBin();
+			List<Integer> levelsCopy = new ArrayList<>();
+			for (Integer level : this.levels) {
+				levelsCopy.add(level);
+			}
+			shareBinCopy.id = this.id;
+			shareBinCopy.levels = levelsCopy;
+			shareBinCopy.share = this.share;
+			return shareBinCopy;
+		}
+	}
+	private List<AccountLevelShareBin> sharesByLevelV1;
+	private List<AccountLevelShareBin> sharesByLevelV2;
+	/** Generated lookup of share-bin by account level */
+	private AccountLevelShareBin[] shareBinsByLevelV1;
+	private AccountLevelShareBin[] shareBinsByLevelV2;
+
+	/** Share of block reward/fees to legacy QORA coin holders, by block height */
+	public static class ShareByHeight {
+		public int height;
+		@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
+		public long share;
+	}
+	private List<ShareByHeight> qoraHoldersShareByHeight;
 
 	/** How many legacy QORA per 1 QORT of block reward. */
 	@XmlJavaTypeAdapter(value = org.qortal.api.AmountTypeAdapter.class)
 	private Long qoraPerQortReward;
 
+	/** Minimum number of accounts before a share bin is considered activated */
+	private int minAccountsToActivateShareBin;
+
+	/** Min level at which share bin activation takes place; lower levels allow less than minAccountsPerShareBin */
+	private int shareBinActivationMinLevel;
+
 	/**
 	 * Number of minted blocks required to reach next level from previous.
 	 * <p>
@@ -141,9 +180,10 @@ public class BlockChain {
 	}
 	private List<BlockTimingByHeight> blockTimingsByHeight;
 
-	private int minAccountLevelToMint = 1;
+	private int minAccountLevelToMint;
+	private int minAccountLevelForBlockSubmissions;
 	private int minAccountLevelToRewardShare;
-	private int maxRewardSharesPerMintingAccount;
+	private int maxRewardSharesPerFounderMintingAccount;
 	private int founderEffectiveMintingLevel;
 
 	/** Minimum time to retain online account signatures (ms) for block validity checks. */
@@ -151,6 +191,21 @@ public class BlockChain {
 	/** Maximum time to retain online account signatures (ms) for block validity checks, to allow for clock variance. */
 	private long onlineAccountSignaturesMaxLifetime;
 
+	/** Feature trigger timestamp for ONLINE_ACCOUNTS_MODULUS time interval increase. Can't use
+	 * featureTriggers because unit tests need to set this value via Reflection. */
+	private long onlineAccountsModulusV2Timestamp;
+
+	/** Feature trigger timestamp for online accounts mempow verification. Can't use featureTriggers
+	 * because unit tests need to set this value via Reflection. */
+	private long onlineAccountsMemoryPoWTimestamp;
+
+	/** Max reward shares by block height */
+	public static class MaxRewardSharesByTimestamp {
+		public long timestamp;
+		public int maxShares;
+	}
+	private List<MaxRewardSharesByTimestamp> maxRewardSharesByTimestamp;
+
 	/** Settings relating to CIYAM AT feature. */
 	public static class CiyamAtSettings {
 		/** Fee per step/op-code executed. */
@@ -299,6 +354,15 @@ public class BlockChain {
 		return this.maxBlockSize;
 	}
 
+	// Online accounts
+	public long getOnlineAccountsModulusV2Timestamp() {
+		return this.onlineAccountsModulusV2Timestamp;
+	}
+
+	public long getOnlineAccountsMemoryPoWTimestamp() {
+		return this.onlineAccountsMemoryPoWTimestamp;
+	}
+
 	/** Returns true if approval-needing transaction types require a txGroupId other than NO_GROUP. */
 	public boolean getRequireGroupForApproval() {
 		return this.requireGroupForApproval;
@@ -316,12 +380,20 @@ public class BlockChain {
 		return this.rewardsByHeight;
 	}
 
-	public List<AccountLevelShareBin> getAccountLevelShareBins() {
-		return this.sharesByLevel;
+	public List<AccountLevelShareBin> getAccountLevelShareBinsV1() {
+		return this.sharesByLevelV1;
 	}
 
-	public AccountLevelShareBin[] getShareBinsByAccountLevel() {
-		return this.shareBinsByLevel;
+	public List<AccountLevelShareBin> getAccountLevelShareBinsV2() {
+		return this.sharesByLevelV2;
+	}
+
+	public AccountLevelShareBin[] getShareBinsByAccountLevelV1() {
+		return this.shareBinsByLevelV1;
+	}
+
+	public AccountLevelShareBin[] getShareBinsByAccountLevelV2() {
+		return this.shareBinsByLevelV2;
 	}
 
 	public List<Integer> getBlocksNeededByLevel() {
@@ -332,24 +404,32 @@ public class BlockChain {
 		return this.cumulativeBlocksByLevel;
 	}
 
-	public long getQoraHoldersShare() {
-		return this.qoraHoldersShare;
-	}
-
 	public long getQoraPerQortReward() {
 		return this.qoraPerQortReward;
 	}
 
+	public int getMinAccountsToActivateShareBin() {
+		return this.minAccountsToActivateShareBin;
+	}
+
+	public int getShareBinActivationMinLevel() {
+		return this.shareBinActivationMinLevel;
+	}
+
 	public int getMinAccountLevelToMint() {
 		return this.minAccountLevelToMint;
 	}
 
+	public int getMinAccountLevelForBlockSubmissions() {
+		return this.minAccountLevelForBlockSubmissions;
+	}
+
 	public int getMinAccountLevelToRewardShare() {
 		return this.minAccountLevelToRewardShare;
 	}
 
-	public int getMaxRewardSharesPerMintingAccount() {
-		return this.maxRewardSharesPerMintingAccount;
+	public int getMaxRewardSharesPerFounderMintingAccount() {
+		return this.maxRewardSharesPerFounderMintingAccount;
 	}
 
 	public int getFounderEffectiveMintingLevel() {
@@ -382,10 +462,31 @@ public class BlockChain {
 		return this.featureTriggers.get(FeatureTrigger.shareBinFix.name()).intValue();
 	}
 
+	public int getSharesByLevelV2Height() {
+		return this.featureTriggers.get(FeatureTrigger.sharesByLevelV2Height.name()).intValue();
+	}
+
+	public long getRewardShareLimitTimestamp() {
+		return this.featureTriggers.get(FeatureTrigger.rewardShareLimitTimestamp.name()).longValue();
+	}
+
 	public long getCalcChainWeightTimestamp() {
 		return this.featureTriggers.get(FeatureTrigger.calcChainWeightTimestamp.name()).longValue();
 	}
 
+	public long getTransactionV5Timestamp() {
+		return this.featureTriggers.get(FeatureTrigger.transactionV5Timestamp.name()).longValue();
+	}
+
+	public long getTransactionV6Timestamp() {
+		return this.featureTriggers.get(FeatureTrigger.transactionV6Timestamp.name()).longValue();
+	}
+
+	public long getDisableReferenceTimestamp() {
+		return this.featureTriggers.get(FeatureTrigger.disableReferenceTimestamp.name()).longValue();
+	}
+
+
 	// More complex getters for aspects that change by height or timestamp
 
 	public long getRewardAtHeight(int ourHeight) {
@@ -405,6 +506,32 @@ public class BlockChain {
 		throw new IllegalStateException(String.format("No block timing info available for height %d", ourHeight));
 	}
 
+	public long getNameRegistrationUnitFeeAtTimestamp(long ourTimestamp) {
+		for (int i = nameRegistrationUnitFees.size() - 1; i >= 0; --i)
+			if (nameRegistrationUnitFees.get(i).timestamp <= ourTimestamp)
+				return nameRegistrationUnitFees.get(i).fee;
+
+		// Default to system-wide unit fee
+		return this.getUnitFee();
+	}
+
+	public int getMaxRewardSharesAtTimestamp(long ourTimestamp) {
+		for (int i = maxRewardSharesByTimestamp.size() - 1; i >= 0; --i)
+			if (maxRewardSharesByTimestamp.get(i).timestamp <= ourTimestamp)
+				return maxRewardSharesByTimestamp.get(i).maxShares;
+
+		return 0;
+	}
+
+	public long getQoraHoldersShareAtHeight(int ourHeight) {
+		// Scan through for QORA share at our height
+		for (int i = qoraHoldersShareByHeight.size() - 1; i >= 0; --i)
+			if (qoraHoldersShareByHeight.get(i).height <= ourHeight)
+				return qoraHoldersShareByHeight.get(i).share;
+
+		return 0;
+	}
+
 	/** Validate blockchain config read from JSON */
 	private void validateConfig() {
 		if (this.genesisInfo == null)
@@ -413,11 +540,14 @@ public class BlockChain {
 		if (this.rewardsByHeight == null)
 			Settings.throwValidationError("No \"rewardsByHeight\" entry found in blockchain config");
 
-		if (this.sharesByLevel == null)
-			Settings.throwValidationError("No \"sharesByLevel\" entry found in blockchain config");
+		if (this.sharesByLevelV1 == null)
+			Settings.throwValidationError("No \"sharesByLevelV1\" entry found in blockchain config");
 
-		if (this.qoraHoldersShare == null)
-			Settings.throwValidationError("No \"qoraHoldersShare\" entry found in blockchain config");
+		if (this.sharesByLevelV2 == null)
+			Settings.throwValidationError("No \"sharesByLevelV2\" entry found in blockchain config");
+
+		if (this.qoraHoldersShareByHeight == null)
+			Settings.throwValidationError("No \"qoraHoldersShareByHeight\" entry found in blockchain config");
 
 		if (this.qoraPerQortReward == null)
 			Settings.throwValidationError("No \"qoraPerQortReward\" entry found in blockchain config");
@@ -454,13 +584,22 @@ public class BlockChain {
 			if (!this.featureTriggers.containsKey(featureTrigger.name()))
 				Settings.throwValidationError(String.format("Missing feature trigger \"%s\" in blockchain config", featureTrigger.name()));
 
-		// Check block reward share bounds
-		long totalShare = this.qoraHoldersShare;
+		// Check block reward share bounds (V1)
+		long totalShareV1 = this.qoraHoldersShareByHeight.get(0).share;
 		// Add share percents for account-level-based rewards
-		for (AccountLevelShareBin accountLevelShareBin : this.sharesByLevel)
-			totalShare += accountLevelShareBin.share;
+		for (AccountLevelShareBin accountLevelShareBin : this.sharesByLevelV1)
+			totalShareV1 += accountLevelShareBin.share;
 
-		if (totalShare < 0 || totalShare > 1_00000000L)
+		if (totalShareV1 < 0 || totalShareV1 > 1_00000000L)
+			Settings.throwValidationError("Total non-founder share out of bounds (0<x<1e8)");
+
+		// Check block reward share bounds (V2)
+		long totalShareV2 = this.qoraHoldersShareByHeight.get(1).share;
+		// Add share percents for account-level-based rewards
+		for (AccountLevelShareBin accountLevelShareBin : this.sharesByLevelV2)
+			totalShareV2 += accountLevelShareBin.share;
+
+		if (totalShareV2 < 0 || totalShareV2 > 1_00000000L)
 			Settings.throwValidationError("Total non-founder share out of bounds (0<x<1e8)");
 	}
 
@@ -476,23 +615,34 @@ public class BlockChain {
 				cumulativeBlocks += this.blocksNeededByLevel.get(level);
 		}
 
-		// Generate lookup-array for account-level share bins
-		AccountLevelShareBin lastAccountLevelShareBin = this.sharesByLevel.get(this.sharesByLevel.size() - 1);
-		final int lastLevel = lastAccountLevelShareBin.levels.get(lastAccountLevelShareBin.levels.size() - 1);
-		this.shareBinsByLevel = new AccountLevelShareBin[lastLevel];
-
-		for (AccountLevelShareBin accountLevelShareBin : this.sharesByLevel)
+		// Generate lookup-array for account-level share bins (V1)
+		AccountLevelShareBin lastAccountLevelShareBinV1 = this.sharesByLevelV1.get(this.sharesByLevelV1.size() - 1);
+		final int lastLevelV1 = lastAccountLevelShareBinV1.levels.get(lastAccountLevelShareBinV1.levels.size() - 1);
+		this.shareBinsByLevelV1 = new AccountLevelShareBin[lastLevelV1];
+		for (AccountLevelShareBin accountLevelShareBin : this.sharesByLevelV1)
 			for (int level : accountLevelShareBin.levels)
 				// level 1 stored at index 0, level 2 stored at index 1, etc.
 				// level 0 not allowed
-				this.shareBinsByLevel[level - 1] = accountLevelShareBin;
+				this.shareBinsByLevelV1[level - 1] = accountLevelShareBin;
+
+		// Generate lookup-array for account-level share bins (V2)
+		AccountLevelShareBin lastAccountLevelShareBinV2 = this.sharesByLevelV2.get(this.sharesByLevelV2.size() - 1);
+		final int lastLevelV2 = lastAccountLevelShareBinV2.levels.get(lastAccountLevelShareBinV2.levels.size() - 1);
+		this.shareBinsByLevelV2 = new AccountLevelShareBin[lastLevelV2];
+		for (AccountLevelShareBin accountLevelShareBin : this.sharesByLevelV2)
+			for (int level : accountLevelShareBin.levels)
+				// level 1 stored at index 0, level 2 stored at index 1, etc.
+				// level 0 not allowed
+				this.shareBinsByLevelV2[level - 1] = accountLevelShareBin;
 
 		// Convert collections to unmodifiable form
 		this.rewardsByHeight = Collections.unmodifiableList(this.rewardsByHeight);
-		this.sharesByLevel = Collections.unmodifiableList(this.sharesByLevel);
+		this.sharesByLevelV1 = Collections.unmodifiableList(this.sharesByLevelV1);
+		this.sharesByLevelV2 = Collections.unmodifiableList(this.sharesByLevelV2);
 		this.blocksNeededByLevel = Collections.unmodifiableList(this.blocksNeededByLevel);
 		this.cumulativeBlocksByLevel = Collections.unmodifiableList(this.cumulativeBlocksByLevel);
 		this.blockTimingsByHeight = Collections.unmodifiableList(this.blockTimingsByHeight);
+		this.qoraHoldersShareByHeight = Collections.unmodifiableList(this.qoraHoldersShareByHeight);
 	}
 
 	/**

src/main/java/org/qortal/controller/AutoUpdate.java

@@ -15,6 +15,7 @@ import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.List;
 import java.util.concurrent.TimeoutException;
+import java.util.stream.Collectors;
 
 import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
@@ -40,6 +41,7 @@ public class AutoUpdate extends Thread {
 
 	public static final String JAR_FILENAME = "qortal.jar";
 	public static final String NEW_JAR_FILENAME = "new-" + JAR_FILENAME;
+	public static final String AGENTLIB_JVM_HOLDER_ARG = "-DQORTAL_agentlib=";
 
 	private static final Logger LOGGER = LogManager.getLogger(AutoUpdate.class);
 	private static final long CHECK_INTERVAL = 20 * 60 * 1000L; // ms
@@ -243,6 +245,11 @@ public class AutoUpdate extends Thread {
 			// JVM arguments
 			javaCmd.addAll(ManagementFactory.getRuntimeMXBean().getInputArguments());
 
+			// Disable, but retain, any -agentlib JVM arg as sub-process might fail if it tries to reuse same port
+			javaCmd = javaCmd.stream()
+					.map(arg -> arg.replace("-agentlib", AGENTLIB_JVM_HOLDER_ARG))
+					.collect(Collectors.toList());
+
 			// Remove JNI options as they won't be supported by command-line 'java'
 			// These are typically added by the AdvancedInstaller Java launcher EXE
 			javaCmd.removeAll(Arrays.asList("abort", "exit", "vfprintf"));
@@ -261,10 +268,19 @@ public class AutoUpdate extends Thread {
 					Translator.INSTANCE.translate("SysTray", "APPLYING_UPDATE_AND_RESTARTING"),
 					MessageType.INFO);
 
-			new ProcessBuilder(javaCmd).start();
+			ProcessBuilder processBuilder = new ProcessBuilder(javaCmd);
+
+			// New process will inherit our stdout and stderr
+			processBuilder.redirectOutput(ProcessBuilder.Redirect.INHERIT);
+			processBuilder.redirectError(ProcessBuilder.Redirect.INHERIT);
+
+			Process process = processBuilder.start();
+
+			// Nothing to pipe to new process, so close output stream (process's stdin)
+			process.getOutputStream().close();
 
 			return true; // applying update OK
-		} catch (IOException e) {
+		} catch (Exception e) {
 			LOGGER.error(String.format("Failed to apply update: %s", e.getMessage()));
 
 			try {

src/main/java/org/qortal/controller/BlockMinter.java

@@ -1,6 +1,8 @@
 package org.qortal.controller;
 
 import java.math.BigInteger;
+import java.text.DecimalFormat;
+import java.text.NumberFormat;
 import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.Iterator;
@@ -16,14 +18,17 @@ import org.qortal.account.PrivateKeyAccount;
 import org.qortal.block.Block;
 import org.qortal.block.Block.ValidationResult;
 import org.qortal.block.BlockChain;
-import org.qortal.data.account.AccountData;
 import org.qortal.data.account.MintingAccountData;
 import org.qortal.data.account.RewardShareData;
 import org.qortal.data.block.BlockData;
 import org.qortal.data.block.BlockSummaryData;
+import org.qortal.data.block.CommonBlockData;
 import org.qortal.data.transaction.TransactionData;
 import org.qortal.network.Network;
 import org.qortal.network.Peer;
+import org.qortal.network.message.BlockSummariesV2Message;
+import org.qortal.network.message.HeightV2Message;
+import org.qortal.network.message.Message;
 import org.qortal.repository.BlockRepository;
 import org.qortal.repository.DataException;
 import org.qortal.repository.Repository;
@@ -48,11 +53,6 @@ public class BlockMinter extends Thread {
 	// Recovery
 	public static final long INVALID_BLOCK_RECOVERY_TIMEOUT = 10 * 60 * 1000L; // ms
 
-	// Min account level to submit blocks
-	// This is an unvalidated version of Blockchain.minAccountLevelToMint
-	// and exists only to reduce block candidates by default.
-	private static int MIN_LEVEL_FOR_BLOCK_SUBMISSION = 3;
-
 	// Constructors
 
 	public BlockMinter() {
@@ -64,8 +64,12 @@ public class BlockMinter extends Thread {
 	public void run() {
 		Thread.currentThread().setName("BlockMinter");
 
-		try (final Repository repository = RepositoryManager.getRepository()) {
-			if (Settings.getInstance().getWipeUnconfirmedOnStart()) {
+		if (Settings.getInstance().isLite()) {
+			// Lite nodes do not mint
+			return;
+		}
+		if (Settings.getInstance().getWipeUnconfirmedOnStart()) {
+			try (final Repository repository = RepositoryManager.getRepository()) {
 				// Wipe existing unconfirmed transactions
 				List<TransactionData> unconfirmedTransactions = repository.getTransactionRepository().getUnconfirmedTransactions();
 
@@ -75,303 +79,373 @@ public class BlockMinter extends Thread {
 				}
 
 				repository.saveChanges();
+			} catch (DataException e) {
+				LOGGER.warn("Repository issue trying to wipe unconfirmed transactions on start-up: {}", e.getMessage());
+				// Fall-through to normal behaviour in case we can recover
 			}
+		}
 
+		BlockData previousBlockData = null;
+
+		// Vars to keep track of blocks that were skipped due to chain weight
+		byte[] parentSignatureForLastLowWeightBlock = null;
+		Long timeOfLastLowWeightBlock = null;
+
+		List<Block> newBlocks = new ArrayList<>();
+
+		try (final Repository repository = RepositoryManager.getRepository()) {
 			// Going to need this a lot...
 			BlockRepository blockRepository = repository.getBlockRepository();
-			BlockData previousBlockData = null;
-
-			List<Block> newBlocks = new ArrayList<>();
 
 			// Flags for tracking change in whether minting is possible,
 			// so we can notify Controller, and further update SysTray, etc.
 			boolean isMintingPossible = false;
 			boolean wasMintingPossible = isMintingPossible;
 			while (running) {
-				repository.discardChanges(); // Free repository locks, if any
-
 				if (isMintingPossible != wasMintingPossible)
 					Controller.getInstance().onMintingPossibleChange(isMintingPossible);
 
 				wasMintingPossible = isMintingPossible;
 
-				// Sleep for a while
-				Thread.sleep(1000);
+				try {
+					// Free up any repository locks
+					repository.discardChanges();
 
-				isMintingPossible = false;
+					// Sleep for a while
+					Thread.sleep(1000);
 
-				final Long now = NTP.getTime();
-				if (now == null)
-					continue;
+					isMintingPossible = false;
 
-				final Long minLatestBlockTimestamp = Controller.getMinimumLatestBlockTimestamp();
-				if (minLatestBlockTimestamp == null)
-					continue;
-
-				// No online accounts? (e.g. during startup)
-				if (Controller.getInstance().getOnlineAccounts().isEmpty())
-					continue;
-
-				List<MintingAccountData> mintingAccountsData = repository.getAccountRepository().getMintingAccounts();
-				// No minting accounts?
-				if (mintingAccountsData.isEmpty())
-					continue;
-
-				// Disregard minting accounts that are no longer valid, e.g. by transfer/loss of founder flag or account level
-				// Note that minting accounts are actually reward-shares in Qortal
-				Iterator<MintingAccountData> madi = mintingAccountsData.iterator();
-				while (madi.hasNext()) {
-					MintingAccountData mintingAccountData = madi.next();
-
-					RewardShareData rewardShareData = repository.getAccountRepository().getRewardShare(mintingAccountData.getPublicKey());
-					if (rewardShareData == null) {
-						// Reward-share doesn't exist - probably cancelled but not yet removed from node's list of minting accounts
-						madi.remove();
+					final Long now = NTP.getTime();
+					if (now == null)
 						continue;
-					}
 
-					Account mintingAccount = new Account(repository, rewardShareData.getMinter());
-					if (!mintingAccount.canMint()) {
-						// Minting-account component of reward-share can no longer mint - disregard
-						madi.remove();
+					final Long minLatestBlockTimestamp = Controller.getMinimumLatestBlockTimestamp();
+					if (minLatestBlockTimestamp == null)
 						continue;
-					}
 
-					// Optional (non-validated) prevention of block submissions below a defined level
-					AccountData accountData = repository.getAccountRepository().getAccount(mintingAccount.getAddress());
-					if (accountData != null) {
-						Integer level = accountData.getLevel();
-						if (level != null && level < MIN_LEVEL_FOR_BLOCK_SUBMISSION) {
+					// No online accounts for current timestamp? (e.g. during startup)
+					if (!OnlineAccountsManager.getInstance().hasOnlineAccounts())
+						continue;
+
+					List<MintingAccountData> mintingAccountsData = repository.getAccountRepository().getMintingAccounts();
+					// No minting accounts?
+					if (mintingAccountsData.isEmpty())
+						continue;
+
+					// Disregard minting accounts that are no longer valid, e.g. by transfer/loss of founder flag or account level
+					// Note that minting accounts are actually reward-shares in Qortal
+					Iterator<MintingAccountData> madi = mintingAccountsData.iterator();
+					while (madi.hasNext()) {
+						MintingAccountData mintingAccountData = madi.next();
+
+						RewardShareData rewardShareData = repository.getAccountRepository().getRewardShare(mintingAccountData.getPublicKey());
+						if (rewardShareData == null) {
+							// Reward-share doesn't exist - probably cancelled but not yet removed from node's list of minting accounts
+							madi.remove();
+							continue;
+						}
+
+						Account mintingAccount = new Account(repository, rewardShareData.getMinter());
+						if (!mintingAccount.canMint()) {
+							// Minting-account component of reward-share can no longer mint - disregard
+							madi.remove();
+							continue;
+						}
+
+						// Optional (non-validated) prevention of block submissions below a defined level.
+						// This is an unvalidated version of Blockchain.minAccountLevelToMint
+						// and exists only to reduce block candidates by default.
+						int level = mintingAccount.getEffectiveMintingLevel();
+						if (level < BlockChain.getInstance().getMinAccountLevelForBlockSubmissions()) {
 							madi.remove();
 							continue;
 						}
 					}
-				}
 
-				List<Peer> peers = Network.getInstance().getHandshakedPeers();
-				BlockData lastBlockData = blockRepository.getLastBlock();
+					// Needs a mutable copy of the unmodifiableList
+					List<Peer> peers = new ArrayList<>(Network.getInstance().getImmutableHandshakedPeers());
+					BlockData lastBlockData = blockRepository.getLastBlock();
 
-				// Disregard peers that have "misbehaved" recently
-				peers.removeIf(Controller.hasMisbehaved);
+					// Disregard peers that have "misbehaved" recently
+					peers.removeIf(Controller.hasMisbehaved);
 
-				// Disregard peers that don't have a recent block, but only if we're not in recovery mode.
-				// In that mode, we want to allow minting on top of older blocks, to recover stalled networks.
-				if (Controller.getInstance().getRecoveryMode() == false)
-					peers.removeIf(Controller.hasNoRecentBlock);
+					// Disregard peers that don't have a recent block, but only if we're not in recovery mode.
+					// In that mode, we want to allow minting on top of older blocks, to recover stalled networks.
+					if (Synchronizer.getInstance().getRecoveryMode() == false)
+						peers.removeIf(Controller.hasNoRecentBlock);
 
-				// Don't mint if we don't have enough up-to-date peers as where would the transactions/consensus come from?
-				if (peers.size() < Settings.getInstance().getMinBlockchainPeers())
-					continue;
-
-				// If we are stuck on an invalid block, we should allow an alternative to be minted
-				boolean recoverInvalidBlock = false;
-				if (Synchronizer.getInstance().timeInvalidBlockLastReceived != null) {
-					// We've had at least one invalid block
-					long timeSinceLastValidBlock = NTP.getTime() - Synchronizer.getInstance().timeValidBlockLastReceived;
-					long timeSinceLastInvalidBlock = NTP.getTime() - Synchronizer.getInstance().timeInvalidBlockLastReceived;
-					if (timeSinceLastValidBlock > INVALID_BLOCK_RECOVERY_TIMEOUT) {
-						if (timeSinceLastInvalidBlock < INVALID_BLOCK_RECOVERY_TIMEOUT) {
-							// Last valid block was more than 10 mins ago, but we've had an invalid block since then
-							// Assume that the chain has stalled because there is no alternative valid candidate
-							// Enter recovery mode to allow alternative, valid candidates to be minted
-							recoverInvalidBlock = true;
-						}
-					}
-				}
-
-				// If our latest block isn't recent then we need to synchronize instead of minting, unless we're in recovery mode.
-				if (!peers.isEmpty() && lastBlockData.getTimestamp() < minLatestBlockTimestamp)
-					if (Controller.getInstance().getRecoveryMode() == false && recoverInvalidBlock == false)
+					// Don't mint if we don't have enough up-to-date peers as where would the transactions/consensus come from?
+					if (peers.size() < Settings.getInstance().getMinBlockchainPeers())
 						continue;
 
-				// There are enough peers with a recent block and our latest block is recent
-				// so go ahead and mint a block if possible.
-				isMintingPossible = true;
-
-				// Check blockchain hasn't changed
-				if (previousBlockData == null || !Arrays.equals(previousBlockData.getSignature(), lastBlockData.getSignature())) {
-					previousBlockData = lastBlockData;
-					newBlocks.clear();
-
-					// Reduce log timeout
-					logTimeout = 10 * 1000L;
-				}
-
-				// Discard accounts we have already built blocks with
-				mintingAccountsData.removeIf(mintingAccountData -> newBlocks.stream().anyMatch(newBlock -> Arrays.equals(newBlock.getBlockData().getMinterPublicKey(), mintingAccountData.getPublicKey())));
-
-				// Do we need to build any potential new blocks?
-				List<PrivateKeyAccount> newBlocksMintingAccounts = mintingAccountsData.stream().map(accountData -> new PrivateKeyAccount(repository, accountData.getPrivateKey())).collect(Collectors.toList());
-
-				// We might need to sit the next block out, if one of our minting accounts signed the previous one
-				final byte[] previousBlockMinter = previousBlockData.getMinterPublicKey();
-				final boolean mintedLastBlock = mintingAccountsData.stream().anyMatch(mintingAccount -> Arrays.equals(mintingAccount.getPublicKey(), previousBlockMinter));
-				if (mintedLastBlock) {
-					LOGGER.trace(String.format("One of our keys signed the last block, so we won't sign the next one"));
-					continue;
-				}
-
-				for (PrivateKeyAccount mintingAccount : newBlocksMintingAccounts) {
-					// First block does the AT heavy-lifting
-					if (newBlocks.isEmpty()) {
-						Block newBlock = Block.mint(repository, previousBlockData, mintingAccount);
-						if (newBlock == null) {
-							// For some reason we can't mint right now
-							moderatedLog(() -> LOGGER.error("Couldn't build a to-be-minted block"));
-							continue;
-						}
-
-						newBlocks.add(newBlock);
-					} else {
-						// The blocks for other minters require less effort...
-						Block newBlock = newBlocks.get(0).remint(mintingAccount);
-						if (newBlock == null) {
-							// For some reason we can't mint right now
-							moderatedLog(() -> LOGGER.error("Couldn't rebuild a to-be-minted block"));
-							continue;
-						}
-
-						newBlocks.add(newBlock);
-					}
-				}
-
-				// No potential block candidates?
-				if (newBlocks.isEmpty())
-					continue;
-
-				// Make sure we're the only thread modifying the blockchain
-				ReentrantLock blockchainLock = Controller.getInstance().getBlockchainLock();
-				if (!blockchainLock.tryLock(30, TimeUnit.SECONDS)) {
-					LOGGER.debug("Couldn't acquire blockchain lock even after waiting 30 seconds");
-					continue;
-				}
-
-				boolean newBlockMinted = false;
-				Block newBlock = null;
-
-				try {
-					// Clear repository session state so we have latest view of data
-					repository.discardChanges();
-
-					// Now that we have blockchain lock, do final check that chain hasn't changed
-					BlockData latestBlockData = blockRepository.getLastBlock();
-					if (!Arrays.equals(lastBlockData.getSignature(), latestBlockData.getSignature()))
-						continue;
-
-					List<Block> goodBlocks = new ArrayList<>();
-					for (Block testBlock : newBlocks) {
-						// Is new block's timestamp valid yet?
-						// We do a separate check as some timestamp checks are skipped for testchains
-						if (testBlock.isTimestampValid() != ValidationResult.OK)
-							continue;
-
-						testBlock.preProcess();
-
-						// Is new block valid yet? (Before adding unconfirmed transactions)
-						ValidationResult result = testBlock.isValid();
-						if (result != ValidationResult.OK) {
-							moderatedLog(() -> LOGGER.error(String.format("To-be-minted block invalid '%s' before adding transactions?", result.name())));
-
-							continue;
-						}
-
-						goodBlocks.add(testBlock);
-					}
-
-					if (goodBlocks.isEmpty())
-						continue;
-
-					// Pick best block
-					final int parentHeight = previousBlockData.getHeight();
-					final byte[] parentBlockSignature = previousBlockData.getSignature();
-
-					BigInteger bestWeight = null;
-
-					for (int bi = 0; bi < goodBlocks.size(); ++bi) {
-						BlockData blockData = goodBlocks.get(bi).getBlockData();
-
-						BlockSummaryData blockSummaryData = new BlockSummaryData(blockData);
-						int minterLevel = Account.getRewardShareEffectiveMintingLevel(repository, blockData.getMinterPublicKey());
-						blockSummaryData.setMinterLevel(minterLevel);
-
-						BigInteger blockWeight = Block.calcBlockWeight(parentHeight, parentBlockSignature, blockSummaryData);
-
-						if (bestWeight == null || blockWeight.compareTo(bestWeight) < 0) {
-							newBlock = goodBlocks.get(bi);
-							bestWeight = blockWeight;
+					// If we are stuck on an invalid block, we should allow an alternative to be minted
+					boolean recoverInvalidBlock = false;
+					if (Synchronizer.getInstance().timeInvalidBlockLastReceived != null) {
+						// We've had at least one invalid block
+						long timeSinceLastValidBlock = NTP.getTime() - Synchronizer.getInstance().timeValidBlockLastReceived;
+						long timeSinceLastInvalidBlock = NTP.getTime() - Synchronizer.getInstance().timeInvalidBlockLastReceived;
+						if (timeSinceLastValidBlock > INVALID_BLOCK_RECOVERY_TIMEOUT) {
+							if (timeSinceLastInvalidBlock < INVALID_BLOCK_RECOVERY_TIMEOUT) {
+								// Last valid block was more than 10 mins ago, but we've had an invalid block since then
+								// Assume that the chain has stalled because there is no alternative valid candidate
+								// Enter recovery mode to allow alternative, valid candidates to be minted
+								recoverInvalidBlock = true;
+							}
 						}
 					}
 
-					// Add unconfirmed transactions
-					addUnconfirmedTransactions(repository, newBlock);
+					// If our latest block isn't recent then we need to synchronize instead of minting, unless we're in recovery mode.
+					if (!peers.isEmpty() && lastBlockData.getTimestamp() < minLatestBlockTimestamp)
+						if (Synchronizer.getInstance().getRecoveryMode() == false && recoverInvalidBlock == false)
+							continue;
 
-					// Sign to create block's signature
-					newBlock.sign();
+					// There are enough peers with a recent block and our latest block is recent
+					// so go ahead and mint a block if possible.
+					isMintingPossible = true;
 
-					// Is newBlock still valid?
-					ValidationResult validationResult = newBlock.isValid();
-					if (validationResult != ValidationResult.OK) {
-						// No longer valid? Report and discard
-						LOGGER.error(String.format("To-be-minted block now invalid '%s' after adding unconfirmed transactions?", validationResult.name()));
-
-						// Rebuild block candidates, just to be sure
+					// Check blockchain hasn't changed
+					if (previousBlockData == null || !Arrays.equals(previousBlockData.getSignature(), lastBlockData.getSignature())) {
+						previousBlockData = lastBlockData;
 						newBlocks.clear();
+
+						// Reduce log timeout
+						logTimeout = 10 * 1000L;
+
+						// Last low weight block is no longer valid
+						parentSignatureForLastLowWeightBlock = null;
+					}
+
+					// Discard accounts we have already built blocks with
+					mintingAccountsData.removeIf(mintingAccountData -> newBlocks.stream().anyMatch(newBlock -> Arrays.equals(newBlock.getBlockData().getMinterPublicKey(), mintingAccountData.getPublicKey())));
+
+					// Do we need to build any potential new blocks?
+					List<PrivateKeyAccount> newBlocksMintingAccounts = mintingAccountsData.stream().map(accountData -> new PrivateKeyAccount(repository, accountData.getPrivateKey())).collect(Collectors.toList());
+
+					// We might need to sit the next block out, if one of our minting accounts signed the previous one
+					byte[] previousBlockMinter = previousBlockData.getMinterPublicKey();
+					boolean mintedLastBlock = mintingAccountsData.stream().anyMatch(mintingAccount -> Arrays.equals(mintingAccount.getPublicKey(), previousBlockMinter));
+					if (mintedLastBlock) {
+						LOGGER.trace(String.format("One of our keys signed the last block, so we won't sign the next one"));
 						continue;
 					}
 
-					// Add to blockchain - something else will notice and broadcast new block to network
-					try {
-						newBlock.process();
+					if (parentSignatureForLastLowWeightBlock != null) {
+						// The last iteration found a higher weight block in the network, so sleep for a while
+						// to allow is to sync the higher weight chain. We are sleeping here rather than when
+						// detected as we don't want to hold the blockchain lock open.
+						LOGGER.info("Sleeping for 10 seconds...");
+						Thread.sleep(10 * 1000L);
+					}
 
-						repository.saveChanges();
+					for (PrivateKeyAccount mintingAccount : newBlocksMintingAccounts) {
+						// First block does the AT heavy-lifting
+						if (newBlocks.isEmpty()) {
+							Block newBlock = Block.mint(repository, previousBlockData, mintingAccount);
+							if (newBlock == null) {
+								// For some reason we can't mint right now
+								moderatedLog(() -> LOGGER.error("Couldn't build a to-be-minted block"));
+								continue;
+							}
 
-						LOGGER.info(String.format("Minted new block: %d", newBlock.getBlockData().getHeight()));
-
-						RewardShareData rewardShareData = repository.getAccountRepository().getRewardShare(newBlock.getBlockData().getMinterPublicKey());
-
-						if (rewardShareData != null) {
-							LOGGER.info(String.format("Minted block %d, sig %.8s, parent sig: %.8s by %s on behalf of %s",
-									newBlock.getBlockData().getHeight(),
-									Base58.encode(newBlock.getBlockData().getSignature()),
-									Base58.encode(newBlock.getParent().getSignature()),
-									rewardShareData.getMinter(),
-									rewardShareData.getRecipient()));
+							newBlocks.add(newBlock);
 						} else {
-							LOGGER.info(String.format("Minted block %d, sig %.8s, parent sig: %.8s by %s",
-									newBlock.getBlockData().getHeight(),
-									Base58.encode(newBlock.getBlockData().getSignature()),
-									Base58.encode(newBlock.getParent().getSignature()),
-									newBlock.getMinter().getAddress()));
+							// The blocks for other minters require less effort...
+							Block newBlock = newBlocks.get(0).remint(mintingAccount);
+							if (newBlock == null) {
+								// For some reason we can't mint right now
+								moderatedLog(() -> LOGGER.error("Couldn't rebuild a to-be-minted block"));
+								continue;
+							}
+
+							newBlocks.add(newBlock);
+						}
+					}
+
+					// No potential block candidates?
+					if (newBlocks.isEmpty())
+						continue;
+
+					// Make sure we're the only thread modifying the blockchain
+					ReentrantLock blockchainLock = Controller.getInstance().getBlockchainLock();
+					if (!blockchainLock.tryLock(30, TimeUnit.SECONDS)) {
+						LOGGER.debug("Couldn't acquire blockchain lock even after waiting 30 seconds");
+						continue;
+					}
+
+					boolean newBlockMinted = false;
+					Block newBlock = null;
+
+					try {
+						// Clear repository session state so we have latest view of data
+						repository.discardChanges();
+
+						// Now that we have blockchain lock, do final check that chain hasn't changed
+						BlockData latestBlockData = blockRepository.getLastBlock();
+						if (!Arrays.equals(lastBlockData.getSignature(), latestBlockData.getSignature()))
+							continue;
+
+						List<Block> goodBlocks = new ArrayList<>();
+						boolean wasInvalidBlockDiscarded = false;
+						Iterator<Block> newBlocksIterator = newBlocks.iterator();
+
+						while (newBlocksIterator.hasNext()) {
+							Block testBlock = newBlocksIterator.next();
+
+							// Is new block's timestamp valid yet?
+							// We do a separate check as some timestamp checks are skipped for testchains
+							if (testBlock.isTimestampValid() != ValidationResult.OK)
+								continue;
+
+							testBlock.preProcess();
+
+							// Is new block valid yet? (Before adding unconfirmed transactions)
+							ValidationResult result = testBlock.isValid();
+							if (result != ValidationResult.OK) {
+								moderatedLog(() -> LOGGER.error(String.format("To-be-minted block invalid '%s' before adding transactions?", result.name())));
+
+								newBlocksIterator.remove();
+								wasInvalidBlockDiscarded = true;
+								/*
+								 * Bail out fast so that we loop around from the top again.
+								 * This gives BlockMinter the possibility to remint this candidate block using another block from newBlocks,
+								 * via the Blocks.remint() method, which avoids having to re-process Block ATs all over again.
+								 * Particularly useful if some aspect of Blocks changes due a timestamp-based feature-trigger (see BlockChain class).
+								 */
+								break;
+							}
+
+							goodBlocks.add(testBlock);
 						}
 
-						// Notify network after we're released blockchain lock
-						newBlockMinted = true;
+						if (wasInvalidBlockDiscarded || goodBlocks.isEmpty())
+							continue;
 
-						// Notify Controller
-						repository.discardChanges(); // clear transaction status to prevent deadlocks
-						Controller.getInstance().onNewBlock(newBlock.getBlockData());
-					} catch (DataException e) {
-						// Unable to process block - report and discard
-						LOGGER.error("Unable to process newly minted block?", e);
-						newBlocks.clear();
+						// Pick best block
+						final int parentHeight = previousBlockData.getHeight();
+						final byte[] parentBlockSignature = previousBlockData.getSignature();
+
+						BigInteger bestWeight = null;
+
+						for (int bi = 0; bi < goodBlocks.size(); ++bi) {
+							BlockData blockData = goodBlocks.get(bi).getBlockData();
+
+							BlockSummaryData blockSummaryData = new BlockSummaryData(blockData);
+							int minterLevel = Account.getRewardShareEffectiveMintingLevel(repository, blockData.getMinterPublicKey());
+							blockSummaryData.setMinterLevel(minterLevel);
+
+							BigInteger blockWeight = Block.calcBlockWeight(parentHeight, parentBlockSignature, blockSummaryData);
+
+							if (bestWeight == null || blockWeight.compareTo(bestWeight) < 0) {
+								newBlock = goodBlocks.get(bi);
+								bestWeight = blockWeight;
+							}
+						}
+
+						try {
+							if (this.higherWeightChainExists(repository, bestWeight)) {
+
+								// Check if the base block has updated since the last time we were here
+								if (parentSignatureForLastLowWeightBlock == null || timeOfLastLowWeightBlock == null ||
+										!Arrays.equals(parentSignatureForLastLowWeightBlock, previousBlockData.getSignature())) {
+									// We've switched to a different chain, so reset the timer
+									timeOfLastLowWeightBlock = NTP.getTime();
+								}
+								parentSignatureForLastLowWeightBlock = previousBlockData.getSignature();
+
+								// If less than 30 seconds has passed since first detection the higher weight chain,
+								// we should skip our block submission to give us the opportunity to sync to the better chain
+								if (NTP.getTime() - timeOfLastLowWeightBlock < 30 * 1000L) {
+									LOGGER.info("Higher weight chain found in peers, so not signing a block this round");
+									LOGGER.info("Time since detected: {}", NTP.getTime() - timeOfLastLowWeightBlock);
+									continue;
+								} else {
+									// More than 30 seconds have passed, so we should submit our block candidate anyway.
+									LOGGER.info("More than 30 seconds passed, so proceeding to submit block candidate...");
+								}
+							} else {
+								LOGGER.debug("No higher weight chain found in peers");
+							}
+						} catch (DataException e) {
+							LOGGER.debug("Unable to check for a higher weight chain. Proceeding anyway...");
+						}
+
+						// Discard any uncommitted changes as a result of the higher weight chain detection
+						repository.discardChanges();
+
+						// Clear variables that track low weight blocks
+						parentSignatureForLastLowWeightBlock = null;
+						timeOfLastLowWeightBlock = null;
+
+						// Add unconfirmed transactions
+						addUnconfirmedTransactions(repository, newBlock);
+
+						// Sign to create block's signature
+						newBlock.sign();
+
+						// Is newBlock still valid?
+						ValidationResult validationResult = newBlock.isValid();
+						if (validationResult != ValidationResult.OK) {
+							// No longer valid? Report and discard
+							LOGGER.error(String.format("To-be-minted block now invalid '%s' after adding unconfirmed transactions?", validationResult.name()));
+
+							// Rebuild block candidates, just to be sure
+							newBlocks.clear();
+							continue;
+						}
+
+						// Add to blockchain - something else will notice and broadcast new block to network
+						try {
+							newBlock.process();
+
+							repository.saveChanges();
+
+							LOGGER.info(String.format("Minted new block: %d", newBlock.getBlockData().getHeight()));
+
+							RewardShareData rewardShareData = repository.getAccountRepository().getRewardShare(newBlock.getBlockData().getMinterPublicKey());
+
+							if (rewardShareData != null) {
+								LOGGER.info(String.format("Minted block %d, sig %.8s, parent sig: %.8s by %s on behalf of %s",
+										newBlock.getBlockData().getHeight(),
+										Base58.encode(newBlock.getBlockData().getSignature()),
+										Base58.encode(newBlock.getParent().getSignature()),
+										rewardShareData.getMinter(),
+										rewardShareData.getRecipient()));
+							} else {
+								LOGGER.info(String.format("Minted block %d, sig %.8s, parent sig: %.8s by %s",
+										newBlock.getBlockData().getHeight(),
+										Base58.encode(newBlock.getBlockData().getSignature()),
+										Base58.encode(newBlock.getParent().getSignature()),
+										newBlock.getMinter().getAddress()));
+							}
+
+							// Notify network after we're released blockchain lock
+							newBlockMinted = true;
+
+							// Notify Controller
+							repository.discardChanges(); // clear transaction status to prevent deadlocks
+							Controller.getInstance().onNewBlock(newBlock.getBlockData());
+						} catch (DataException e) {
+							// Unable to process block - report and discard
+							LOGGER.error("Unable to process newly minted block?", e);
+							newBlocks.clear();
+						}
+					} finally {
+						blockchainLock.unlock();
 					}
-				} finally {
-					blockchainLock.unlock();
-				}
 
-				if (newBlockMinted) {
-					// Broadcast our new chain to network
-					BlockData newBlockData = newBlock.getBlockData();
+					if (newBlockMinted) {
+						// Broadcast our new chain to network
+						Network.getInstance().broadcastOurChain();
+					}
 
-					Network network = Network.getInstance();
-					network.broadcast(broadcastPeer -> network.buildHeightMessage(broadcastPeer, newBlockData));
+				} catch (InterruptedException e) {
+					// We've been interrupted - time to exit
+					return;
 				}
 			}
 		} catch (DataException e) {
-			LOGGER.warn("Repository issue while running block minter", e);
-		} catch (InterruptedException e) {
-			// We've been interrupted - time to exit
-			return;
+			LOGGER.warn("Repository issue while running block minter - NO LONGER MINTING", e);
 		}
 	}
 
@@ -429,7 +503,7 @@ public class BlockMinter extends Thread {
 			throw new DataException("Ignoring attempt to mint testing block for non-test chain!");
 
 		// Ensure mintingAccount is 'online' so blocks can be minted
-		Controller.getInstance().ensureTestingAccountsOnline(mintingAndOnlineAccounts);
+		OnlineAccountsManager.getInstance().ensureTestingAccountsOnline(mintingAndOnlineAccounts);
 
 		PrivateKeyAccount mintingAccount = mintingAndOnlineAccounts[0];
 
@@ -469,6 +543,66 @@ public class BlockMinter extends Thread {
 		}
 	}
 
+	private BigInteger getOurChainWeightSinceBlock(Repository repository, BlockSummaryData commonBlock, List<BlockSummaryData> peerBlockSummaries) throws DataException {
+		final int commonBlockHeight = commonBlock.getHeight();
+		final byte[] commonBlockSig = commonBlock.getSignature();
+		int mutualHeight = commonBlockHeight;
+
+		// Fetch our corresponding block summaries
+		final BlockData ourLatestBlockData = repository.getBlockRepository().getLastBlock();
+		List<BlockSummaryData> ourBlockSummaries = repository.getBlockRepository()
+				.getBlockSummaries(commonBlockHeight + 1, ourLatestBlockData.getHeight());
+		if (!ourBlockSummaries.isEmpty()) {
+			Synchronizer.getInstance().populateBlockSummariesMinterLevels(repository, ourBlockSummaries);
+		}
+
+		if (ourBlockSummaries != null && peerBlockSummaries != null) {
+			mutualHeight += Math.min(ourBlockSummaries.size(), peerBlockSummaries.size());
+		}
+		return Block.calcChainWeight(commonBlockHeight, commonBlockSig, ourBlockSummaries, mutualHeight);
+	}
+
+	private boolean higherWeightChainExists(Repository repository, BigInteger blockCandidateWeight) throws DataException {
+		if (blockCandidateWeight == null) {
+			// Can't make decisions without knowing the block candidate weight
+			return false;
+		}
+		NumberFormat formatter = new DecimalFormat("0.###E0");
+
+		List<Peer> peers = Network.getInstance().getImmutableHandshakedPeers();
+		// Loop through handshaked peers and check for any new block candidates
+		for (Peer peer : peers) {
+			if (peer.getCommonBlockData() != null && peer.getCommonBlockData().getCommonBlockSummary() != null) {
+				// This peer has common block data
+				CommonBlockData commonBlockData = peer.getCommonBlockData();
+				BlockSummaryData commonBlockSummaryData = commonBlockData.getCommonBlockSummary();
+				if (commonBlockData.getChainWeight() != null && peer.getCommonBlockData().getBlockSummariesAfterCommonBlock() != null) {
+					// The synchronizer has calculated this peer's chain weight
+					if (!Synchronizer.getInstance().containsInvalidBlockSummary(peer.getCommonBlockData().getBlockSummariesAfterCommonBlock())) {
+						// .. and it doesn't hold any invalid blocks
+						BigInteger ourChainWeightSinceCommonBlock = this.getOurChainWeightSinceBlock(repository, commonBlockSummaryData, commonBlockData.getBlockSummariesAfterCommonBlock());
+						BigInteger ourChainWeight = ourChainWeightSinceCommonBlock.add(blockCandidateWeight);
+						BigInteger peerChainWeight = commonBlockData.getChainWeight();
+						if (peerChainWeight.compareTo(ourChainWeight) >= 0) {
+							// This peer has a higher weight chain than ours
+							LOGGER.info("Peer {} is on a higher weight chain ({}) than ours ({})", peer, formatter.format(peerChainWeight), formatter.format(ourChainWeight));
+							return true;
+
+						} else {
+							LOGGER.debug("Peer {} is on a lower weight chain ({}) than ours ({})", peer, formatter.format(peerChainWeight), formatter.format(ourChainWeight));
+						}
+					} else {
+						LOGGER.debug("Peer {} has an invalid block", peer);
+					}
+				} else {
+					LOGGER.debug("Peer {} has no chain weight", peer);
+				}
+			} else {
+				LOGGER.debug("Peer {} has no common block data", peer);
+			}
+		}
+		return false;
+	}
 	private static void moderatedLog(Runnable logFunction) {
 		// We only log if logging at TRACE or previous log timeout has expired
 		if (!LOGGER.isTraceEnabled() && lastLogTimestamp != null && lastLogTimestamp + logTimeout > System.currentTimeMillis())

src/main/java/org/qortal/controller/LiteNode.java

@@ -0,0 +1,189 @@
+package org.qortal.controller;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.qortal.data.account.AccountBalanceData;
+import org.qortal.data.account.AccountData;
+import org.qortal.data.naming.NameData;
+import org.qortal.data.transaction.TransactionData;
+import org.qortal.network.Network;
+import org.qortal.network.Peer;
+import org.qortal.network.message.*;
+
+import java.security.SecureRandom;
+import java.util.*;
+
+import static org.qortal.network.message.MessageType.*;
+
+public class LiteNode {
+
+    private static final Logger LOGGER = LogManager.getLogger(LiteNode.class);
+
+    private static LiteNode instance;
+
+
+    public Map<Integer, Long> pendingRequests = Collections.synchronizedMap(new HashMap<>());
+
+    public int MAX_TRANSACTIONS_PER_MESSAGE = 100;
+
+
+    public LiteNode() {
+
+    }
+
+    public static synchronized LiteNode getInstance() {
+        if (instance == null) {
+            instance = new LiteNode();
+        }
+
+        return instance;
+    }
+
+
+    /**
+     * Fetch account data from peers for given QORT address
+     * @param address - the QORT address to query
+     * @return accountData - the account data for this address, or null if not retrieved
+     */
+    public AccountData fetchAccountData(String address) {
+        GetAccountMessage getAccountMessage = new GetAccountMessage(address);
+        AccountMessage accountMessage = (AccountMessage) this.sendMessage(getAccountMessage, ACCOUNT);
+        if (accountMessage == null) {
+            return null;
+        }
+        return accountMessage.getAccountData();
+    }
+
+    /**
+     * Fetch account balance data from peers for given QORT address and asset ID
+     * @param address - the QORT address to query
+     * @return balance - the balance for this address and assetId, or null if not retrieved
+     */
+    public AccountBalanceData fetchAccountBalance(String address, long assetId) {
+        GetAccountBalanceMessage getAccountMessage = new GetAccountBalanceMessage(address, assetId);
+        AccountBalanceMessage accountMessage = (AccountBalanceMessage) this.sendMessage(getAccountMessage, ACCOUNT_BALANCE);
+        if (accountMessage == null) {
+            return null;
+        }
+        return accountMessage.getAccountBalanceData();
+    }
+
+    /**
+     * Fetch list of transactions for given QORT address
+     * @param address - the QORT address to query
+     * @param limit - the maximum number of results to return
+     * @param offset - the starting index
+     * @return a list of TransactionData objects, or null if not retrieved
+     */
+    public List<TransactionData> fetchAccountTransactions(String address, int limit, int offset) {
+        List<TransactionData> allTransactions = new ArrayList<>();
+        if (limit == 0) {
+            limit = Integer.MAX_VALUE;
+        }
+        int batchSize = Math.min(limit, MAX_TRANSACTIONS_PER_MESSAGE);
+
+        while (allTransactions.size() < limit) {
+            GetAccountTransactionsMessage getAccountTransactionsMessage = new GetAccountTransactionsMessage(address, batchSize, offset);
+            TransactionsMessage transactionsMessage = (TransactionsMessage) this.sendMessage(getAccountTransactionsMessage, TRANSACTIONS);
+            if (transactionsMessage == null) {
+                // An error occurred, so give up instead of returning partial results
+                return null;
+            }
+            allTransactions.addAll(transactionsMessage.getTransactions());
+            if (transactionsMessage.getTransactions().size() < batchSize) {
+                // No more transactions to fetch
+                break;
+            }
+            offset += batchSize;
+        }
+        return allTransactions;
+    }
+
+    /**
+     * Fetch list of names for given QORT address
+     * @param address - the QORT address to query
+     * @return a list of NameData objects, or null if not retrieved
+     */
+    public List<NameData> fetchAccountNames(String address) {
+        GetAccountNamesMessage getAccountNamesMessage = new GetAccountNamesMessage(address);
+        NamesMessage namesMessage = (NamesMessage) this.sendMessage(getAccountNamesMessage, NAMES);
+        if (namesMessage == null) {
+            return null;
+        }
+        return namesMessage.getNameDataList();
+    }
+
+    /**
+     * Fetch info about a registered name
+     * @param name - the name to query
+     * @return a NameData object, or null if not retrieved
+     */
+    public NameData fetchNameData(String name) {
+        GetNameMessage getNameMessage = new GetNameMessage(name);
+        NamesMessage namesMessage = (NamesMessage) this.sendMessage(getNameMessage, NAMES);
+        if (namesMessage == null) {
+            return null;
+        }
+        List<NameData> nameDataList = namesMessage.getNameDataList();
+        if (nameDataList == null || nameDataList.size() != 1) {
+            return null;
+        }
+        // We are only expecting a single item in the list
+        return nameDataList.get(0);
+    }
+
+
+    private Message sendMessage(Message message, MessageType expectedResponseMessageType) {
+        // This asks a random peer for the data
+        // TODO: ask multiple peers, and disregard everything if there are any significant differences in the responses
+
+        // Needs a mutable copy of the unmodifiableList
+        List<Peer> peers = new ArrayList<>(Network.getInstance().getImmutableHandshakedPeers());
+
+        // Disregard peers that have "misbehaved" recently
+        peers.removeIf(Controller.hasMisbehaved);
+
+        // Disregard peers that only have genesis block
+        // TODO: peers.removeIf(Controller.hasOnlyGenesisBlock);
+
+        // Disregard peers that are on an old version
+        peers.removeIf(Controller.hasOldVersion);
+
+        // Disregard peers that are on a known inferior chain tip
+        // TODO: peers.removeIf(Controller.hasInferiorChainTip);
+
+        if (peers.isEmpty()) {
+            LOGGER.info("No peers available to send {} message to", message.getType());
+            return null;
+        }
+
+        // Pick random peer
+        int index = new SecureRandom().nextInt(peers.size());
+        Peer peer = peers.get(index);
+
+        LOGGER.info("Sending {} message to peer {}...", message.getType(), peer);
+
+        Message responseMessage;
+
+        try {
+            responseMessage = peer.getResponse(message);
+
+        } catch (InterruptedException e) {
+            return null;
+        }
+
+        if (responseMessage == null) {
+            LOGGER.info("Peer {} didn't respond to {} message", peer, message.getType());
+            return null;
+        }
+        else if (responseMessage.getType() != expectedResponseMessageType) {
+            LOGGER.info("Peer responded with unexpected message type {} (should be {})", peer, responseMessage.getType(), expectedResponseMessageType);
+            return null;
+        }
+
+        LOGGER.info("Peer {} responded with {} message", peer, responseMessage.getType());
+
+        return responseMessage;
+    }
+
+}

src/main/java/org/qortal/controller/OnlineAccountsManager.java

@@ -0,0 +1,871 @@
+package org.qortal.controller;
+
+import com.google.common.hash.HashCode;
+import com.google.common.primitives.Longs;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.qortal.account.Account;
+import org.qortal.account.PrivateKeyAccount;
+import org.qortal.block.Block;
+import org.qortal.block.BlockChain;
+import org.qortal.crypto.Crypto;
+import org.qortal.crypto.MemoryPoW;
+import org.qortal.crypto.Qortal25519Extras;
+import org.qortal.data.account.MintingAccountData;
+import org.qortal.data.account.RewardShareData;
+import org.qortal.data.network.OnlineAccountData;
+import org.qortal.network.Network;
+import org.qortal.network.Peer;
+import org.qortal.network.message.*;
+import org.qortal.repository.DataException;
+import org.qortal.repository.Repository;
+import org.qortal.repository.RepositoryManager;
+import org.qortal.settings.Settings;
+import org.qortal.utils.Base58;
+import org.qortal.utils.NTP;
+import org.qortal.utils.NamedThreadFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.util.*;
+import java.util.concurrent.*;
+import java.util.stream.Collectors;
+
+public class OnlineAccountsManager {
+
+    private static final Logger LOGGER = LogManager.getLogger(OnlineAccountsManager.class);
+
+    // 'Current' as in 'now'
+
+    /**
+     * How long online accounts signatures last before they expire.
+     */
+    private static final long ONLINE_TIMESTAMP_MODULUS_V1 = 5 * 60 * 1000L;
+    private static final long ONLINE_TIMESTAMP_MODULUS_V2 = 30 * 60 * 1000L;
+
+    /**
+     * How many 'current' timestamp-sets of online accounts we cache.
+     */
+    private static final int MAX_CACHED_TIMESTAMP_SETS = 2;
+
+    /**
+     * How many timestamp-sets of online accounts we cache for 'latest blocks'.
+     */
+    private static final int MAX_BLOCKS_CACHED_ONLINE_ACCOUNTS = 3;
+
+    private static final long ONLINE_ACCOUNTS_QUEUE_INTERVAL = 100L; //ms
+    private static final long ONLINE_ACCOUNTS_TASKS_INTERVAL = 10 * 1000L; // ms
+    private static final long ONLINE_ACCOUNTS_COMPUTE_INTERVAL = 5 * 1000L; // ms
+    private static final long ONLINE_ACCOUNTS_BROADCAST_INTERVAL = 60 * 1000L; // ms
+    // After switching to a new online timestamp, we "burst" the online accounts requests
+    // at an increased interval for a specified amount of time
+    private static final long ONLINE_ACCOUNTS_BROADCAST_BURST_INTERVAL = 5 * 1000L; // ms
+    private static final long ONLINE_ACCOUNTS_BROADCAST_BURST_LENGTH = 5 * 60 * 1000L; // ms
+
+    private static final long INITIAL_SLEEP_INTERVAL = 30 * 1000L;
+
+    // MemoryPoW
+    public final int POW_BUFFER_SIZE = 1 * 1024 * 1024; // bytes
+    public int POW_DIFFICULTY = 18; // leading zero bits
+
+    private final ScheduledExecutorService executor = Executors.newScheduledThreadPool(4, new NamedThreadFactory("OnlineAccounts"));
+    private volatile boolean isStopping = false;
+
+    private final List<OnlineAccountData> onlineAccountsImportQueue = Collections.synchronizedList(new ArrayList<>());
+
+    /**
+     * Cache of 'current' online accounts, keyed by timestamp
+     */
+    private final Map<Long, Set<OnlineAccountData>> currentOnlineAccounts = new ConcurrentHashMap<>();
+    /**
+     * Cache of hash-summary of 'current' online accounts, keyed by timestamp, then leading byte of public key.
+     */
+    private final Map<Long, Map<Byte, byte[]>> currentOnlineAccountsHashes = new ConcurrentHashMap<>();
+
+    /**
+     * Cache of online accounts for latest blocks - not necessarily 'current' / now.
+     * <i>Probably</i> only accessed / modified by a single Synchronizer thread.
+     */
+    private final SortedMap<Long, Set<OnlineAccountData>> latestBlocksOnlineAccounts = new ConcurrentSkipListMap<>();
+
+    private long lastOnlineAccountsRequest = 0;
+
+    private boolean hasOurOnlineAccounts = false;
+
+    public static long getOnlineTimestampModulus() {
+        Long now = NTP.getTime();
+        if (now != null && now >= BlockChain.getInstance().getOnlineAccountsModulusV2Timestamp()) {
+            return ONLINE_TIMESTAMP_MODULUS_V2;
+        }
+        return ONLINE_TIMESTAMP_MODULUS_V1;
+    }
+    public static Long getCurrentOnlineAccountTimestamp() {
+        Long now = NTP.getTime();
+        if (now == null)
+            return null;
+
+        long onlineTimestampModulus = getOnlineTimestampModulus();
+        return (now / onlineTimestampModulus) * onlineTimestampModulus;
+    }
+
+    public static long toOnlineAccountTimestamp(long timestamp) {
+        return (timestamp / getOnlineTimestampModulus()) * getOnlineTimestampModulus();
+    }
+
+    private OnlineAccountsManager() {
+    }
+
+    private static class SingletonContainer {
+        private static final OnlineAccountsManager INSTANCE = new OnlineAccountsManager();
+    }
+
+    public static OnlineAccountsManager getInstance() {
+        return SingletonContainer.INSTANCE;
+    }
+
+    public void start() {
+        // Expire old online accounts signatures
+        executor.scheduleAtFixedRate(this::expireOldOnlineAccounts, ONLINE_ACCOUNTS_TASKS_INTERVAL, ONLINE_ACCOUNTS_TASKS_INTERVAL, TimeUnit.MILLISECONDS);
+
+        // Request online accounts from peers
+        executor.scheduleAtFixedRate(this::requestRemoteOnlineAccounts, ONLINE_ACCOUNTS_BROADCAST_BURST_INTERVAL, ONLINE_ACCOUNTS_BROADCAST_BURST_INTERVAL, TimeUnit.MILLISECONDS);
+
+        // Process import queue
+        executor.scheduleWithFixedDelay(this::processOnlineAccountsImportQueue, ONLINE_ACCOUNTS_QUEUE_INTERVAL, ONLINE_ACCOUNTS_QUEUE_INTERVAL, TimeUnit.MILLISECONDS);
+
+        // Sleep for some time before scheduling sendOurOnlineAccountsInfo()
+        // This allows some time for initial online account lists to be retrieved, and
+        // reduces the chances of the same nonce being computed twice
+        try {
+            Thread.sleep(INITIAL_SLEEP_INTERVAL);
+        } catch (InterruptedException e) {
+            throw new RuntimeException(e);
+        }
+
+        // Send our online accounts
+        executor.scheduleAtFixedRate(this::sendOurOnlineAccountsInfo, ONLINE_ACCOUNTS_COMPUTE_INTERVAL, ONLINE_ACCOUNTS_COMPUTE_INTERVAL, TimeUnit.MILLISECONDS);
+    }
+
+    public void shutdown() {
+        isStopping = true;
+        executor.shutdownNow();
+    }
+
+    // Testing support
+    public void ensureTestingAccountsOnline(PrivateKeyAccount... onlineAccounts) {
+        if (!BlockChain.getInstance().isTestChain()) {
+            LOGGER.warn("Ignoring attempt to ensure test account is online for non-test chain!");
+            return;
+        }
+
+        final Long onlineAccountsTimestamp = getCurrentOnlineAccountTimestamp();
+        if (onlineAccountsTimestamp == null)
+            return;
+
+        byte[] timestampBytes = Longs.toByteArray(onlineAccountsTimestamp);
+        final boolean mempowActive = onlineAccountsTimestamp >= BlockChain.getInstance().getOnlineAccountsMemoryPoWTimestamp();
+
+        Set<OnlineAccountData> replacementAccounts = new HashSet<>();
+        for (PrivateKeyAccount onlineAccount : onlineAccounts) {
+            // Check mintingAccount is actually reward-share?
+
+            byte[] signature = Qortal25519Extras.signForAggregation(onlineAccount.getPrivateKey(), timestampBytes);
+            byte[] publicKey = onlineAccount.getPublicKey();
+
+            Integer nonce = mempowActive ? new Random().nextInt(500000) : null;
+
+            OnlineAccountData ourOnlineAccountData = new OnlineAccountData(onlineAccountsTimestamp, signature, publicKey, nonce);
+            replacementAccounts.add(ourOnlineAccountData);
+        }
+
+        this.currentOnlineAccounts.clear();
+        addAccounts(replacementAccounts);
+    }
+
+    // Online accounts import queue
+
+    private void processOnlineAccountsImportQueue() {
+        if (this.onlineAccountsImportQueue.isEmpty())
+            // Nothing to do
+            return;
+
+        LOGGER.debug("Processing online accounts import queue (size: {})", this.onlineAccountsImportQueue.size());
+
+        // Take a copy of onlineAccountsImportQueue so we can safely remove whilst iterating
+        List<OnlineAccountData> onlineAccountsImportQueueCopy = new ArrayList<>(this.onlineAccountsImportQueue);
+
+        Set<OnlineAccountData> onlineAccountsToAdd = new HashSet<>();
+        try (final Repository repository = RepositoryManager.getRepository()) {
+            for (OnlineAccountData onlineAccountData : onlineAccountsImportQueueCopy) {
+                if (isStopping)
+                    return;
+
+                // Skip this account if it's already validated
+                Set<OnlineAccountData> onlineAccounts = this.currentOnlineAccounts.computeIfAbsent(onlineAccountData.getTimestamp(), k -> ConcurrentHashMap.newKeySet());
+                if (onlineAccounts.contains(onlineAccountData)) {
+                    // We have already validated this online account
+                    onlineAccountsImportQueue.remove(onlineAccountData);
+                    continue;
+                }
+
+                boolean isValid = this.isValidCurrentAccount(repository, onlineAccountData);
+                if (isValid)
+                    addAccounts(Arrays.asList(onlineAccountData));
+
+                // Remove from queue
+                onlineAccountsImportQueue.remove(onlineAccountData);
+            }
+        } catch (DataException e) {
+            LOGGER.error("Repository issue while verifying online accounts", e);
+        }
+    }
+
+    private boolean importQueueContainsExactMatch(OnlineAccountData acc) {
+        // Check if an item exists where all properties match exactly
+        // This is needed because signature and nonce are not compared in OnlineAccountData.equals()
+        synchronized (onlineAccountsImportQueue) {
+            return onlineAccountsImportQueue.stream().anyMatch(otherAcc ->
+                    acc.getTimestamp() == otherAcc.getTimestamp() &&
+                            Arrays.equals(acc.getPublicKey(), otherAcc.getPublicKey()) &&
+                            acc.getNonce() == otherAcc.getNonce() &&
+                            Arrays.equals(acc.getSignature(), otherAcc.getSignature())
+            );
+        }
+    }
+
+    /**
+     * Check if supplied onlineAccountData is superior (i.e. has a nonce value) than existing record.
+     * Two entries are considered equal even if the nonce differs, to prevent multiple variations
+     * co-existing. For this reason, we need to be able to check if a new OnlineAccountData entry should
+     * replace the existing one, which may be missing the nonce.
+     * @param onlineAccountData
+     * @return true if supplied data is superior to existing entry
+     */
+    private boolean isOnlineAccountsDataSuperior(OnlineAccountData onlineAccountData) {
+        if (onlineAccountData.getNonce() == null || onlineAccountData.getNonce() < 0) {
+            // New online account data has no usable nonce value, so it won't be better than anything we already have
+            return false;
+        }
+
+        // New online account data has a nonce value, so check if there is any existing data to compare against
+        Set<OnlineAccountData> existingOnlineAccountsForTimestamp = this.currentOnlineAccounts.get(onlineAccountData.getTimestamp());
+        if (existingOnlineAccountsForTimestamp == null) {
+            // No existing online accounts data with this timestamp yet
+            return false;
+        }
+
+        // Check if a duplicate entry exists
+        OnlineAccountData existingOnlineAccountData = null;
+        for (OnlineAccountData existingAccount : existingOnlineAccountsForTimestamp) {
+            if (existingAccount.equals(onlineAccountData)) {
+                // Found existing online account data
+                existingOnlineAccountData = existingAccount;
+                break;
+            }
+        }
+
+        if (existingOnlineAccountData == null) {
+            // No existing online accounts data, so nothing to compare
+            return false;
+        }
+
+        if (existingOnlineAccountData.getNonce() == null || existingOnlineAccountData.getNonce() < 0) {
+            // Existing data has no usable nonce value(s) so we want to replace it with the new one
+            return true;
+        }
+
+        // Both new and old data have nonce values so the new data isn't considered superior
+        return false;
+    }
+
+
+    // Utilities
+
+    public static byte[] xorByteArrayInPlace(byte[] inplaceArray, byte[] otherArray) {
+        if (inplaceArray == null)
+            return Arrays.copyOf(otherArray, otherArray.length);
+
+        // Start from index 1 to enforce static leading byte
+        for (int i = 1; i < otherArray.length; i++)
+            inplaceArray[i] ^= otherArray[i];
+
+        return inplaceArray;
+    }
+
+    private static boolean isValidCurrentAccount(Repository repository, OnlineAccountData onlineAccountData) throws DataException {
+        final Long now = NTP.getTime();
+        if (now == null)
+            return false;
+
+        byte[] rewardSharePublicKey = onlineAccountData.getPublicKey();
+        long onlineAccountTimestamp = onlineAccountData.getTimestamp();
+
+        // Check timestamp is 'recent' here
+        if (Math.abs(onlineAccountTimestamp - now) > getOnlineTimestampModulus() * 2) {
+            LOGGER.trace(() -> String.format("Rejecting online account %s with out of range timestamp %d", Base58.encode(rewardSharePublicKey), onlineAccountTimestamp));
+            return false;
+        }
+
+        // Check timestamp is a multiple of online timestamp modulus
+        if (onlineAccountTimestamp % getOnlineTimestampModulus() != 0) {
+            LOGGER.trace(() -> String.format("Rejecting online account %s with invalid timestamp %d", Base58.encode(rewardSharePublicKey), onlineAccountTimestamp));
+            return false;
+        }
+
+        // Verify signature
+        byte[] data = Longs.toByteArray(onlineAccountData.getTimestamp());
+        boolean isSignatureValid = Qortal25519Extras.verifyAggregated(rewardSharePublicKey, onlineAccountData.getSignature(), data);
+        if (!isSignatureValid) {
+            LOGGER.trace(() -> String.format("Rejecting invalid online account %s", Base58.encode(rewardSharePublicKey)));
+            return false;
+        }
+
+        // Qortal: check online account is actually reward-share
+        RewardShareData rewardShareData = repository.getAccountRepository().getRewardShare(rewardSharePublicKey);
+        if (rewardShareData == null) {
+            // Reward-share doesn't even exist - probably not a good sign
+            LOGGER.trace(() -> String.format("Rejecting unknown online reward-share public key %s", Base58.encode(rewardSharePublicKey)));
+            return false;
+        }
+
+        Account mintingAccount = new Account(repository, rewardShareData.getMinter());
+        if (!mintingAccount.canMint()) {
+            // Minting-account component of reward-share can no longer mint - disregard
+            LOGGER.trace(() -> String.format("Rejecting online reward-share with non-minting account %s", mintingAccount.getAddress()));
+            return false;
+        }
+
+        // Validate mempow if feature trigger is active (or if online account's timestamp is past the trigger timestamp)
+        long memoryPoWStartTimestamp = BlockChain.getInstance().getOnlineAccountsMemoryPoWTimestamp();
+        if (now >= memoryPoWStartTimestamp || onlineAccountTimestamp >= memoryPoWStartTimestamp) {
+            if (!getInstance().verifyMemoryPoW(onlineAccountData, now)) {
+                LOGGER.trace(() -> String.format("Rejecting online reward-share for account %s due to invalid PoW nonce", mintingAccount.getAddress()));
+                return false;
+            }
+        }
+
+        return true;
+    }
+
+    /** Adds accounts, maybe rebuilds hashes, returns whether any new accounts were added / hashes rebuilt. */
+    private boolean addAccounts(Collection<OnlineAccountData> onlineAccountsToAdd) {
+        // For keeping track of which hashes to rebuild
+        Map<Long, Set<Byte>> hashesToRebuild = new HashMap<>();
+
+        for (OnlineAccountData onlineAccountData : onlineAccountsToAdd) {
+            boolean isNewEntry = this.addAccount(onlineAccountData);
+
+            if (isNewEntry)
+                hashesToRebuild.computeIfAbsent(onlineAccountData.getTimestamp(), k -> new HashSet<>()).add(onlineAccountData.getPublicKey()[0]);
+        }
+
+        if (hashesToRebuild.isEmpty())
+            return false;
+
+        for (var entry : hashesToRebuild.entrySet()) {
+            Long timestamp = entry.getKey();
+
+            LOGGER.trace(() -> String.format("Rehashing for timestamp %d and leading bytes %s",
+                            timestamp,
+                            entry.getValue().stream().sorted(Byte::compareUnsigned).map(leadingByte -> String.format("%02x", leadingByte)).collect(Collectors.joining(", "))
+                    )
+            );
+
+            for (Byte leadingByte : entry.getValue()) {
+                byte[] pubkeyHash = currentOnlineAccounts.get(timestamp).stream()
+                        .map(OnlineAccountData::getPublicKey)
+                        .filter(publicKey -> leadingByte == publicKey[0])
+                        .reduce(null, OnlineAccountsManager::xorByteArrayInPlace);
+
+                currentOnlineAccountsHashes.computeIfAbsent(timestamp, k -> new ConcurrentHashMap<>()).put(leadingByte, pubkeyHash);
+
+                LOGGER.trace(() -> String.format("Rebuilt hash %s for timestamp %d and leading byte %02x using %d public keys",
+                        HashCode.fromBytes(pubkeyHash),
+                        timestamp,
+                        leadingByte,
+                        currentOnlineAccounts.get(timestamp).stream()
+                                .map(OnlineAccountData::getPublicKey)
+                                .filter(publicKey -> leadingByte == publicKey[0])
+                                .count()
+                ));
+            }
+        }
+
+        LOGGER.trace(String.format("we have online accounts for timestamps: %s", String.join(", ", this.currentOnlineAccounts.keySet().stream().map(l -> Long.toString(l)).collect(Collectors.joining(", ")))));
+
+        return true;
+    }
+
+    private boolean addAccount(OnlineAccountData onlineAccountData) {
+        byte[] rewardSharePublicKey = onlineAccountData.getPublicKey();
+        long onlineAccountTimestamp = onlineAccountData.getTimestamp();
+
+        Set<OnlineAccountData> onlineAccounts = this.currentOnlineAccounts.computeIfAbsent(onlineAccountTimestamp, k -> ConcurrentHashMap.newKeySet());
+
+        boolean isSuperiorEntry = isOnlineAccountsDataSuperior(onlineAccountData);
+        if (isSuperiorEntry)
+            // Remove existing inferior entry so it can be re-added below (it's likely the existing copy is missing a nonce value)
+            onlineAccounts.remove(onlineAccountData);
+
+        boolean isNewEntry = onlineAccounts.add(onlineAccountData);
+
+        if (isNewEntry)
+            LOGGER.trace(() -> String.format("Added online account %s with timestamp %d", Base58.encode(rewardSharePublicKey), onlineAccountTimestamp));
+        else
+            LOGGER.trace(() -> String.format("Not updating existing online account %s with timestamp %d", Base58.encode(rewardSharePublicKey), onlineAccountTimestamp));
+
+        return isNewEntry;
+    }
+
+    /**
+     * Expire old entries.
+     */
+    private void expireOldOnlineAccounts() {
+        final Long now = NTP.getTime();
+        if (now == null)
+            return;
+
+        final long cutoffThreshold = now - MAX_CACHED_TIMESTAMP_SETS * getOnlineTimestampModulus();
+        this.currentOnlineAccounts.keySet().removeIf(timestamp -> timestamp < cutoffThreshold);
+        this.currentOnlineAccountsHashes.keySet().removeIf(timestamp -> timestamp < cutoffThreshold);
+    }
+
+    /**
+     * Request data from other peers
+     */
+    private void requestRemoteOnlineAccounts() {
+        final Long now = NTP.getTime();
+        if (now == null)
+            return;
+
+        // Don't bother if we're not up to date
+        if (!Controller.getInstance().isUpToDate())
+            return;
+
+        long onlineAccountsTimestamp = getCurrentOnlineAccountTimestamp();
+        if (now - onlineAccountsTimestamp >= ONLINE_ACCOUNTS_BROADCAST_BURST_LENGTH) {
+            // New online timestamp started more than 5 mins ago - we probably don't need to request so frequently
+
+            if (Controller.uptime() < ONLINE_ACCOUNTS_BROADCAST_BURST_LENGTH) {
+                // The node recently started up, so we should request at the burst interval
+                // This could allow accounts to move around the network more easily when an auto update is occurring
+            }
+            else if (now - lastOnlineAccountsRequest < ONLINE_ACCOUNTS_BROADCAST_INTERVAL) {
+                // We already requested online accounts in the last minute, so no need to request again
+                return;
+            }
+        }
+
+        LOGGER.debug("Requesting online accounts via broadcast...");
+
+        lastOnlineAccountsRequest = now;
+        Message messageV3 = new GetOnlineAccountsV3Message(currentOnlineAccountsHashes);
+        Network.getInstance().broadcast(peer -> messageV3);
+    }
+
+    /**
+     * Send online accounts that are minting on this node.
+     */
+    private void sendOurOnlineAccountsInfo() {
+        // 'current' timestamp
+        final Long onlineAccountsTimestamp = getCurrentOnlineAccountTimestamp();
+        if (onlineAccountsTimestamp == null)
+            return;
+
+        Long now = NTP.getTime();
+        if (now == null) {
+            return;
+        }
+
+        // Don't submit if we're more than 2 hours out of sync (unless we're in recovery mode)
+        final Long minLatestBlockTimestamp = now - (2 * 60 * 60 * 1000L);
+        if (!Controller.getInstance().isUpToDate(minLatestBlockTimestamp) && !Synchronizer.getInstance().getRecoveryMode()) {
+            return;
+        }
+
+        // 'next' timestamp (prioritize this as it's the most important, if mempow active)
+        final long nextOnlineAccountsTimestamp = toOnlineAccountTimestamp(now) + getOnlineTimestampModulus();
+        if (isMemoryPoWActive(now)) {
+            boolean success = computeOurAccountsForTimestamp(nextOnlineAccountsTimestamp);
+            if (!success) {
+                // We didn't compute the required nonce value(s), and so can't proceed until they have been retried
+                return;
+            }
+        }
+
+        // 'current' timestamp
+        computeOurAccountsForTimestamp(onlineAccountsTimestamp);
+    }
+
+    private boolean computeOurAccountsForTimestamp(long onlineAccountsTimestamp) {
+        List<MintingAccountData> mintingAccounts;
+        try (final Repository repository = RepositoryManager.getRepository()) {
+            mintingAccounts = repository.getAccountRepository().getMintingAccounts();
+
+            // We have no accounts to send
+            if (mintingAccounts.isEmpty())
+                return false;
+
+            // Only active reward-shares allowed
+            Iterator<MintingAccountData> iterator = mintingAccounts.iterator();
+            while (iterator.hasNext()) {
+                MintingAccountData mintingAccountData = iterator.next();
+
+                RewardShareData rewardShareData = repository.getAccountRepository().getRewardShare(mintingAccountData.getPublicKey());
+                if (rewardShareData == null) {
+                    // Reward-share doesn't even exist - probably not a good sign
+                    iterator.remove();
+                    continue;
+                }
+
+                Account mintingAccount = new Account(repository, rewardShareData.getMinter());
+                if (!mintingAccount.canMint()) {
+                    // Minting-account component of reward-share can no longer mint - disregard
+                    iterator.remove();
+                    continue;
+                }
+            }
+        } catch (DataException e) {
+            LOGGER.warn(String.format("Repository issue trying to fetch minting accounts: %s", e.getMessage()));
+            return false;
+        }
+
+        byte[] timestampBytes = Longs.toByteArray(onlineAccountsTimestamp);
+        List<OnlineAccountData> ourOnlineAccounts = new ArrayList<>();
+
+        int remaining = mintingAccounts.size();
+        for (MintingAccountData mintingAccountData : mintingAccounts) {
+            remaining--;
+            byte[] privateKey = mintingAccountData.getPrivateKey();
+            byte[] publicKey = Crypto.toPublicKey(privateKey);
+
+            // We don't want to compute the online account nonce and signature again if it already exists
+            Set<OnlineAccountData> onlineAccounts = this.currentOnlineAccounts.computeIfAbsent(onlineAccountsTimestamp, k -> ConcurrentHashMap.newKeySet());
+            boolean alreadyExists = onlineAccounts.stream().anyMatch(a -> Arrays.equals(a.getPublicKey(), publicKey));
+            if (alreadyExists) {
+                this.hasOurOnlineAccounts = true;
+
+                if (remaining > 0) {
+                    // Move on to next account
+                    continue;
+                }
+                else {
+                    // Everything exists, so return true
+                    return true;
+                }
+            }
+
+            // Generate bytes for mempow
+            byte[] mempowBytes;
+            try {
+                mempowBytes = this.getMemoryPoWBytes(publicKey, onlineAccountsTimestamp);
+            }
+            catch (IOException e) {
+                LOGGER.info("Unable to create bytes for MemoryPoW. Moving on to next account...");
+                continue;
+            }
+
+            // Compute nonce
+            Integer nonce;
+            if (isMemoryPoWActive(NTP.getTime())) {
+                try {
+                    nonce = this.computeMemoryPoW(mempowBytes, publicKey, onlineAccountsTimestamp);
+                    if (nonce == null) {
+                        // A nonce is required
+                        return false;
+                    }
+                } catch (TimeoutException e) {
+                    LOGGER.info(String.format("Timed out computing nonce for account %.8s", Base58.encode(publicKey)));
+                    return false;
+                }
+            }
+            else {
+                // Send -1 if we haven't computed a nonce due to feature trigger timestamp
+                nonce = -1;
+            }
+
+            byte[] signature = Qortal25519Extras.signForAggregation(privateKey, timestampBytes);
+
+            // Our account is online
+            OnlineAccountData ourOnlineAccountData = new OnlineAccountData(onlineAccountsTimestamp, signature, publicKey, nonce);
+
+            // Make sure to verify before adding
+            if (verifyMemoryPoW(ourOnlineAccountData, NTP.getTime())) {
+                ourOnlineAccounts.add(ourOnlineAccountData);
+            }
+        }
+
+        this.hasOurOnlineAccounts = !ourOnlineAccounts.isEmpty();
+
+        boolean hasInfoChanged = addAccounts(ourOnlineAccounts);
+
+        if (!hasInfoChanged)
+            return false;
+
+        Network.getInstance().broadcast(peer -> new OnlineAccountsV3Message(ourOnlineAccounts));
+
+        LOGGER.debug("Broadcasted {} online account{} with timestamp {}", ourOnlineAccounts.size(), (ourOnlineAccounts.size() != 1 ? "s" : ""), onlineAccountsTimestamp);
+
+        return true;
+    }
+
+
+
+    // MemoryPoW
+
+    private boolean isMemoryPoWActive(Long timestamp) {
+        if (timestamp >= BlockChain.getInstance().getOnlineAccountsMemoryPoWTimestamp() || Settings.getInstance().isOnlineAccountsMemPoWEnabled()) {
+            return true;
+        }
+        return false;
+    }
+    private byte[] getMemoryPoWBytes(byte[] publicKey, long onlineAccountsTimestamp) throws IOException {
+        byte[] timestampBytes = Longs.toByteArray(onlineAccountsTimestamp);
+
+        ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
+        outputStream.write(publicKey);
+        outputStream.write(timestampBytes);
+
+        return outputStream.toByteArray();
+    }
+
+    private Integer computeMemoryPoW(byte[] bytes, byte[] publicKey, long onlineAccountsTimestamp) throws TimeoutException {
+        if (!isMemoryPoWActive(NTP.getTime())) {
+            LOGGER.info("Mempow start timestamp not yet reached, and onlineAccountsMemPoWEnabled not enabled in settings");
+            return null;
+        }
+
+        LOGGER.info(String.format("Computing nonce for account %.8s and timestamp %d...", Base58.encode(publicKey), onlineAccountsTimestamp));
+
+        // Calculate the time until the next online timestamp and use it as a timeout when computing the nonce
+        Long startTime = NTP.getTime();
+        final long nextOnlineAccountsTimestamp = toOnlineAccountTimestamp(startTime) + getOnlineTimestampModulus();
+        long timeUntilNextTimestamp = nextOnlineAccountsTimestamp - startTime;
+
+        Integer nonce = MemoryPoW.compute2(bytes, POW_BUFFER_SIZE, POW_DIFFICULTY, timeUntilNextTimestamp);
+
+        double totalSeconds = (NTP.getTime() - startTime) / 1000.0f;
+        int minutes = (int) ((totalSeconds % 3600) / 60);
+        int seconds = (int) (totalSeconds % 60);
+        double hashRate = nonce / totalSeconds;
+
+        LOGGER.info(String.format("Computed nonce for timestamp %d and account %.8s: %d. Buffer size: %d. Difficulty: %d. " +
+                        "Time taken: %02d:%02d. Hashrate: %f", onlineAccountsTimestamp, Base58.encode(publicKey),
+                nonce, POW_BUFFER_SIZE, POW_DIFFICULTY, minutes, seconds, hashRate));
+
+        return nonce;
+    }
+
+    public boolean verifyMemoryPoW(OnlineAccountData onlineAccountData, Long timestamp) {
+        long memoryPoWStartTimestamp = BlockChain.getInstance().getOnlineAccountsMemoryPoWTimestamp();
+        if (timestamp < memoryPoWStartTimestamp && onlineAccountData.getTimestamp() < memoryPoWStartTimestamp) {
+            // Not active yet, so treat it as valid
+            return true;
+        }
+
+        int nonce = onlineAccountData.getNonce();
+
+        byte[] mempowBytes;
+        try {
+            mempowBytes = this.getMemoryPoWBytes(onlineAccountData.getPublicKey(), onlineAccountData.getTimestamp());
+        } catch (IOException e) {
+            return false;
+        }
+
+        // Verify the nonce
+        return MemoryPoW.verify2(mempowBytes, POW_BUFFER_SIZE, POW_DIFFICULTY, nonce);
+    }
+
+
+    /**
+     * Returns whether online accounts manager has any online accounts with timestamp recent enough to be considered currently online.
+     */
+    // BlockMinter: only calls this to check whether returned list is empty or not, to determine whether minting is even possible or not
+    public boolean hasOnlineAccounts() {
+        // 'current' timestamp
+        final Long onlineAccountsTimestamp = getCurrentOnlineAccountTimestamp();
+        if (onlineAccountsTimestamp == null)
+            return false;
+
+        return this.currentOnlineAccounts.containsKey(onlineAccountsTimestamp);
+    }
+
+    /**
+     * Whether we have submitted - or attempted to submit - our online account
+     * signature(s) to the network.
+     * @return true if our signature(s) have been submitted recently.
+     */
+    public boolean hasActiveOnlineAccountSignatures() {
+        final Long minLatestBlockTimestamp = NTP.getTime() - (2 * 60 * 60 * 1000L);
+        boolean isUpToDate = Controller.getInstance().isUpToDate(minLatestBlockTimestamp);
+
+        return isUpToDate && hasOurOnlineAccounts();
+    }
+
+    public boolean hasOurOnlineAccounts() {
+        return this.hasOurOnlineAccounts;
+    }
+
+    /**
+     * Returns list of online accounts matching given timestamp.
+     */
+    // Block::mint() - only wants online accounts with (online) timestamp that matches block's (online) timestamp so they can be added to new block
+    public List<OnlineAccountData> getOnlineAccounts(long onlineTimestamp) {
+        LOGGER.info(String.format("caller's timestamp: %d, our timestamps: %s", onlineTimestamp, String.join(", ", this.currentOnlineAccounts.keySet().stream().map(l -> Long.toString(l)).collect(Collectors.joining(", ")))));
+
+        return new ArrayList<>(Set.copyOf(this.currentOnlineAccounts.getOrDefault(onlineTimestamp, Collections.emptySet())));
+    }
+
+    /**
+     * Returns list of online accounts with timestamp recent enough to be considered currently online.
+     */
+    // API: calls this to return list of online accounts - probably expects ALL timestamps - but going to get 'current' from now on
+    public List<OnlineAccountData> getOnlineAccounts() {
+        // 'current' timestamp
+        final Long onlineAccountsTimestamp = getCurrentOnlineAccountTimestamp();
+        if (onlineAccountsTimestamp == null)
+            return Collections.emptyList();
+
+        return getOnlineAccounts(onlineAccountsTimestamp);
+    }
+
+    // Block processing
+
+    /**
+     * Removes previously validated entries from block's online accounts.
+     * <p>
+     * Checks both 'current' and block caches.
+     * <p>
+     * Typically called by {@link Block#areOnlineAccountsValid()}
+     */
+    public void removeKnown(Set<OnlineAccountData> blocksOnlineAccounts, Long timestamp) {
+        Set<OnlineAccountData> onlineAccounts = this.currentOnlineAccounts.get(timestamp);
+
+        // If not 'current' timestamp - try block cache instead
+        if (onlineAccounts == null)
+            onlineAccounts = this.latestBlocksOnlineAccounts.get(timestamp);
+
+        if (onlineAccounts != null)
+            blocksOnlineAccounts.removeAll(onlineAccounts);
+    }
+
+    /**
+     * Adds block's online accounts to one of OnlineAccountManager's caches.
+     * <p>
+     * It is assumed that the online accounts have been verified.
+     * <p>
+     * Typically called by {@link Block#areOnlineAccountsValid()}
+     */
+    public void addBlocksOnlineAccounts(Set<OnlineAccountData> blocksOnlineAccounts, Long timestamp) {
+        // We want to add to 'current' in preference if possible
+        if (this.currentOnlineAccounts.containsKey(timestamp)) {
+            addAccounts(blocksOnlineAccounts);
+            return;
+        }
+
+        // Add to block cache instead
+        this.latestBlocksOnlineAccounts.computeIfAbsent(timestamp, k -> ConcurrentHashMap.newKeySet())
+                .addAll(blocksOnlineAccounts);
+
+        // If block cache has grown too large then we need to trim.
+        if (this.latestBlocksOnlineAccounts.size() > MAX_BLOCKS_CACHED_ONLINE_ACCOUNTS) {
+            // However, be careful to trim the opposite end to the entry we just added!
+            Long firstKey = this.latestBlocksOnlineAccounts.firstKey();
+            if (!firstKey.equals(timestamp))
+                this.latestBlocksOnlineAccounts.remove(firstKey);
+            else
+                this.latestBlocksOnlineAccounts.remove(this.latestBlocksOnlineAccounts.lastKey());
+        }
+    }
+
+
+    // Network handlers
+
+    public void onNetworkGetOnlineAccountsV3Message(Peer peer, Message message) {
+        GetOnlineAccountsV3Message getOnlineAccountsMessage = (GetOnlineAccountsV3Message) message;
+
+        Map<Long, Map<Byte, byte[]>> peersHashes = getOnlineAccountsMessage.getHashesByTimestampThenByte();
+        List<OnlineAccountData> outgoingOnlineAccounts = new ArrayList<>();
+
+        // Warning: no double-checking/fetching - we must be ConcurrentMap compatible!
+        // So no contains()-then-get() or multiple get()s on the same key/map.
+        // We also use getOrDefault() with emptySet() on currentOnlineAccounts in case corresponding timestamp entry isn't there.
+        for (var ourOuterMapEntry : currentOnlineAccountsHashes.entrySet()) {
+            Long timestamp = ourOuterMapEntry.getKey();
+
+            var ourInnerMap = ourOuterMapEntry.getValue();
+            var peersInnerMap = peersHashes.get(timestamp);
+
+            if (peersInnerMap == null) {
+                // Peer doesn't have this timestamp, so if it's valid (i.e. not too old) then we'd have to send all of ours
+                Set<OnlineAccountData> timestampsOnlineAccounts = this.currentOnlineAccounts.getOrDefault(timestamp, Collections.emptySet());
+                outgoingOnlineAccounts.addAll(timestampsOnlineAccounts);
+
+                LOGGER.trace(() -> String.format("Going to send all %d online accounts for timestamp %d", timestampsOnlineAccounts.size(), timestamp));
+            } else {
+                // Quick cache of which leading bytes to send so we only have to filter once
+                Set<Byte> outgoingLeadingBytes = new HashSet<>();
+
+                // We have entries for this timestamp so compare against peer's entries
+                for (var ourInnerMapEntry : ourInnerMap.entrySet()) {
+                    Byte leadingByte = ourInnerMapEntry.getKey();
+                    byte[] peersHash = peersInnerMap.get(leadingByte);
+
+                    if (!Arrays.equals(ourInnerMapEntry.getValue(), peersHash)) {
+                        // For this leading byte: hashes don't match or peer doesn't have entry
+                        // Send all online accounts for this timestamp and leading byte
+                        outgoingLeadingBytes.add(leadingByte);
+                    }
+                }
+
+                int beforeAddSize = outgoingOnlineAccounts.size();
+
+                this.currentOnlineAccounts.getOrDefault(timestamp, Collections.emptySet()).stream()
+                        .filter(account -> outgoingLeadingBytes.contains(account.getPublicKey()[0]))
+                        .forEach(outgoingOnlineAccounts::add);
+
+                if (outgoingOnlineAccounts.size() > beforeAddSize)
+                    LOGGER.trace(String.format("Going to send %d online accounts for timestamp %d and leading bytes %s",
+                            outgoingOnlineAccounts.size() - beforeAddSize,
+                            timestamp,
+                            outgoingLeadingBytes.stream().sorted(Byte::compareUnsigned).map(leadingByte -> String.format("%02x", leadingByte)).collect(Collectors.joining(", "))
+                            )
+                    );
+            }
+        }
+
+        peer.sendMessage(new OnlineAccountsV3Message(outgoingOnlineAccounts));
+
+        LOGGER.trace("Sent {} online accounts to {}", outgoingOnlineAccounts.size(), peer);
+    }
+
+    public void onNetworkOnlineAccountsV3Message(Peer peer, Message message) {
+        OnlineAccountsV3Message onlineAccountsMessage = (OnlineAccountsV3Message) message;
+
+        List<OnlineAccountData> peersOnlineAccounts = onlineAccountsMessage.getOnlineAccounts();
+        LOGGER.trace("Received {} online accounts from {}", peersOnlineAccounts.size(), peer);
+
+        int importCount = 0;
+
+        // Add any online accounts to the queue that aren't already present
+        for (OnlineAccountData onlineAccountData : peersOnlineAccounts) {
+
+            Set<OnlineAccountData> onlineAccounts = this.currentOnlineAccounts.computeIfAbsent(onlineAccountData.getTimestamp(), k -> ConcurrentHashMap.newKeySet());
+            if (onlineAccounts.contains(onlineAccountData))
+                // We have already validated this online account
+                continue;
+
+            if (this.importQueueContainsExactMatch(onlineAccountData))
+                // Identical online account data already present in queue
+                continue;
+
+            boolean isNewEntry = onlineAccountsImportQueue.add(onlineAccountData);
+
+            if (isNewEntry)
+                importCount++;
+        }
+
+        if (importCount > 0)
+            LOGGER.debug("Added {} online accounts to queue", importCount);
+    }
+}

src/main/java/org/qortal/controller/PirateChainWalletController.java

@@ -0,0 +1,398 @@
+package org.qortal.controller;
+
+import com.rust.litewalletjni.LiteWalletJni;
+import org.apache.commons.io.FileUtils;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.json.JSONObject;
+import org.qortal.arbitrary.ArbitraryDataFile;
+import org.qortal.arbitrary.ArbitraryDataReader;
+import org.qortal.arbitrary.ArbitraryDataResource;
+import org.qortal.arbitrary.exception.MissingDataException;
+import org.qortal.crosschain.ForeignBlockchainException;
+import org.qortal.crosschain.PirateWallet;
+import org.qortal.data.arbitrary.ArbitraryResourceStatus;
+import org.qortal.data.transaction.ArbitraryTransactionData;
+import org.qortal.data.transaction.TransactionData;
+import org.qortal.network.Network;
+import org.qortal.network.Peer;
+import org.qortal.repository.DataException;
+import org.qortal.repository.Repository;
+import org.qortal.repository.RepositoryManager;
+import org.qortal.settings.Settings;
+import org.qortal.transaction.ArbitraryTransaction;
+import org.qortal.utils.ArbitraryTransactionUtils;
+import org.qortal.utils.Base58;
+import org.qortal.utils.FilesystemUtils;
+import org.qortal.utils.NTP;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.List;
+import java.util.Objects;
+
+public class PirateChainWalletController extends Thread {
+
+    protected static final Logger LOGGER = LogManager.getLogger(PirateChainWalletController.class);
+
+    private static PirateChainWalletController instance;
+
+    final private static long SAVE_INTERVAL = 60 * 60 * 1000L; // 1 hour
+    private long lastSaveTime = 0L;
+
+    private boolean running;
+    private PirateWallet currentWallet = null;
+    private boolean shouldLoadWallet = false;
+    private String loadStatus = null;
+
+    private static String qdnWalletSignature = "EsfUw54perxkEtfoUoL7Z97XPrNsZRZXePVZPz3cwRm9qyEPSofD5KmgVpDqVitQp7LhnZRmL6z2V9hEe1YS45T";
+
+
+    private PirateChainWalletController() {
+        this.running = true;
+    }
+
+    public static PirateChainWalletController getInstance() {
+        if (instance == null)
+            instance = new PirateChainWalletController();
+
+        return instance;
+    }
+
+    @Override
+    public void run() {
+        Thread.currentThread().setName("Pirate Chain Wallet Controller");
+
+        try {
+            while (running && !Controller.isStopping()) {
+                Thread.sleep(1000);
+
+                // Wait until we have a request to load the wallet
+                if (!shouldLoadWallet) {
+                    continue;
+                }
+
+                if (!LiteWalletJni.isLoaded()) {
+                    this.loadLibrary();
+
+                    // If still not loaded, sleep to prevent too many requests
+                    if (!LiteWalletJni.isLoaded()) {
+                        Thread.sleep(5 * 1000);
+                        continue;
+                    }
+                }
+
+                // Wallet is downloaded, so clear the status
+                this.loadStatus = null;
+
+                if (this.currentWallet == null) {
+                    // Nothing to do yet
+                    continue;
+                }
+                if (this.currentWallet.isNullSeedWallet()) {
+                    // Don't sync the null seed wallet
+                    continue;
+                }
+
+                LOGGER.debug("Syncing Pirate Chain wallet...");
+                String response = LiteWalletJni.execute("sync", "");
+                LOGGER.debug("sync response: {}", response);
+                JSONObject json = new JSONObject(response);
+                if (json.has("result")) {
+                    String result = json.getString("result");
+
+                    // We may have to set wallet to ready if this is the first ever successful sync
+                    if (Objects.equals(result, "success")) {
+                        this.currentWallet.setReady(true);
+                    }
+                }
+
+                // Rate limit sync attempts
+                Thread.sleep(30000);
+
+                // Save wallet if needed
+                Long now = NTP.getTime();
+                if (now != null && now-SAVE_INTERVAL >= this.lastSaveTime) {
+                    this.saveCurrentWallet();
+                }
+            }
+        } catch (InterruptedException e) {
+            // Fall-through to exit
+        }
+    }
+
+    public void shutdown() {
+        // Save the wallet
+        this.saveCurrentWallet();
+
+        this.running = false;
+        this.interrupt();
+    }
+
+
+    // QDN & wallet libraries
+
+    private void loadLibrary() throws InterruptedException {
+        try (final Repository repository = RepositoryManager.getRepository()) {
+
+            // Check if architecture is supported
+            String libFileName = PirateChainWalletController.getRustLibFilename();
+            if (libFileName == null) {
+                String osName = System.getProperty("os.name");
+                String osArchitecture = System.getProperty("os.arch");
+                this.loadStatus = String.format("Unsupported architecture (%s %s)", osName, osArchitecture);
+                return;
+            }
+
+            // Check if the library exists in the wallets folder
+            Path libDirectory = PirateChainWalletController.getRustLibOuterDirectory();
+            Path libPath = Paths.get(libDirectory.toString(), libFileName);
+            if (Files.exists(libPath)) {
+                // Already downloaded; we can load the library right away
+                LiteWalletJni.loadLibrary();
+                return;
+            }
+
+            // Library not found, so check if we've fetched the resource from QDN
+            ArbitraryTransactionData t = this.getTransactionData(repository);
+            if (t == null) {
+                // Can't find the transaction - maybe on a different chain?
+                return;
+            }
+
+            // Wait until we have a sufficient number of peers to attempt QDN downloads
+            List<Peer> handshakedPeers = Network.getInstance().getImmutableHandshakedPeers();
+            if (handshakedPeers.size() < Settings.getInstance().getMinBlockchainPeers()) {
+                // Wait for more peers
+                this.loadStatus = String.format("Searching for peers...");
+                return;
+            }
+
+            // Build resource
+            ArbitraryDataReader arbitraryDataReader = new ArbitraryDataReader(t.getName(),
+                    ArbitraryDataFile.ResourceIdType.NAME, t.getService(), t.getIdentifier());
+            try {
+                arbitraryDataReader.loadSynchronously(false);
+            } catch (MissingDataException e) {
+                LOGGER.info("Missing data when loading Pirate Chain library");
+            }
+
+            // Check its status
+            ArbitraryResourceStatus status = ArbitraryTransactionUtils.getStatus(
+                    t.getService(), t.getName(), t.getIdentifier(), false);
+
+            if (status.getStatus() != ArbitraryResourceStatus.Status.READY) {
+                LOGGER.info("Not ready yet: {}", status.getTitle());
+                this.loadStatus = String.format("Downloading files from QDN... (%d / %d)", status.getLocalChunkCount(), status.getTotalChunkCount());
+                return;
+            }
+
+            // Files are downloaded, so copy the necessary files to the wallets folder
+            // Delete the wallets/*/lib directory first, in case earlier versions of the wallet are present
+            Path walletsLibDirectory = PirateChainWalletController.getWalletsLibDirectory();
+            if (Files.exists(walletsLibDirectory)) {
+                FilesystemUtils.safeDeleteDirectory(walletsLibDirectory, false);
+            }
+            Files.createDirectories(libDirectory);
+            FileUtils.copyDirectory(arbitraryDataReader.getFilePath().toFile(), libDirectory.toFile());
+
+            // Clear reader cache so only one copy exists
+            ArbitraryDataResource resource = new ArbitraryDataResource(t.getName(),
+                    ArbitraryDataFile.ResourceIdType.NAME, t.getService(), t.getIdentifier());
+            resource.deleteCache();
+
+            // Finally, load the library
+            LiteWalletJni.loadLibrary();
+
+        } catch (DataException e) {
+            LOGGER.error("Repository issue when loading Pirate Chain library", e);
+        } catch (IOException e) {
+            LOGGER.error("Error when loading Pirate Chain library", e);
+        }
+    }
+
+    private ArbitraryTransactionData getTransactionData(Repository repository) {
+        try {
+            byte[] signature = Base58.decode(qdnWalletSignature);
+            TransactionData transactionData = repository.getTransactionRepository().fromSignature(signature);
+            if (!(transactionData instanceof ArbitraryTransactionData))
+                return null;
+
+            ArbitraryTransaction arbitraryTransaction = new ArbitraryTransaction(repository, transactionData);
+            if (arbitraryTransaction != null) {
+                return (ArbitraryTransactionData) arbitraryTransaction.getTransactionData();
+            }
+
+            return null;
+        } catch (DataException e) {
+            return null;
+        }
+    }
+
+    public static String getRustLibFilename() {
+        String osName = System.getProperty("os.name");
+        String osArchitecture = System.getProperty("os.arch");
+
+        if (osName.equals("Mac OS X") && osArchitecture.equals("x86_64")) {
+            return "librust-macos-x86_64.dylib";
+        }
+        else if ((osName.equals("Linux") || osName.equals("FreeBSD")) && osArchitecture.equals("aarch64")) {
+            return "librust-linux-aarch64.so";
+        }
+        else if ((osName.equals("Linux") || osName.equals("FreeBSD")) && osArchitecture.equals("amd64")) {
+            return "librust-linux-x86_64.so";
+        }
+        else if (osName.contains("Windows") && osArchitecture.equals("amd64")) {
+            return "librust-windows-x86_64.dll";
+        }
+
+        return null;
+    }
+
+    public static Path getWalletsLibDirectory() {
+        return Paths.get(Settings.getInstance().getWalletsPath(), "PirateChain", "lib");
+    }
+
+    public static Path getRustLibOuterDirectory() {
+        String sigPrefix = qdnWalletSignature.substring(0, 8);
+        return Paths.get(Settings.getInstance().getWalletsPath(), "PirateChain", "lib", sigPrefix);
+    }
+
+
+    // Wallet functions
+
+    public boolean initWithEntropy58(String entropy58) {
+        return this.initWithEntropy58(entropy58, false);
+    }
+
+    public boolean initNullSeedWallet() {
+        return this.initWithEntropy58(Base58.encode(new byte[32]), true);
+    }
+
+    private boolean initWithEntropy58(String entropy58, boolean isNullSeedWallet) {
+        // If the JNI library isn't loaded yet then we can't proceed
+        if (!LiteWalletJni.isLoaded()) {
+            shouldLoadWallet = true;
+            return false;
+        }
+
+        byte[] entropyBytes = Base58.decode(entropy58);
+
+        if (entropyBytes == null || entropyBytes.length != 32) {
+            LOGGER.info("Invalid entropy bytes");
+            return false;
+        }
+
+        if (this.currentWallet != null) {
+            if (this.currentWallet.entropyBytesEqual(entropyBytes)) {
+                // Wallet already active - nothing to do
+                return true;
+            }
+            else {
+                // Different wallet requested - close the existing one and switch over
+                this.closeCurrentWallet();
+            }
+        }
+
+        try {
+            this.currentWallet = new PirateWallet(entropyBytes, isNullSeedWallet);
+            if (!this.currentWallet.isReady()) {
+                // Don't persist wallets that aren't ready
+                this.currentWallet = null;
+            }
+            return true;
+        } catch (IOException e) {
+            LOGGER.info("Unable to initialize wallet: {}", e.getMessage());
+        }
+
+        return false;
+    }
+
+    private void saveCurrentWallet() {
+        if (this.currentWallet == null) {
+            // Nothing to do
+            return;
+        }
+        try {
+            if (this.currentWallet.save()) {
+                Long now = NTP.getTime();
+                if (now != null) {
+                    this.lastSaveTime = now;
+                }
+            }
+        } catch (IOException e) {
+            LOGGER.info("Unable to save wallet");
+        }
+    }
+
+    public PirateWallet getCurrentWallet() {
+        return this.currentWallet;
+    }
+
+    private void closeCurrentWallet() {
+        this.saveCurrentWallet();
+        this.currentWallet = null;
+    }
+
+    public void ensureInitialized() throws ForeignBlockchainException {
+        if (!LiteWalletJni.isLoaded() || this.currentWallet == null || !this.currentWallet.isInitialized()) {
+            throw new ForeignBlockchainException("Pirate wallet isn't initialized yet");
+        }
+    }
+
+    public void ensureNotNullSeed() throws ForeignBlockchainException {
+        // Safety check to make sure funds aren't sent to a null seed wallet
+        if (this.currentWallet == null || this.currentWallet.isNullSeedWallet()) {
+            throw new ForeignBlockchainException("Invalid wallet");
+        }
+    }
+
+    public void ensureSynchronized() throws ForeignBlockchainException {
+        if (this.currentWallet == null || !this.currentWallet.isSynchronized()) {
+            throw new ForeignBlockchainException("Wallet isn't synchronized yet");
+        }
+
+        String response = LiteWalletJni.execute("syncStatus", "");
+        JSONObject json = new JSONObject(response);
+        if (json.has("syncing")) {
+            boolean isSyncing = Boolean.valueOf(json.getString("syncing"));
+            if (isSyncing) {
+                long syncedBlocks = json.getLong("synced_blocks");
+                long totalBlocks = json.getLong("total_blocks");
+
+                throw new ForeignBlockchainException(String.format("Sync in progress (%d / %d). Please try again later.", syncedBlocks, totalBlocks));
+            }
+        }
+    }
+
+    public String getSyncStatus() {
+        if (this.currentWallet == null || !this.currentWallet.isInitialized()) {
+            if (this.loadStatus != null) {
+                return this.loadStatus;
+            }
+
+            return "Not initialized yet";
+        }
+
+        String syncStatusResponse = LiteWalletJni.execute("syncStatus", "");
+        org.json.JSONObject json = new JSONObject(syncStatusResponse);
+        if (json.has("syncing")) {
+            boolean isSyncing = Boolean.valueOf(json.getString("syncing"));
+            if (isSyncing) {
+                long syncedBlocks = json.getLong("synced_blocks");
+                long totalBlocks = json.getLong("total_blocks");
+                return String.format("Sync in progress (%d / %d)", syncedBlocks, totalBlocks);
+            }
+        }
+
+        boolean isSynchronized = this.currentWallet.isSynchronized();
+        if (isSynchronized) {
+            return "Synchronized";
+        }
+
+        return "Initializing wallet...";
+    }
+
+}

src/main/java/org/qortal/controller/Synchronizer.java

@@ -1,9 +1,11 @@
 package org.qortal.controller;
 
 import java.math.BigInteger;
+import java.security.SecureRandom;
 import java.text.DecimalFormat;
 import java.text.NumberFormat;
 import java.util.*;
+import java.util.concurrent.TimeUnit;
 import java.util.concurrent.locks.ReentrantLock;
 import java.util.stream.Collectors;
 
@@ -17,29 +19,23 @@ import org.qortal.block.BlockChain;
 import org.qortal.data.block.BlockData;
 import org.qortal.data.block.BlockSummaryData;
 import org.qortal.data.block.CommonBlockData;
-import org.qortal.data.network.PeerChainTipData;
 import org.qortal.data.transaction.RewardShareTransactionData;
 import org.qortal.data.transaction.TransactionData;
+import org.qortal.event.Event;
+import org.qortal.event.EventBus;
+import org.qortal.network.Network;
 import org.qortal.network.Peer;
-import org.qortal.network.message.BlockMessage;
-import org.qortal.network.message.BlockSummariesMessage;
-import org.qortal.network.message.GetBlockMessage;
-import org.qortal.network.message.GetBlockSummariesMessage;
-import org.qortal.network.message.GetSignaturesV2Message;
-import org.qortal.network.message.Message;
-import org.qortal.network.message.SignaturesMessage;
-import org.qortal.network.message.Message.MessageType;
+import org.qortal.network.message.*;
 import org.qortal.repository.DataException;
 import org.qortal.repository.Repository;
 import org.qortal.repository.RepositoryManager;
 import org.qortal.settings.Settings;
 import org.qortal.transaction.Transaction;
 import org.qortal.utils.Base58;
+import org.qortal.utils.ByteArray;
 import org.qortal.utils.NTP;
 
-import static org.qortal.network.Peer.FETCH_BLOCKS_TIMEOUT;
-
-public class Synchronizer {
+public class Synchronizer extends Thread {
 
 	private static final Logger LOGGER = LogManager.getLogger(Synchronizer.class);
 
@@ -57,26 +53,65 @@ public class Synchronizer {
 	/** Maximum number of block signatures we ask from peer in one go */
 	private static final int MAXIMUM_REQUEST_SIZE = 200; // XXX move to Settings?
 
+	private static final long RECOVERY_MODE_TIMEOUT = 10 * 60 * 1000L; // ms
 
 
+	private boolean running;
+
+	/** Latest block signatures from other peers that we know are on inferior chains. */
+	List<ByteArray> inferiorChainSignatures = new ArrayList<>();
+
+	/** Recovery mode, which is used to bring back a stalled network */
+	private boolean recoveryMode = false;
+	private boolean peersAvailable = true; // peersAvailable must default to true
+	private long timePeersLastAvailable = 0;
 
 	// Keep track of the size of the last re-org, so it can be logged
 	private int lastReorgSize;
 
+	/** Synchronization object for sync variables below */
+	public final Object syncLock = new Object();
+	/** Whether we are attempting to synchronize. */
+	private volatile boolean isSynchronizing = false;
+	/** Temporary estimate of synchronization progress for SysTray use. */
+	private volatile int syncPercent = 0;
+
+	private static volatile boolean requestSync = false;
+	private boolean syncRequestPending = false;
+
 	// Keep track of invalid blocks so that we don't keep trying to sync them
-	private Map<String, Long> invalidBlockSignatures = Collections.synchronizedMap(new HashMap<>());
+	private Map<ByteArray, Long> invalidBlockSignatures = Collections.synchronizedMap(new HashMap<>());
 	public Long timeValidBlockLastReceived = null;
 	public Long timeInvalidBlockLastReceived = null;
 
 	private static Synchronizer instance;
 
 	public enum SynchronizationResult {
-		OK, NOTHING_TO_DO, GENESIS_ONLY, NO_COMMON_BLOCK, TOO_DIVERGENT, NO_REPLY, INFERIOR_CHAIN, INVALID_DATA, NO_BLOCKCHAIN_LOCK, REPOSITORY_ISSUE, SHUTTING_DOWN;
+		OK, NOTHING_TO_DO, GENESIS_ONLY, NO_COMMON_BLOCK, TOO_DIVERGENT, NO_REPLY, INFERIOR_CHAIN, INVALID_DATA, NO_BLOCKCHAIN_LOCK, REPOSITORY_ISSUE, SHUTTING_DOWN, CHAIN_TIP_TOO_OLD;
+	}
+
+	public static class NewChainTipEvent implements Event {
+		private final BlockData priorChainTip;
+		private final BlockData newChainTip;
+
+		public NewChainTipEvent(BlockData priorChainTip, BlockData newChainTip) {
+			this.priorChainTip = priorChainTip;
+			this.newChainTip = newChainTip;
+		}
+
+		public BlockData getPriorChainTip() {
+			return this.priorChainTip;
+		}
+
+		public BlockData getNewChainTip() {
+			return this.newChainTip;
+		}
 	}
 
 	// Constructors
 
 	private Synchronizer() {
+		this.running = true;
 	}
 
 	public static Synchronizer getInstance() {
@@ -87,6 +122,306 @@ public class Synchronizer {
 	}
 
 
+	@Override
+	public void run() {
+		Thread.currentThread().setName("Synchronizer");
+
+		if (Settings.getInstance().isLite()) {
+			// Lite nodes don't need to sync
+			return;
+		}
+
+		try {
+			while (running && !Controller.isStopping()) {
+				Thread.sleep(1000);
+
+				if (requestSync) {
+					requestSync = false;
+					boolean success = Synchronizer.getInstance().potentiallySynchronize();
+					if (!success) {
+						// Something went wrong, so try again next time
+						requestSync = true;
+					}
+					// Remember that we have a pending sync request if this attempt failed
+					syncRequestPending = !success;
+				}
+			}
+		} catch (InterruptedException e) {
+			// Clear interrupted flag so we can shutdown trim threads
+			Thread.interrupted();
+			// Fall-through to exit
+		}
+	}
+
+	public void shutdown() {
+		this.running = false;
+		this.interrupt();
+	}
+
+
+
+	public boolean isSynchronizing() {
+		return this.isSynchronizing;
+	}
+
+	public boolean isSyncRequestPending() {
+		return this.syncRequestPending;
+	}
+
+	public Integer getSyncPercent() {
+		synchronized (this.syncLock) {
+			// Report as 100% synced if the latest block is within the last 60 mins
+			final Long minLatestBlockTimestamp = NTP.getTime() - (60 * 60 * 1000L);
+			if (Controller.getInstance().isUpToDate(minLatestBlockTimestamp)) {
+				return 100;
+			}
+
+			return this.isSynchronizing ? this.syncPercent : null;
+		}
+	}
+
+	public void requestSync() {
+		requestSync = true;
+	}
+
+	public boolean isSyncRequested() {
+		return requestSync;
+	}
+
+	public boolean getRecoveryMode() {
+		return this.recoveryMode;
+	}
+
+
+	public boolean potentiallySynchronize() throws InterruptedException {
+		// Already synchronizing via another thread?
+		if (this.isSynchronizing)
+			return true;
+
+		// Needs a mutable copy of the unmodifiableList
+		List<Peer> peers = new ArrayList<>(Network.getInstance().getImmutableHandshakedPeers());
+
+		// Disregard peers that have "misbehaved" recently
+		peers.removeIf(Controller.hasMisbehaved);
+
+		// Disregard peers that only have genesis block
+		peers.removeIf(Controller.hasOnlyGenesisBlock);
+
+		// Disregard peers that don't have a recent block
+		peers.removeIf(Controller.hasNoRecentBlock);
+
+		// Disregard peers that are on an old version
+		peers.removeIf(Controller.hasOldVersion);
+
+		checkRecoveryModeForPeers(peers);
+		if (recoveryMode) {
+			// Needs a mutable copy of the unmodifiableList
+			peers = new ArrayList<>(Network.getInstance().getImmutableHandshakedPeers());
+			peers.removeIf(Controller.hasOnlyGenesisBlock);
+			peers.removeIf(Controller.hasMisbehaved);
+			peers.removeIf(Controller.hasOldVersion);
+		}
+
+		// Check we have enough peers to potentially synchronize
+		if (peers.size() < Settings.getInstance().getMinBlockchainPeers())
+			return true;
+
+		// Disregard peers that have no block signature or the same block signature as us
+		peers.removeIf(Controller.hasNoOrSameBlock);
+
+		// Disregard peers that are on the same block as last sync attempt and we didn't like their chain
+		peers.removeIf(Controller.hasInferiorChainTip);
+
+		final int peersBeforeComparison = peers.size();
+
+		// Request recent block summaries from the remaining peers, and locate our common block with each
+		Synchronizer.getInstance().findCommonBlocksWithPeers(peers);
+
+		// Compare the peers against each other, and against our chain, which will return an updated list excluding those without common blocks
+		peers = Synchronizer.getInstance().comparePeers(peers);
+
+		// We may have added more inferior chain tips when comparing peers, so remove any peers that are currently on those chains
+		peers.removeIf(Controller.hasInferiorChainTip);
+
+		// Remove any peers that are no longer on a recent block since the last check
+		// Except for times when we're in recovery mode, in which case we need to keep them
+		if (!recoveryMode) {
+			peers.removeIf(Controller.hasNoRecentBlock);
+		}
+
+		final int peersRemoved = peersBeforeComparison - peers.size();
+		if (peersRemoved > 0 && peers.size() > 0)
+			LOGGER.debug(String.format("Ignoring %d peers on inferior chains. Peers remaining: %d", peersRemoved, peers.size()));
+
+		if (peers.isEmpty())
+			return true;
+
+		if (peers.size() > 1) {
+			StringBuilder finalPeersString = new StringBuilder();
+			for (Peer peer : peers)
+				finalPeersString = finalPeersString.length() > 0 ? finalPeersString.append(", ").append(peer) : finalPeersString.append(peer);
+			LOGGER.debug(String.format("Choosing random peer from: [%s]", finalPeersString.toString()));
+		}
+
+		// Pick random peer to sync with
+		int index = new SecureRandom().nextInt(peers.size());
+		Peer peer = peers.get(index);
+
+		SynchronizationResult syncResult = actuallySynchronize(peer, false);
+		if (syncResult == SynchronizationResult.NO_BLOCKCHAIN_LOCK) {
+			// No blockchain lock - force a retry by returning false
+			return false;
+		}
+
+		return true;
+	}
+
+	public SynchronizationResult actuallySynchronize(Peer peer, boolean force) throws InterruptedException {
+		boolean hasStatusChanged = false;
+		BlockData priorChainTip = Controller.getInstance().getChainTip();
+
+		synchronized (this.syncLock) {
+			this.syncPercent = (priorChainTip.getHeight() * 100) / peer.getChainTipData().getHeight();
+
+			// Only update SysTray if we're potentially changing height
+			if (this.syncPercent < 100) {
+				this.isSynchronizing = true;
+				hasStatusChanged = true;
+			}
+		}
+		peer.setSyncInProgress(true);
+
+		if (hasStatusChanged)
+			Controller.getInstance().updateSysTray();
+
+		try {
+			SynchronizationResult syncResult = Synchronizer.getInstance().synchronize(peer, force);
+			switch (syncResult) {
+				case GENESIS_ONLY:
+				case NO_COMMON_BLOCK:
+				case TOO_DIVERGENT:
+				case INVALID_DATA: {
+					// These are more serious results that warrant a cool-off
+					LOGGER.info(String.format("Failed to synchronize with peer %s (%s) - cooling off", peer, syncResult.name()));
+
+					// Don't use this peer again for a while
+					Network.getInstance().peerMisbehaved(peer);
+					break;
+				}
+
+				case INFERIOR_CHAIN: {
+					// Update our list of inferior chain tips
+					ByteArray inferiorChainSignature = ByteArray.wrap(peer.getChainTipData().getSignature());
+					if (!inferiorChainSignatures.contains(inferiorChainSignature))
+						inferiorChainSignatures.add(inferiorChainSignature);
+
+					// These are minor failure results so fine to try again
+					LOGGER.debug(() -> String.format("Refused to synchronize with peer %s (%s)", peer, syncResult.name()));
+
+					// Notify peer of our superior chain
+					Message message = Network.getInstance().buildHeightOrChainTipInfo(peer);
+					if (message == null || !peer.sendMessage(message))
+						peer.disconnect("failed to notify peer of our superior chain");
+					break;
+				}
+
+				case NO_REPLY:
+				case NO_BLOCKCHAIN_LOCK:
+				case REPOSITORY_ISSUE:
+				case CHAIN_TIP_TOO_OLD:
+					// These are minor failure results so fine to try again
+					LOGGER.debug(() -> String.format("Failed to synchronize with peer %s (%s)", peer, syncResult.name()));
+					break;
+
+				case SHUTTING_DOWN:
+					// Just quietly exit
+					break;
+
+				case OK:
+					// fall-through...
+				case NOTHING_TO_DO: {
+					// Update our list of inferior chain tips
+					ByteArray inferiorChainSignature = ByteArray.wrap(peer.getChainTipData().getSignature());
+					if (!inferiorChainSignatures.contains(inferiorChainSignature))
+						inferiorChainSignatures.add(inferiorChainSignature);
+
+					LOGGER.debug(() -> String.format("Synchronized with peer %s (%s)", peer, syncResult.name()));
+					break;
+				}
+			}
+
+			if (!running) {
+				// We've stopped
+				return SynchronizationResult.SHUTTING_DOWN;
+			}
+
+			// Has our chain tip changed?
+			BlockData newChainTip;
+
+			try (final Repository repository = RepositoryManager.getRepository()) {
+				newChainTip = repository.getBlockRepository().getLastBlock();
+			} catch (DataException e) {
+				LOGGER.warn(String.format("Repository issue when trying to fetch post-synchronization chain tip: %s", e.getMessage()));
+				return syncResult;
+			}
+
+			if (!Arrays.equals(newChainTip.getSignature(), priorChainTip.getSignature())) {
+				// Reset our cache of inferior chains
+				inferiorChainSignatures.clear();
+
+				Network.getInstance().broadcastOurChain();
+
+				EventBus.INSTANCE.notify(new NewChainTipEvent(priorChainTip, newChainTip));
+			}
+
+			return syncResult;
+		} finally {
+			this.isSynchronizing = false;
+			peer.setSyncInProgress(false);
+		}
+	}
+
+	private boolean checkRecoveryModeForPeers(List<Peer> qualifiedPeers) {
+		List<Peer> handshakedPeers = Network.getInstance().getImmutableHandshakedPeers();
+
+		if (handshakedPeers.size() > 0) {
+			// There is at least one handshaked peer
+			if (qualifiedPeers.isEmpty()) {
+				// There are no 'qualified' peers - i.e. peers that have a recent block we can sync to
+				boolean werePeersAvailable = peersAvailable;
+				peersAvailable = false;
+
+				// If peers only just became unavailable, update our record of the time they were last available
+				if (werePeersAvailable)
+					timePeersLastAvailable = NTP.getTime();
+
+				// If enough time has passed, enter recovery mode, which lifts some restrictions on who we can sync with and when we can mint
+				if (NTP.getTime() - timePeersLastAvailable > RECOVERY_MODE_TIMEOUT) {
+					if (recoveryMode == false) {
+						LOGGER.info(String.format("Peers have been unavailable for %d minutes. Entering recovery mode...", RECOVERY_MODE_TIMEOUT/60/1000));
+						recoveryMode = true;
+					}
+				}
+			} else {
+				// We now have at least one peer with a recent block, so we can exit recovery mode and sync normally
+				peersAvailable = true;
+				if (recoveryMode) {
+					LOGGER.info("Peers have become available again. Exiting recovery mode...");
+					recoveryMode = false;
+				}
+			}
+		}
+		return recoveryMode;
+	}
+
+	public void addInferiorChainSignature(byte[] inferiorSignature) {
+		// Update our list of inferior chain tips
+		ByteArray inferiorChainSignature = ByteArray.wrap(inferiorSignature);
+		if (!inferiorChainSignatures.contains(inferiorChainSignature))
+			inferiorChainSignatures.add(inferiorChainSignature);
+	}
+
+
 	/**
 	 * Iterate through a list of supplied peers, and attempt to find our common block with each.
 	 * If a common block is found, its summary will be retained in the peer's commonBlockSummary property, for processing later.
@@ -177,13 +512,13 @@ public class Synchronizer {
 			final BlockData ourLatestBlockData = repository.getBlockRepository().getLastBlock();
 			final int ourInitialHeight = ourLatestBlockData.getHeight();
 
-			PeerChainTipData peerChainTipData = peer.getChainTipData();
-			int peerHeight = peerChainTipData.getLastHeight();
-			byte[] peersLastBlockSignature = peerChainTipData.getLastBlockSignature();
+			BlockSummaryData peerChainTipData = peer.getChainTipData();
+			int peerHeight = peerChainTipData.getHeight();
+			byte[] peersLastBlockSignature = peerChainTipData.getSignature();
 
 			byte[] ourLastBlockSignature = ourLatestBlockData.getSignature();
 			LOGGER.debug(String.format("Fetching summaries from peer %s at height %d, sig %.8s, ts %d; our height %d, sig %.8s, ts %d", peer,
-					peerHeight, Base58.encode(peersLastBlockSignature), peer.getChainTipData().getLastBlockTimestamp(),
+					peerHeight, Base58.encode(peersLastBlockSignature), peerChainTipData.getTimestamp(),
 					ourInitialHeight, Base58.encode(ourLastBlockSignature), ourLatestBlockData.getTimestamp()));
 
 			List<BlockSummaryData> peerBlockSummaries = new ArrayList<>();
@@ -232,7 +567,7 @@ public class Synchronizer {
 				// If our latest block is very old, it's best that we don't try and determine the best peers to sync to.
 				// This is because it can involve very large chain comparisons, which is too intensive.
 				// In reality, most forking problems occur near the chain tips, so we will reserve this functionality for those situations.
-				final Long minLatestBlockTimestamp = Controller.getMinimumLatestBlockTimestamp();
+				Long minLatestBlockTimestamp = Controller.getMinimumLatestBlockTimestamp();
 				if (minLatestBlockTimestamp == null)
 					return peers;
 
@@ -259,6 +594,8 @@ public class Synchronizer {
 				// Create a placeholder to track of common blocks that we can discard due to being inferior chains
 				int dropPeersAfterCommonBlockHeight = 0;
 
+				NumberFormat accurateFormatter = new DecimalFormat("0.################E0");
+
 				// Remove peers with no common block data
 				Iterator iterator = peers.iterator();
 				while (iterator.hasNext()) {
@@ -279,7 +616,7 @@ public class Synchronizer {
 							// We have already determined that the correct chain diverged from a lower height. We are safe to skip these peers.
 							for (Peer peer : peersSharingCommonBlock) {
 								LOGGER.debug(String.format("Peer %s has common block at height %d but the superior chain is at height %d. Removing it from this round.", peer, commonBlockSummary.getHeight(), dropPeersAfterCommonBlockHeight));
-								Controller.getInstance().addInferiorChainSignature(peer.getChainTipData().getLastBlockSignature());
+								//this.addInferiorChainSignature(peer.getChainTipData().getLastBlockSignature());
 							}
 							continue;
 						}
@@ -290,16 +627,18 @@ public class Synchronizer {
 					int minChainLength = this.calculateMinChainLengthOfPeers(peersSharingCommonBlock, commonBlockSummary);
 
 					// Fetch block summaries from each peer
-					for (Peer peer : peersSharingCommonBlock) {
+					Iterator peersSharingCommonBlockIterator = peersSharingCommonBlock.iterator();
+					while (peersSharingCommonBlockIterator.hasNext()) {
+						Peer peer = (Peer) peersSharingCommonBlockIterator.next();
 
 						// If we're shutting down, just return the latest peer list
 						if (Controller.isStopping())
 							return peers;
 
 						// Count the number of blocks this peer has beyond our common block
-						final PeerChainTipData peerChainTipData = peer.getChainTipData();
-						final int peerHeight = peerChainTipData.getLastHeight();
-						final byte[] peerLastBlockSignature = peerChainTipData.getLastBlockSignature();
+						final BlockSummaryData peerChainTipData = peer.getChainTipData();
+						final int peerHeight = peerChainTipData.getHeight();
+						final byte[] peerLastBlockSignature = peerChainTipData.getSignature();
 						final int peerAdditionalBlocksAfterCommonBlock = peerHeight - commonBlockSummary.getHeight();
 						// Limit the number of blocks we are comparing. FUTURE: we could request more in batches, but there may not be a case when this is needed
 						int summariesRequired = Math.min(peerAdditionalBlocksAfterCommonBlock, MAXIMUM_REQUEST_SIZE);
@@ -347,6 +686,8 @@ public class Synchronizer {
 						if (this.containsInvalidBlockSummary(peer.getCommonBlockData().getBlockSummariesAfterCommonBlock())) {
 							LOGGER.debug("Ignoring peer %s because it holds an invalid block", peer);
 							peers.remove(peer);
+							peersSharingCommonBlockIterator.remove();
+							continue;
 						}
 
 						// Reduce minChainLength if needed. If we don't have any blocks, this peer will be excluded from chain weight comparisons later in the process, so we shouldn't update minChainLength
@@ -381,13 +722,13 @@ public class Synchronizer {
 					if (ourBlockSummaries.size() > 0)
 						ourChainWeight = Block.calcChainWeight(commonBlockSummary.getHeight(), commonBlockSummary.getSignature(), ourBlockSummaries, maxHeightForChainWeightComparisons);
 
-					NumberFormat formatter = new DecimalFormat("0.###E0");
-					NumberFormat accurateFormatter = new DecimalFormat("0.################E0");
-					LOGGER.debug(String.format("Our chain weight based on %d blocks is %s", (usingSameLengthChainWeight ? minChainLength : ourBlockSummaries.size()), formatter.format(ourChainWeight)));
+					LOGGER.debug(String.format("Our chain weight based on %d blocks is %s", (usingSameLengthChainWeight ? minChainLength : ourBlockSummaries.size()), accurateFormatter.format(ourChainWeight)));
 
 					LOGGER.debug(String.format("Listing peers with common block %.8s...", Base58.encode(commonBlockSummary.getSignature())));
 					for (Peer peer : peersSharingCommonBlock) {
-						final int peerHeight = peer.getChainTipData().getLastHeight();
+						BlockSummaryData peerChainTipData = peer.getChainTipData();
+						final int peerHeight = peerChainTipData.getHeight();
+						final Long peerLastBlockTimestamp = peerChainTipData.getTimestamp();
 						final int peerAdditionalBlocksAfterCommonBlock = peerHeight - commonBlockSummary.getHeight();
 						final CommonBlockData peerCommonBlockData = peer.getCommonBlockData();
 
@@ -398,6 +739,14 @@ public class Synchronizer {
 							continue;
 						}
 
+						// If peer is our of date (since our last check), we should exclude it from this round
+						minLatestBlockTimestamp = Controller.getMinimumLatestBlockTimestamp();
+						if (peerLastBlockTimestamp == null || peerLastBlockTimestamp < minLatestBlockTimestamp) {
+							LOGGER.debug(String.format("Peer %s is out of date - removing it from this round", peer));
+							peers.remove(peer);
+							continue;
+						}
+
 						final List<BlockSummaryData> peerBlockSummariesAfterCommonBlock = peerCommonBlockData.getBlockSummariesAfterCommonBlock();
 						populateBlockSummariesMinterLevels(repository, peerBlockSummariesAfterCommonBlock);
 
@@ -405,7 +754,7 @@ public class Synchronizer {
 						LOGGER.debug(String.format("About to calculate chain weight based on %d blocks for peer %s with common block %.8s (peer has %d blocks after common block)", (usingSameLengthChainWeight ? minChainLength : peerBlockSummariesAfterCommonBlock.size()), peer, Base58.encode(commonBlockSummary.getSignature()), peerAdditionalBlocksAfterCommonBlock));
 						BigInteger peerChainWeight = Block.calcChainWeight(commonBlockSummary.getHeight(), commonBlockSummary.getSignature(), peerBlockSummariesAfterCommonBlock, maxHeightForChainWeightComparisons);
 						peer.getCommonBlockData().setChainWeight(peerChainWeight);
-						LOGGER.debug(String.format("Chain weight of peer %s based on %d blocks (%d - %d) is %s", peer, (usingSameLengthChainWeight ? minChainLength : peerBlockSummariesAfterCommonBlock.size()), peerBlockSummariesAfterCommonBlock.get(0).getHeight(), peerBlockSummariesAfterCommonBlock.get(peerBlockSummariesAfterCommonBlock.size()-1).getHeight(), formatter.format(peerChainWeight)));
+						LOGGER.debug(String.format("Chain weight of peer %s based on %d blocks (%d - %d) is %s", peer, (usingSameLengthChainWeight ? minChainLength : peerBlockSummariesAfterCommonBlock.size()), peerBlockSummariesAfterCommonBlock.get(0).getHeight(), peerBlockSummariesAfterCommonBlock.get(peerBlockSummariesAfterCommonBlock.size()-1).getHeight(), accurateFormatter.format(peerChainWeight)));
 
 						// Compare against our chain - if our blockchain has greater weight then don't synchronize with peer (or any others in this group)
 						if (ourChainWeight.compareTo(peerChainWeight) > 0) {
@@ -476,7 +825,7 @@ public class Synchronizer {
 		// Calculate the length of the shortest peer chain sharing this common block
 		int minChainLength = 0;
 		for (Peer peer : peersSharingCommonBlock) {
-			final int peerHeight = peer.getChainTipData().getLastHeight();
+			final int peerHeight = peer.getChainTipData().getHeight();
 			final int peerAdditionalBlocksAfterCommonBlock = peerHeight - commonBlockSummary.getHeight();
 
 			if (peerAdditionalBlocksAfterCommonBlock < minChainLength || minChainLength == 0)
@@ -495,6 +844,10 @@ public class Synchronizer {
 
 	/* Invalid block signature tracking */
 
+	public Map<ByteArray, Long> getInvalidBlockSignatures() {
+		return this.invalidBlockSignatures;
+	}
+
 	private void addInvalidBlockSignature(byte[] signature) {
 		Long now = NTP.getTime();
 		if (now == null) {
@@ -502,8 +855,7 @@ public class Synchronizer {
 		}
 
 		// Add or update existing entry
-		String sig58 = Base58.encode(signature);
-		invalidBlockSignatures.put(sig58, now);
+		invalidBlockSignatures.put(ByteArray.wrap(signature), now);
 	}
 	private void deleteOlderInvalidSignatures(Long now) {
 		if (now == null) {
@@ -522,17 +874,16 @@ public class Synchronizer {
 			}
 		}
 	}
-	private boolean containsInvalidBlockSummary(List<BlockSummaryData> blockSummaries) {
+	public boolean containsInvalidBlockSummary(List<BlockSummaryData> blockSummaries) {
 		if (blockSummaries == null || invalidBlockSignatures == null) {
 			return false;
 		}
 
 		// Loop through our known invalid blocks and check each one against supplied block summaries
-		for (String invalidSignature58 : invalidBlockSignatures.keySet()) {
-			byte[] invalidSignature = Base58.decode(invalidSignature58);
+		for (ByteArray invalidSignature : invalidBlockSignatures.keySet()) {
 			for (BlockSummaryData blockSummary : blockSummaries) {
 				byte[] signature = blockSummary.getSignature();
-				if (Arrays.equals(signature, invalidSignature)) {
+				if (Arrays.equals(signature, invalidSignature.value)) {
 					return true;
 				}
 			}
@@ -545,10 +896,9 @@ public class Synchronizer {
 		}
 
 		// Loop through our known invalid blocks and check each one against supplied block signatures
-		for (String invalidSignature58 : invalidBlockSignatures.keySet()) {
-			byte[] invalidSignature = Base58.decode(invalidSignature58);
+		for (ByteArray invalidSignature : invalidBlockSignatures.keySet()) {
 			for (byte[] signature : blockSignatures) {
-				if (Arrays.equals(signature, invalidSignature)) {
+				if (Arrays.equals(signature, invalidSignature.value)) {
 					return true;
 				}
 			}
@@ -571,9 +921,11 @@ public class Synchronizer {
 		// Make sure we're the only thread modifying the blockchain
 		// If we're already synchronizing with another peer then this will also return fast
 		ReentrantLock blockchainLock = Controller.getInstance().getBlockchainLock();
-		if (!blockchainLock.tryLock())
+		if (!blockchainLock.tryLock(3, TimeUnit.SECONDS)) {
 			// Wasn't peer's fault we couldn't sync
+			LOGGER.info("Synchronizer couldn't acquire blockchain lock");
 			return SynchronizationResult.NO_BLOCKCHAIN_LOCK;
+		}
 
 		try {
 			try (final Repository repository = RepositoryManager.getRepository()) {
@@ -581,13 +933,13 @@ public class Synchronizer {
 					final BlockData ourLatestBlockData = repository.getBlockRepository().getLastBlock();
 					final int ourInitialHeight = ourLatestBlockData.getHeight();
 
-					PeerChainTipData peerChainTipData = peer.getChainTipData();
-					int peerHeight = peerChainTipData.getLastHeight();
-					byte[] peersLastBlockSignature = peerChainTipData.getLastBlockSignature();
+					BlockSummaryData peerChainTipData = peer.getChainTipData();
+					int peerHeight = peerChainTipData.getHeight();
+					byte[] peersLastBlockSignature = peerChainTipData.getSignature();
 
 					byte[] ourLastBlockSignature = ourLatestBlockData.getSignature();
 					String syncString = String.format("Synchronizing with peer %s at height %d, sig %.8s, ts %d; our height %d, sig %.8s, ts %d", peer,
-							peerHeight, Base58.encode(peersLastBlockSignature), peer.getChainTipData().getLastBlockTimestamp(),
+							peerHeight, Base58.encode(peersLastBlockSignature), peerChainTipData.getTimestamp(),
 							ourInitialHeight, Base58.encode(ourLastBlockSignature), ourLatestBlockData.getTimestamp());
 					LOGGER.info(syncString);
 
@@ -793,7 +1145,7 @@ public class Synchronizer {
 			return SynchronizationResult.REPOSITORY_ISSUE;
 
 		if (ourLatestBlockData.getTimestamp() < minLatestBlockTimestamp) {
-			LOGGER.info(String.format("Ditching our chain after height %d as our latest block is very old", commonBlockHeight));
+			LOGGER.info(String.format("Ditching our chain after height %d", commonBlockHeight));
 		} else {
 			// Compare chain weights
 
@@ -853,8 +1205,9 @@ public class Synchronizer {
 			BigInteger ourChainWeight = Block.calcChainWeight(commonBlockHeight, commonBlockSig, ourBlockSummaries, mutualHeight);
 			BigInteger peerChainWeight = Block.calcChainWeight(commonBlockHeight, commonBlockSig, peerBlockSummaries, mutualHeight);
 
-			NumberFormat formatter = new DecimalFormat("0.###E0");
-			LOGGER.debug(String.format("Our chain weight: %s, peer's chain weight: %s (higher is better)", formatter.format(ourChainWeight), formatter.format(peerChainWeight)));
+			NumberFormat accurateFormatter = new DecimalFormat("0.################E0");
+			LOGGER.debug(String.format("commonBlockHeight: %d, commonBlockSig: %.8s, ourBlockSummaries.size(): %d, peerBlockSummaries.size(): %d", commonBlockHeight, Base58.encode(commonBlockSig), ourBlockSummaries.size(), peerBlockSummaries.size()));
+			LOGGER.debug(String.format("Our chain weight: %s, peer's chain weight: %s (higher is better)", accurateFormatter.format(ourChainWeight), accurateFormatter.format(peerChainWeight)));
 
 			// If our blockchain has greater weight then don't synchronize with peer
 			if (ourChainWeight.compareTo(peerChainWeight) >= 0) {
@@ -957,6 +1310,16 @@ public class Synchronizer {
 				return SynchronizationResult.INVALID_DATA;
 			}
 
+			// Final check to make sure the peer isn't out of date (except for when we're in recovery mode)
+			if (!recoveryMode && peer.getChainTipData() != null) {
+				final Long minLatestBlockTimestamp = Controller.getMinimumLatestBlockTimestamp();
+				final Long peerLastBlockTimestamp = peer.getChainTipData().getTimestamp();
+				if (peerLastBlockTimestamp == null || peerLastBlockTimestamp < minLatestBlockTimestamp) {
+					LOGGER.info(String.format("Peer %s is out of date, so abandoning sync attempt", peer));
+					return SynchronizationResult.CHAIN_TIP_TOO_OLD;
+				}
+			}
+
 			byte[] nextPeerSignature = peerBlockSignatures.get(0);
 			int nextHeight = height + 1;
 
@@ -1190,12 +1553,19 @@ public class Synchronizer {
 		Message getBlockSummariesMessage = new GetBlockSummariesMessage(parentSignature, numberRequested);
 
 		Message message = peer.getResponse(getBlockSummariesMessage);
-		if (message == null || message.getType() != MessageType.BLOCK_SUMMARIES)
+		if (message == null)
 			return null;
 
-		BlockSummariesMessage blockSummariesMessage = (BlockSummariesMessage) message;
+		if (message.getType() == MessageType.BLOCK_SUMMARIES) {
+			BlockSummariesMessage blockSummariesMessage = (BlockSummariesMessage) message;
+			return blockSummariesMessage.getBlockSummaries();
+		}
+		else if (message.getType() == MessageType.BLOCK_SUMMARIES_V2) {
+			BlockSummariesV2Message blockSummariesMessage = (BlockSummariesV2Message) message;
+			return blockSummariesMessage.getBlockSummaries();
+		}
 
-		return blockSummariesMessage.getBlockSummaries();
+		return null;
 	}
 
 	private List<byte[]> getBlockSignatures(Peer peer, byte[] parentSignature, int numberRequested) throws InterruptedException {
@@ -1214,15 +1584,26 @@ public class Synchronizer {
 		Message getBlockMessage = new GetBlockMessage(signature);
 
 		Message message = peer.getResponse(getBlockMessage);
-		if (message == null || message.getType() != MessageType.BLOCK)
+		if (message == null)
 			return null;
 
-		BlockMessage blockMessage = (BlockMessage) message;
+		switch (message.getType()) {
+			case BLOCK: {
+				BlockMessage blockMessage = (BlockMessage) message;
+				return new Block(repository, blockMessage.getBlockData(), blockMessage.getTransactions(), blockMessage.getAtStates());
+			}
 
-		return new Block(repository, blockMessage.getBlockData(), blockMessage.getTransactions(), blockMessage.getAtStates());
+			case BLOCK_V2: {
+				BlockV2Message blockMessage = (BlockV2Message) message;
+				return new Block(repository, blockMessage.getBlockData(), blockMessage.getTransactions(), blockMessage.getAtStatesHash());
+			}
+
+			default:
+				return null;
+		}
 	}
 
-	private void populateBlockSummariesMinterLevels(Repository repository, List<BlockSummaryData> blockSummaries) throws DataException {
+	public void populateBlockSummariesMinterLevels(Repository repository, List<BlockSummaryData> blockSummaries) throws DataException {
 		final int firstBlockHeight = blockSummaries.get(0).getHeight();
 
 		for (int i = 0; i < blockSummaries.size(); ++i) {

src/main/java/org/qortal/controller/TransactionImporter.java

@@ -0,0 +1,441 @@
+package org.qortal.controller;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.qortal.data.block.BlockData;
+import org.qortal.data.transaction.TransactionData;
+import org.qortal.network.Network;
+import org.qortal.network.Peer;
+import org.qortal.network.message.GetTransactionMessage;
+import org.qortal.network.message.Message;
+import org.qortal.network.message.TransactionMessage;
+import org.qortal.network.message.TransactionSignaturesMessage;
+import org.qortal.repository.DataException;
+import org.qortal.repository.Repository;
+import org.qortal.repository.RepositoryManager;
+import org.qortal.settings.Settings;
+import org.qortal.transaction.Transaction;
+import org.qortal.transform.TransformationException;
+import org.qortal.utils.Base58;
+import org.qortal.utils.NTP;
+
+import java.util.*;
+import java.util.concurrent.locks.ReentrantLock;
+import java.util.stream.Collectors;
+
+public class TransactionImporter extends Thread {
+
+    private static final Logger LOGGER = LogManager.getLogger(TransactionImporter.class);
+
+    private static TransactionImporter instance;
+    private volatile boolean isStopping = false;
+
+    private static final int MAX_INCOMING_TRANSACTIONS = 5000;
+
+    /** Minimum time before considering an invalid unconfirmed transaction as "stale" */
+    public static final long INVALID_TRANSACTION_STALE_TIMEOUT = 30 * 60 * 1000L; // ms
+    /** Minimum frequency to re-request stale unconfirmed transactions from peers, to recheck validity */
+    public static final long INVALID_TRANSACTION_RECHECK_INTERVAL = 60 * 60 * 1000L; // ms\
+    /** Minimum frequency to re-request expired unconfirmed transactions from peers, to recheck validity
+     * This mainly exists to stop expired transactions from bloating the list */
+    public static final long EXPIRED_TRANSACTION_RECHECK_INTERVAL = 10 * 60 * 1000L; // ms
+
+
+    /** Map of incoming transaction that are in the import queue. Key is transaction data, value is whether signature has been validated. */
+    private final Map<TransactionData, Boolean> incomingTransactions = Collections.synchronizedMap(new HashMap<>());
+
+    /** Map of recent invalid unconfirmed transactions. Key is base58 transaction signature, value is do-not-request expiry timestamp. */
+    private final Map<String, Long> invalidUnconfirmedTransactions = Collections.synchronizedMap(new HashMap<>());
+
+
+    public static synchronized TransactionImporter getInstance() {
+        if (instance == null) {
+            instance = new TransactionImporter();
+        }
+
+        return instance;
+    }
+
+    @Override
+    public void run() {
+        Thread.currentThread().setName("Transaction Importer");
+
+        try {
+            while (!Controller.isStopping()) {
+                Thread.sleep(500L);
+
+                // Process incoming transactions queue
+                validateTransactionsInQueue();
+                importTransactionsInQueue();
+
+                // Clean up invalid incoming transactions list
+                cleanupInvalidTransactionsList(NTP.getTime());
+            }
+        } catch (InterruptedException e) {
+            // Fall through to exit thread
+        }
+    }
+
+    public void shutdown() {
+        isStopping = true;
+        this.interrupt();
+    }
+
+
+    // Incoming transactions queue
+
+    private boolean incomingTransactionQueueContains(byte[] signature) {
+        synchronized (incomingTransactions) {
+            return incomingTransactions.keySet().stream().anyMatch(t -> Arrays.equals(t.getSignature(), signature));
+        }
+    }
+
+    private void removeIncomingTransaction(byte[] signature) {
+        incomingTransactions.keySet().removeIf(t -> Arrays.equals(t.getSignature(), signature));
+    }
+
+    /**
+     * Retrieve all pending unconfirmed transactions that have had their signatures validated.
+     * @return a list of TransactionData objects, with valid signatures.
+     */
+    private List<TransactionData> getCachedSigValidTransactions() {
+        synchronized (this.incomingTransactions) {
+            return this.incomingTransactions.entrySet().stream()
+                    .filter(t -> Boolean.TRUE.equals(t.getValue()))
+                    .map(Map.Entry::getKey)
+                    .collect(Collectors.toList());
+        }
+    }
+
+    /**
+     * Validate the signatures of any transactions pending import, then update their
+     * entries in the queue to mark them as valid/invalid.
+     *
+     * No database lock is required.
+     */
+    private void validateTransactionsInQueue() {
+        if (this.incomingTransactions.isEmpty()) {
+            // Nothing to do?
+            return;
+        }
+
+        try (final Repository repository = RepositoryManager.getRepository()) {
+            // Take a snapshot of incomingTransactions, so we don't need to lock it while processing
+            Map<TransactionData, Boolean> incomingTransactionsCopy = Map.copyOf(this.incomingTransactions);
+
+            int unvalidatedCount = Collections.frequency(incomingTransactionsCopy.values(), Boolean.FALSE);
+            int validatedCount = 0;
+
+            if (unvalidatedCount > 0) {
+                LOGGER.debug("Validating signatures in incoming transactions queue (size {})...", unvalidatedCount);
+            }
+
+            // A list of all currently pending transactions that have valid signatures
+            List<Transaction> sigValidTransactions = new ArrayList<>();
+
+            // A list of signatures that became valid in this round
+            List<byte[]> newlyValidSignatures = new ArrayList<>();
+
+            boolean isLiteNode = Settings.getInstance().isLite();
+
+            // We need the latest block in order to check for expired transactions
+            BlockData latestBlock = Controller.getInstance().getChainTip();
+
+            // Signature validation round - does not require blockchain lock
+            for (Map.Entry<TransactionData, Boolean> transactionEntry : incomingTransactionsCopy.entrySet()) {
+                // Quick exit?
+                if (isStopping) {
+                    return;
+                }
+
+                TransactionData transactionData = transactionEntry.getKey();
+                Transaction transaction = Transaction.fromData(repository, transactionData);
+                String signature58 = Base58.encode(transactionData.getSignature());
+
+                Long now = NTP.getTime();
+                if (now == null) {
+                    return;
+                }
+
+                // Drop expired transactions before they are considered "sig valid"
+                if (latestBlock != null && transaction.getDeadline() <= latestBlock.getTimestamp()) {
+                    LOGGER.debug("Removing expired {} transaction {} from import queue", transactionData.getType().name(), signature58);
+                    removeIncomingTransaction(transactionData.getSignature());
+                    invalidUnconfirmedTransactions.put(signature58, (now + EXPIRED_TRANSACTION_RECHECK_INTERVAL));
+                    continue;
+                }
+
+                // Only validate signature if we haven't already done so
+                Boolean isSigValid = transactionEntry.getValue();
+                if (!Boolean.TRUE.equals(isSigValid)) {
+                    if (isLiteNode) {
+                        // Lite nodes can't easily validate transactions, so for now we will have to assume that everything is valid
+                        sigValidTransactions.add(transaction);
+                        newlyValidSignatures.add(transactionData.getSignature());
+                        // Add mark signature as valid if transaction still exists in import queue
+                        incomingTransactions.computeIfPresent(transactionData, (k, v) -> Boolean.TRUE);
+                        continue;
+                    }
+
+                    if (!transaction.isSignatureValid()) {
+                        LOGGER.debug("Ignoring {} transaction {} with invalid signature", transactionData.getType().name(), signature58);
+                        removeIncomingTransaction(transactionData.getSignature());
+
+                        // Also add to invalidIncomingTransactions map
+                        now = NTP.getTime();
+                        if (now != null) {
+                            Long expiry = now + INVALID_TRANSACTION_RECHECK_INTERVAL;
+                            LOGGER.trace("Adding invalid transaction {} to invalidUnconfirmedTransactions...", signature58);
+                            // Add to invalidUnconfirmedTransactions so that we don't keep requesting it
+                            invalidUnconfirmedTransactions.put(signature58, expiry);
+                        }
+
+                        // We're done with this transaction
+                        continue;
+                    }
+
+                    // Count the number that were validated in this round, for logging purposes
+                    validatedCount++;
+
+                    // Add mark signature as valid if transaction still exists in import queue
+                    incomingTransactions.computeIfPresent(transactionData, (k, v) -> Boolean.TRUE);
+
+                    // Signature validated in this round
+                    newlyValidSignatures.add(transactionData.getSignature());
+
+                } else {
+                    LOGGER.trace(() -> String.format("Transaction %s known to have valid signature", Base58.encode(transactionData.getSignature())));
+                }
+
+                // Signature valid - add to shortlist
+                sigValidTransactions.add(transaction);
+            }
+
+            if (unvalidatedCount > 0) {
+                LOGGER.debug("Finished validating signatures in incoming transactions queue (valid this round: {}, total pending import: {})...", validatedCount, sigValidTransactions.size());
+            }
+
+            if (!newlyValidSignatures.isEmpty()) {
+                LOGGER.debug("Broadcasting {} newly valid signatures ahead of import", newlyValidSignatures.size());
+                Message newTransactionSignatureMessage = new TransactionSignaturesMessage(newlyValidSignatures);
+                Network.getInstance().broadcast(broadcastPeer -> newTransactionSignatureMessage);
+            }
+
+        } catch (DataException e) {
+            LOGGER.error("Repository issue while processing incoming transactions", e);
+        }
+    }
+
+    /**
+     * Import any transactions in the queue that have valid signatures.
+     *
+     * A database lock is required.
+     */
+    private void importTransactionsInQueue() {
+        List<TransactionData> sigValidTransactions = this.getCachedSigValidTransactions();
+        if (sigValidTransactions.isEmpty()) {
+            // Don't bother locking if there are no new transactions to process
+            return;
+        }
+
+        if (Synchronizer.getInstance().isSyncRequested() || Synchronizer.getInstance().isSynchronizing()) {
+            // Prioritize syncing, and don't attempt to lock
+            return;
+        }
+
+        ReentrantLock blockchainLock = Controller.getInstance().getBlockchainLock();
+        if (!blockchainLock.tryLock()) {
+            LOGGER.debug("Too busy to import incoming transactions queue");
+            return;
+        }
+
+        LOGGER.debug("Importing incoming transactions queue (size {})...", sigValidTransactions.size());
+
+        int processedCount = 0;
+        try (final Repository repository = RepositoryManager.getRepository()) {
+
+            // Import transactions with valid signatures
+            try {
+                for (int i = 0; i < sigValidTransactions.size(); ++i) {
+                    if (isStopping) {
+                        return;
+                    }
+
+                    if (Synchronizer.getInstance().isSyncRequestPending()) {
+                        LOGGER.debug("Breaking out of transaction importing with {} remaining, because a sync request is pending", sigValidTransactions.size() - i);
+                        return;
+                    }
+
+                    TransactionData transactionData = sigValidTransactions.get(i);
+                    Transaction transaction = Transaction.fromData(repository, transactionData);
+
+                    Transaction.ValidationResult validationResult = transaction.importAsUnconfirmed();
+                    processedCount++;
+
+                    switch (validationResult) {
+                        case TRANSACTION_ALREADY_EXISTS: {
+                            LOGGER.trace(() -> String.format("Ignoring existing transaction %s", Base58.encode(transactionData.getSignature())));
+                            break;
+                        }
+
+                        case NO_BLOCKCHAIN_LOCK: {
+                            // Is this even possible considering we acquired blockchain lock above?
+                            LOGGER.trace(() -> String.format("Couldn't lock blockchain to import unconfirmed transaction %s", Base58.encode(transactionData.getSignature())));
+                            break;
+                        }
+
+                        case OK: {
+                            LOGGER.debug(() -> String.format("Imported %s transaction %s", transactionData.getType().name(), Base58.encode(transactionData.getSignature())));
+                            break;
+                        }
+
+                        // All other invalid cases:
+                        default: {
+                            final String signature58 = Base58.encode(transactionData.getSignature());
+                            LOGGER.debug(() -> String.format("Ignoring invalid (%s) %s transaction %s", validationResult.name(), transactionData.getType().name(), signature58));
+
+                            Long now = NTP.getTime();
+                            if (now != null && now - transactionData.getTimestamp() > INVALID_TRANSACTION_STALE_TIMEOUT) {
+                                Long expiryLength = INVALID_TRANSACTION_RECHECK_INTERVAL;
+
+                                if (validationResult == Transaction.ValidationResult.TIMESTAMP_TOO_OLD) {
+                                    // Use shorter recheck interval for expired transactions
+                                    expiryLength = EXPIRED_TRANSACTION_RECHECK_INTERVAL;
+                                }
+
+                                Long expiry = now + expiryLength;
+                                LOGGER.trace("Adding stale invalid transaction {} to invalidUnconfirmedTransactions...", signature58);
+                                // Invalid, unconfirmed transaction has become stale - add to invalidUnconfirmedTransactions so that we don't keep requesting it
+                                invalidUnconfirmedTransactions.put(signature58, expiry);
+                            }
+                        }
+                    }
+
+                    // Transaction has been processed, even if only to reject it
+                    removeIncomingTransaction(transactionData.getSignature());
+                }
+            } finally {
+                LOGGER.debug("Finished importing {} incoming transaction{}", processedCount, (processedCount == 1 ? "" : "s"));
+                blockchainLock.unlock();
+            }
+        } catch (DataException e) {
+            LOGGER.error("Repository issue while importing incoming transactions", e);
+        }
+    }
+
+    private void cleanupInvalidTransactionsList(Long now) {
+        if (now == null) {
+            return;
+        }
+        // Periodically remove invalid unconfirmed transactions from the list, so that they can be fetched again
+        invalidUnconfirmedTransactions.entrySet().removeIf(entry -> entry.getValue() == null || entry.getValue() < now);
+    }
+
+
+    // Network handlers
+
+    public void onNetworkTransactionMessage(Peer peer, Message message) {
+        TransactionMessage transactionMessage = (TransactionMessage) message;
+        TransactionData transactionData = transactionMessage.getTransactionData();
+
+        if (this.incomingTransactions.size() < MAX_INCOMING_TRANSACTIONS) {
+            synchronized (this.incomingTransactions) {
+                if (!incomingTransactionQueueContains(transactionData.getSignature())) {
+                    this.incomingTransactions.put(transactionData, Boolean.FALSE);
+                }
+            }
+        }
+    }
+
+    public void onNetworkGetTransactionMessage(Peer peer, Message message) {
+        GetTransactionMessage getTransactionMessage = (GetTransactionMessage) message;
+        byte[] signature = getTransactionMessage.getSignature();
+
+        try (final Repository repository = RepositoryManager.getRepository()) {
+            // Firstly check the sig-valid transactions that are currently queued for import
+            TransactionData transactionData = this.getCachedSigValidTransactions().stream()
+                    .filter(t -> Arrays.equals(signature, t.getSignature()))
+                    .findFirst().orElse(null);
+
+            if (transactionData == null) {
+                // Not found in import queue, so try the database
+                transactionData = repository.getTransactionRepository().fromSignature(signature);
+            }
+
+            if (transactionData == null) {
+                // Still not found - so we don't have this transaction
+                LOGGER.debug(() -> String.format("Ignoring GET_TRANSACTION request from peer %s for unknown transaction %s", peer, Base58.encode(signature)));
+                // Send no response at all???
+                return;
+            }
+
+            Message transactionMessage = new TransactionMessage(transactionData);
+            transactionMessage.setId(message.getId());
+            if (!peer.sendMessage(transactionMessage))
+                peer.disconnect("failed to send transaction");
+        } catch (DataException e) {
+            LOGGER.error(String.format("Repository issue while sending transaction %s to peer %s", Base58.encode(signature), peer), e);
+        } catch (TransformationException e) {
+            LOGGER.error(String.format("Serialization issue while sending transaction %s to peer %s", Base58.encode(signature), peer), e);
+        }
+    }
+
+    public void onNetworkGetUnconfirmedTransactionsMessage(Peer peer, Message message) {
+        try (final Repository repository = RepositoryManager.getRepository()) {
+            List<byte[]> signatures = Collections.emptyList();
+
+            // If we're NOT up-to-date then don't send out unconfirmed transactions
+            // as it's possible they are already included in a later block that we don't have.
+            if (Controller.getInstance().isUpToDate())
+                signatures = repository.getTransactionRepository().getUnconfirmedTransactionSignatures();
+
+            Message transactionSignaturesMessage = new TransactionSignaturesMessage(signatures);
+            if (!peer.sendMessage(transactionSignaturesMessage))
+                peer.disconnect("failed to send unconfirmed transaction signatures");
+        } catch (DataException e) {
+            LOGGER.error(String.format("Repository issue while sending unconfirmed transaction signatures to peer %s", peer), e);
+        }
+    }
+
+    public void onNetworkTransactionSignaturesMessage(Peer peer, Message message) {
+        TransactionSignaturesMessage transactionSignaturesMessage = (TransactionSignaturesMessage) message;
+        List<byte[]> signatures = transactionSignaturesMessage.getSignatures();
+
+        try (final Repository repository = RepositoryManager.getRepository()) {
+            for (byte[] signature : signatures) {
+                String signature58 = Base58.encode(signature);
+                if (invalidUnconfirmedTransactions.containsKey(signature58)) {
+                    // Previously invalid transaction - don't keep requesting it
+                    // It will be periodically removed from invalidUnconfirmedTransactions to allow for rechecks
+                    continue;
+                }
+
+                // Ignore if this transaction is in the queue
+                if (incomingTransactionQueueContains(signature)) {
+                    LOGGER.trace(() -> String.format("Ignoring existing queued transaction %s from peer %s", Base58.encode(signature), peer));
+                    continue;
+                }
+
+                // Do we have it already? (Before requesting transaction data itself)
+                if (repository.getTransactionRepository().exists(signature)) {
+                    LOGGER.trace(() -> String.format("Ignoring existing transaction %s from peer %s", Base58.encode(signature), peer));
+                    continue;
+                }
+
+                // Check isInterrupted() here and exit fast
+                if (Thread.currentThread().isInterrupted())
+                    return;
+
+                // Fetch actual transaction data from peer
+                Message getTransactionMessage = new GetTransactionMessage(signature);
+                if (!peer.sendMessage(getTransactionMessage)) {
+                    peer.disconnect("failed to request transaction");
+                    return;
+                }
+            }
+        } catch (DataException e) {
+            LOGGER.error(String.format("Repository issue while processing unconfirmed transactions from peer %s", peer), e);
+        }
+    }
+
+}

src/main/java/org/qortal/controller/arbitrary/ArbitraryDataBuildManager.java

@@ -37,11 +37,16 @@ public class ArbitraryDataBuildManager extends Thread {
 
     @Override
     public void run() {
+        Thread.currentThread().setName("Arbitrary Data Build Manager");
+
         try {
             // Use a fixed thread pool to execute the arbitrary data build actions (currently just a single thread)
             // This can be expanded to have multiple threads processing the build queue when needed
-            ExecutorService arbitraryDataBuildExecutor = Executors.newFixedThreadPool(1);
-            arbitraryDataBuildExecutor.execute(new ArbitraryDataBuilderThread());
+            int threadCount = 5;
+            ExecutorService arbitraryDataBuildExecutor = Executors.newFixedThreadPool(threadCount);
+            for (int i = 0; i < threadCount; i++) {
+                arbitraryDataBuildExecutor.execute(new ArbitraryDataBuilderThread());
+            }
 
             while (!isStopping) {
                 // Nothing to do yet
@@ -101,7 +106,7 @@ public class ArbitraryDataBuildManager extends Thread {
             return true;
         }
 
-        LOGGER.info("Added {} to build queue", queueItem);
+        log(queueItem, String.format("Added %s to build queue", queueItem));
 
         // Added to queue
         return true;
@@ -149,7 +154,7 @@ public class ArbitraryDataBuildManager extends Thread {
             return true;
         }
 
-        LOGGER.info("Added {} to failed builds list", queueItem);
+        log(queueItem, String.format("Added %s to failed builds list", queueItem));
 
         // Added to queue
         return true;
@@ -182,4 +187,17 @@ public class ArbitraryDataBuildManager extends Thread {
     public boolean getBuildInProgress() {
         return this.buildInProgress;
     }
+
+    private void log(ArbitraryDataBuildQueueItem queueItem, String message) {
+        if (queueItem == null) {
+            return;
+        }
+
+        if (queueItem.isHighPriority()) {
+            LOGGER.info(message);
+        }
+        else {
+            LOGGER.debug(message);
+        }
+    }
 }

src/main/java/org/qortal/controller/arbitrary/ArbitraryDataBuilderThread.java

@@ -9,6 +9,7 @@ import org.qortal.repository.DataException;
 import org.qortal.utils.NTP;
 
 import java.io.IOException;
+import java.util.Comparator;
 import java.util.Map;
 
 
@@ -20,13 +21,14 @@ public class ArbitraryDataBuilderThread implements Runnable {
 
     }
 
+    @Override
     public void run() {
-        Thread.currentThread().setName("Arbitrary Data Build Manager");
+        Thread.currentThread().setName("Arbitrary Data Builder Thread");
         ArbitraryDataBuildManager buildManager = ArbitraryDataBuildManager.getInstance();
 
         while (!Controller.isStopping()) {
             try {
-                Thread.sleep(1000);
+                Thread.sleep(100);
 
                 if (buildManager.arbitraryDataBuildQueue == null) {
                     continue;
@@ -35,48 +37,57 @@ public class ArbitraryDataBuilderThread implements Runnable {
                     continue;
                 }
 
-                // Find resources that are queued for building
-                Map.Entry<String, ArbitraryDataBuildQueueItem> next = buildManager.arbitraryDataBuildQueue
-                        .entrySet().stream()
-                        .filter(e -> e.getValue().isQueued())
-                        .findFirst().get();
-
-                if (next == null) {
-                    continue;
-                }
-
                 Long now = NTP.getTime();
                 if (now == null) {
                     continue;
                 }
 
-                ArbitraryDataBuildQueueItem queueItem = next.getValue();
+                ArbitraryDataBuildQueueItem queueItem = null;
 
-                if (queueItem == null) {
-                    this.removeFromQueue(queueItem);
+                // Find resources that are queued for building (sorted by highest priority first)
+                synchronized (buildManager.arbitraryDataBuildQueue) {
+                    Map.Entry<String, ArbitraryDataBuildQueueItem> next = buildManager.arbitraryDataBuildQueue
+                            .entrySet().stream()
+                            .filter(e -> e.getValue().isQueued())
+                            .sorted(Comparator.comparing(item -> item.getValue().getPriority()))
+                            .reduce((first, second) -> second).orElse(null);
+
+                    if (next == null) {
+                        continue;
+                    }
+
+                    queueItem = next.getValue();
+
+                    if (queueItem == null) {
+                        this.removeFromQueue(queueItem);
+                        continue;
+                    }
+
+                    // Ignore builds that have failed recently
+                    if (buildManager.isInFailedBuildsList(queueItem)) {
+                        this.removeFromQueue(queueItem);
+                        continue;
+                    }
+
+                    // Set the start timestamp, to prevent other threads from building it at the same time
+                    queueItem.prepareForBuild();
                 }
 
-                // Ignore builds that have failed recently
-                if (buildManager.isInFailedBuildsList(queueItem)) {
-                    continue;
-                }
-
-
                 try {
                     // Perform the build
-                    LOGGER.info("Building {}...", queueItem);
+                    log(queueItem, String.format("Building %s... priority: %d", queueItem, queueItem.getPriority()));
                     queueItem.build();
                     this.removeFromQueue(queueItem);
-                    LOGGER.info("Finished building {}", queueItem);
+                    log(queueItem, String.format("Finished building %s", queueItem));
 
                 } catch (MissingDataException e) {
-                    LOGGER.info("Missing data for {}: {}", queueItem, e.getMessage());
+                    log(queueItem, String.format("Missing data for %s: %s", queueItem, e.getMessage()));
                     queueItem.setFailed(true);
                     this.removeFromQueue(queueItem);
                     // Don't add to the failed builds list, as we may want to retry sooner
 
                 } catch (IOException | DataException | RuntimeException e) {
-                    LOGGER.info("Error building {}: {}", queueItem, e.getMessage());
+                    log(queueItem, String.format("Error building %s: %s", queueItem, e.getMessage()));
                     // Something went wrong - so remove it from the queue, and add to failed builds list
                     queueItem.setFailed(true);
                     buildManager.addToFailedBuildsList(queueItem);
@@ -95,4 +106,17 @@ public class ArbitraryDataBuilderThread implements Runnable {
         }
         ArbitraryDataBuildManager.getInstance().arbitraryDataBuildQueue.remove(queueItem.getUniqueKey());
     }
+
+    private void log(ArbitraryDataBuildQueueItem queueItem, String message) {
+        if (queueItem == null) {
+            return;
+        }
+
+        if (queueItem.isHighPriority()) {
+            LOGGER.info(message);
+        }
+        else {
+            LOGGER.debug(message);
+        }
+    }
 }

src/main/java/org/qortal/controller/arbitrary/ArbitraryDataCleanupManager.java

@@ -108,6 +108,10 @@ public class ArbitraryDataCleanupManager extends Thread {
 				try (final Repository repository = RepositoryManager.getRepository()) {
 					List<byte[]> signatures = repository.getTransactionRepository().getSignaturesMatchingCriteria(null, null, null, ARBITRARY_TX_TYPE, null, null, null, ConfirmationStatus.BOTH, limit, offset, true);
 					// LOGGER.info("Found {} arbitrary transactions at offset: {}, limit: {}", signatures.size(), offset, limit);
+					if (isStopping) {
+						return;
+					}
+
 					if (signatures == null || signatures.isEmpty()) {
 						offset = 0;
 						continue;
@@ -117,6 +121,10 @@ public class ArbitraryDataCleanupManager extends Thread {
 
 					// Loop through the signatures in this batch
 					for (int i=0; i<signatures.size(); i++) {
+						if (isStopping) {
+							return;
+						}
+
 						byte[] signature = signatures.get(i);
 						if (signature == null) {
 							continue;
@@ -172,9 +180,6 @@ public class ArbitraryDataCleanupManager extends Thread {
 									arbitraryTransactionData.getName(), Base58.encode(signature)));
 
 							ArbitraryTransactionUtils.deleteCompleteFileAndChunks(arbitraryTransactionData);
-
-							// We should also remove peers for this transaction from the lookup table to save space
-							this.removePeersHostingTransactionData(repository, arbitraryTransactionData);
 							continue;
 						}
 
@@ -214,7 +219,11 @@ public class ArbitraryDataCleanupManager extends Thread {
 				try (final Repository repository = RepositoryManager.getRepository()) {
 
 					// Check if there are any hosted files that don't have matching transactions
-					this.checkForExpiredTransactions(repository);
+					// UPDATE: This has been disabled for now as it was deleting valid transactions
+					// and causing chunks to go missing on the network. If ever re-enabled, we MUST
+					// ensure that original copies of data aren't deleted, and that sufficient time
+					// is allowed (ideally several hours) before treating a transaction as missing.
+					// this.checkForExpiredTransactions(repository);
 
 					// Delete additional data at random if we're over our storage limit
 					// Use the DELETION_THRESHOLD so that we only start deleting once the hard limit is reached
@@ -231,6 +240,9 @@ public class ArbitraryDataCleanupManager extends Thread {
 					// Delete random data associated with name if we're over our storage limit for this name
 					// Use the DELETION_THRESHOLD, for the same reasons as above
 					for (String followedName : storageManager.followedNames()) {
+						if (isStopping) {
+							return;
+						}
 						if (!storageManager.isStorageSpaceAvailableForName(repository, followedName, DELETION_THRESHOLD)) {
 							this.storageLimitReachedForName(repository, followedName);
 						}
@@ -253,6 +265,9 @@ public class ArbitraryDataCleanupManager extends Thread {
 
 		// Loop through each path and find those without matching signatures
 		for (Path path : allPaths) {
+			if (isStopping) {
+				break;
+			}
 			try {
 				String[] contents = path.toFile().list();
 				if (contents == null || contents.length == 0) {
@@ -279,6 +294,9 @@ public class ArbitraryDataCleanupManager extends Thread {
 	private void checkForExpiredTransactions(Repository repository) {
 		List<Path> expiredPaths = this.findPathsWithNoAssociatedTransaction(repository);
 		for (Path expiredPath : expiredPaths) {
+			if (isStopping) {
+				return;
+			}
 			LOGGER.info("Found path with no associated transaction: {}", expiredPath.toString());
 			this.safeDeleteDirectory(expiredPath.toFile(), "no matching transaction");
 		}
@@ -300,6 +318,9 @@ public class ArbitraryDataCleanupManager extends Thread {
 		// when they reach their storage limit
 		Path dataPath = Paths.get(Settings.getInstance().getDataPath());
 		for (int i=0; i<CHUNK_DELETION_BATCH_SIZE; i++) {
+			if (isStopping) {
+				return;
+			}
 			this.deleteRandomFile(repository, dataPath.toFile(), null);
 		}
 
@@ -318,6 +339,9 @@ public class ArbitraryDataCleanupManager extends Thread {
 		// when they reach their storage limit
 		Path dataPath = Paths.get(Settings.getInstance().getDataPath());
 		for (int i=0; i<CHUNK_DELETION_BATCH_SIZE; i++) {
+			if (isStopping) {
+				return;
+			}
 			this.deleteRandomFile(repository, dataPath.toFile(), name);
 		}
 	}
@@ -410,16 +434,6 @@ public class ArbitraryDataCleanupManager extends Thread {
 		return false;
 	}
 
-	private void removePeersHostingTransactionData(Repository repository, ArbitraryTransactionData transactionData) {
-		byte[] signature = transactionData.getSignature();
-		try {
-			repository.getArbitraryRepository().deleteArbitraryPeersWithSignature(signature);
-			repository.saveChanges();
-		} catch (DataException e) {
-			LOGGER.debug("Unable to delete peers from lookup table for signature: {}", Base58.encode(signature));
-		}
-	}
-
 	private void cleanupTempDirectory(String folder, long now, long minAge) {
 		String baseDir = Settings.getInstance().getTempDataPath();
 		Path tempDir = Paths.get(baseDir, folder);
@@ -429,6 +443,9 @@ public class ArbitraryDataCleanupManager extends Thread {
 		final File[] directories = tempDir.toFile().listFiles();
 		if (directories != null) {
 			for (final File directory : directories) {
+				if (isStopping) {
+					return;
+				}
 				contentsCount++;
 
 				// We're expecting the contents of each subfolder to be a directory
@@ -464,6 +481,9 @@ public class ArbitraryDataCleanupManager extends Thread {
 		final File[] directories = readerCacheNamesPath.toFile().listFiles();
 		if (directories != null) {
 			for (final File directory : directories) {
+				if (isStopping) {
+					return;
+				}
 
 				// Delete data relating to blocked names
 				String name = directory.getName();
@@ -489,6 +509,9 @@ public class ArbitraryDataCleanupManager extends Thread {
 		final File[] directories = readerNameCachePath.toFile().listFiles();
 		if (directories != null) {
 			for (final File directory : directories) {
+				if (isStopping) {
+					return;
+				}
 				// Each directory is a "service" type
 				String service = directory.getName();
 				this.cleanupReaderCacheForNameAndService(name, service, now);

src/main/java/org/qortal/controller/arbitrary/ArbitraryDataFileListManager.java

@@ -5,6 +5,9 @@ import org.apache.logging.log4j.Logger;
 import org.qortal.arbitrary.ArbitraryDataFile;
 import org.qortal.arbitrary.ArbitraryDataFileChunk;
 import org.qortal.controller.Controller;
+import org.qortal.data.arbitrary.ArbitraryDirectConnectionInfo;
+import org.qortal.data.arbitrary.ArbitraryFileListResponseInfo;
+import org.qortal.data.arbitrary.ArbitraryRelayInfo;
 import org.qortal.data.transaction.ArbitraryTransactionData;
 import org.qortal.data.transaction.TransactionData;
 import org.qortal.network.Network;
@@ -22,12 +25,15 @@ import org.qortal.utils.Triple;
 
 import java.util.*;
 
+import static org.qortal.controller.arbitrary.ArbitraryDataFileManager.MAX_FILE_HASH_RESPONSES;
+
 public class ArbitraryDataFileListManager {
 
     private static final Logger LOGGER = LogManager.getLogger(ArbitraryDataFileListManager.class);
 
     private static ArbitraryDataFileListManager instance;
 
+    private static String MIN_PEER_VERSION_FOR_FILE_LIST_STATS = "3.2.0";
 
     /**
      * Map of recent incoming requests for ARBITRARY transaction data file lists.
@@ -57,9 +63,12 @@ public class ArbitraryDataFileListManager {
 
 
     /** Maximum number of seconds that a file list relay request is able to exist on the network */
-    private static long RELAY_REQUEST_MAX_DURATION = 5000L;
+    public static long RELAY_REQUEST_MAX_DURATION = 5000L;
     /** Maximum number of hops that a file list relay request is allowed to make */
-    private static int RELAY_REQUEST_MAX_HOPS = 3;
+    public static int RELAY_REQUEST_MAX_HOPS = 4;
+
+    /** Minimum peer version to use relay */
+    public static String RELAY_MIN_PEER_VERSION = "3.4.0";
 
 
     private ArbitraryDataFileListManager() {
@@ -103,15 +112,38 @@ public class ArbitraryDataFileListManager {
         }
 
         long timeSinceLastAttempt = NTP.getTime() - lastAttemptTimestamp;
-        if (timeSinceLastAttempt > 5 * 60 * 1000L) {
-            // We haven't tried for at least 5 minutes
 
-            if (networkBroadcastCount < 5) {
-                // We've made less than 5 total attempts
+        // Allow a second attempt after 15 seconds, and another after 30 seconds
+        if (timeSinceLastAttempt > 15 * 1000L) {
+            // We haven't tried for at least 15 seconds
+
+            if (networkBroadcastCount < 3) {
+                // We've made less than 3 total attempts
                 return true;
             }
         }
 
+        // Then allow another 3 attempts, each 5 minutes apart
+        if (timeSinceLastAttempt > 5 * 60 * 1000L) {
+            // We haven't tried for at least 5 minutes
+
+            if (networkBroadcastCount < 6) {
+                // We've made less than 6 total attempts
+                return true;
+            }
+        }
+
+        // Then allow another 4 attempts, each 30 minutes apart
+        if (timeSinceLastAttempt > 30 * 60 * 1000L) {
+            // We haven't tried for at least 5 minutes
+
+            if (networkBroadcastCount < 10) {
+                // We've made less than 10 total attempts
+                return true;
+            }
+        }
+
+        // From then on, only try once every 24 hours, to reduce network spam
         if (timeSinceLastAttempt > 24 * 60 * 60 * 1000L) {
             // We haven't tried for at least 24 hours
             return true;
@@ -165,8 +197,8 @@ public class ArbitraryDataFileListManager {
             }
         }
 
-        if (timeSinceLastAttempt > 24 * 60 * 60 * 1000L) {
-            // We haven't tried for at least 24 hours
+        if (timeSinceLastAttempt > 60 * 60 * 1000L) {
+            // We haven't tried for at least 1 hour
             return true;
         }
 
@@ -223,9 +255,11 @@ public class ArbitraryDataFileListManager {
     }
 
 
-    // Lookup file lists by signature
+    // Lookup file lists by signature (and optionally hashes)
 
     public boolean fetchArbitraryDataFileList(ArbitraryTransactionData arbitraryTransactionData) {
+        byte[] digest = arbitraryTransactionData.getData();
+        byte[] metadataHash = arbitraryTransactionData.getMetadataHash();
         byte[] signature = arbitraryTransactionData.getSignature();
         String signature58 = Base58.encode(signature);
 
@@ -242,16 +276,31 @@ public class ArbitraryDataFileListManager {
                 return ArbitraryDataFileManager.getInstance().fetchDataFilesFromPeersForSignature(signature);
             }
 
-            LOGGER.debug("Skipping file list request for signature {} due to rate limit", signature58);
+            LOGGER.trace("Skipping file list request for signature {} due to rate limit", signature58);
             return false;
         }
         this.addToSignatureRequests(signature58, true, false);
 
-        List<Peer> handshakedPeers = Network.getInstance().getHandshakedPeers();
-        LOGGER.debug(String.format("Sending data file list request for signature %s to %d peers...", signature58, handshakedPeers.size()));
+        List<Peer> handshakedPeers = Network.getInstance().getImmutableHandshakedPeers();
+        List<byte[]> missingHashes = null;
+
+        // Find hashes that we are missing
+        try {
+            ArbitraryDataFile arbitraryDataFile = ArbitraryDataFile.fromHash(digest, signature);
+            arbitraryDataFile.setMetadataHash(metadataHash);
+            missingHashes = arbitraryDataFile.missingHashes();
+        } catch (DataException e) {
+            // Leave missingHashes as null, so that all hashes are requested
+        }
+        int hashCount = missingHashes != null ? missingHashes.size() : 0;
+
+        LOGGER.debug(String.format("Sending data file list request for signature %s with %d hashes to %d peers...", signature58, hashCount, handshakedPeers.size()));
+
+        // Send our address as requestingPeer, to allow for potential direct connections with seeds/peers
+        String requestingPeer = Network.getInstance().getOurExternalIpAddressAndPort();
 
         // Build request
-        Message getArbitraryDataFileListMessage = new GetArbitraryDataFileListMessage(signature, now, 0);
+        Message getArbitraryDataFileListMessage = new GetArbitraryDataFileListMessage(signature, missingHashes, now, 0, requestingPeer);
 
         // Save our request into requests map
         Triple<String, Peer, Long> requestEntry = new Triple<>(signature58, null, NTP.getTime());
@@ -291,7 +340,78 @@ public class ArbitraryDataFileListManager {
         return true;
     }
 
+    public boolean fetchArbitraryDataFileList(Peer peer, byte[] signature) {
+        String signature58 = Base58.encode(signature);
 
+        // Require an NTP sync
+        Long now = NTP.getTime();
+        if (now == null) {
+            return false;
+        }
+
+        int hashCount = 0;
+        LOGGER.debug(String.format("Sending data file list request for signature %s with %d hashes to peer %s...", signature58, hashCount, peer));
+
+        // Build request
+        // Use a time in the past, so that the recipient peer doesn't try and relay it
+        // Also, set hashes to null since it's easier to request all hashes than it is to determine which ones we need
+        // This could be optimized in the future
+        long timestamp = now - 60000L;
+        List<byte[]> hashes = null;
+        Message getArbitraryDataFileListMessage = new GetArbitraryDataFileListMessage(signature, hashes, timestamp, 0, null);
+
+        // Save our request into requests map
+        Triple<String, Peer, Long> requestEntry = new Triple<>(signature58, null, NTP.getTime());
+
+        // Assign random ID to this message
+        int id;
+        do {
+            id = new Random().nextInt(Integer.MAX_VALUE - 1) + 1;
+
+            // Put queue into map (keyed by message ID) so we can poll for a response
+            // If putIfAbsent() doesn't return null, then this ID is already taken
+        } while (arbitraryDataFileListRequests.put(id, requestEntry) != null);
+        getArbitraryDataFileListMessage.setId(id);
+
+        // Send the request
+        peer.sendMessage(getArbitraryDataFileListMessage);
+
+        // Poll to see if data has arrived
+        final long singleWait = 100;
+        long totalWait = 0;
+        while (totalWait < ArbitraryDataManager.ARBITRARY_REQUEST_TIMEOUT) {
+            try {
+                Thread.sleep(singleWait);
+            } catch (InterruptedException e) {
+                break;
+            }
+
+            requestEntry = arbitraryDataFileListRequests.get(id);
+            if (requestEntry == null)
+                return false;
+
+            if (requestEntry.getA() == null)
+                break;
+
+            totalWait += singleWait;
+        }
+        return true;
+    }
+
+    public void deleteFileListRequestsForSignature(byte[] signature) {
+        String signature58 = Base58.encode(signature);
+        for (Iterator<Map.Entry<Integer, Triple<String, Peer, Long>>> it = arbitraryDataFileListRequests.entrySet().iterator(); it.hasNext();) {
+            Map.Entry<Integer, Triple<String, Peer, Long>> entry = it.next();
+            if (entry == null || entry.getKey() == null || entry.getValue() != null) {
+                continue;
+            }
+            if (Objects.equals(entry.getValue().getA(), signature58)) {
+                // Update requests map to reflect that we've received all chunks
+                Triple<String, Peer, Long> newEntry = new Triple<>(null, null, entry.getValue().getC());
+                arbitraryDataFileListRequests.put(entry.getKey(), newEntry);
+            }
+        }
+    }
 
     // Network handlers
 
@@ -304,7 +424,14 @@ public class ArbitraryDataFileListManager {
         ArbitraryDataFileListMessage arbitraryDataFileListMessage = (ArbitraryDataFileListMessage) message;
         LOGGER.debug("Received hash list from peer {} with {} hashes", peer, arbitraryDataFileListMessage.getHashes().size());
 
-        // Do we have a pending request for this data? // TODO: might we want to relay all of them anyway?
+        if (LOGGER.isDebugEnabled() && arbitraryDataFileListMessage.getRequestTime() != null) {
+            long totalRequestTime = NTP.getTime() - arbitraryDataFileListMessage.getRequestTime();
+            LOGGER.debug("totalRequestTime: {}, requestHops: {}, peerAddress: {}, isRelayPossible: {}",
+                    totalRequestTime, arbitraryDataFileListMessage.getRequestHops(),
+                    arbitraryDataFileListMessage.getPeerAddress(), arbitraryDataFileListMessage.isRelayPossible());
+        }
+
+        // Do we have a pending request for this data?
         Triple<String, Peer, Long> request = arbitraryDataFileListRequests.get(message.getId());
         if (request == null || request.getA() == null) {
             return;
@@ -324,7 +451,6 @@ public class ArbitraryDataFileListManager {
         }
 
         ArbitraryTransactionData arbitraryTransactionData = null;
-        ArbitraryDataFileManager arbitraryDataFileManager = ArbitraryDataFileManager.getInstance();
 
         // Check transaction exists and hashes are correct
         try (final Repository repository = RepositoryManager.getRepository()) {
@@ -350,13 +476,29 @@ public class ArbitraryDataFileListManager {
 //				}
 //			}
 
-            // Update requests map to reflect that we've received it
-            Triple<String, Peer, Long> newEntry = new Triple<>(null, null, request.getC());
-            arbitraryDataFileListRequests.put(message.getId(), newEntry);
-
             if (!isRelayRequest || !Settings.getInstance().isRelayModeEnabled()) {
-                // Go and fetch the actual data, since this isn't a relay request
-                arbitraryDataFileManager.fetchArbitraryDataFiles(repository, peer, signature, arbitraryTransactionData, hashes);
+                Long now = NTP.getTime();
+
+                if (ArbitraryDataFileManager.getInstance().arbitraryDataFileHashResponses.size() < MAX_FILE_HASH_RESPONSES) {
+                    // Keep track of the hashes this peer reports to have access to
+                    for (byte[] hash : hashes) {
+                        String hash58 = Base58.encode(hash);
+
+                        // Treat null request hops as 100, so that they are able to be sorted (and put to the end of the list)
+                        int requestHops = arbitraryDataFileListMessage.getRequestHops() != null ? arbitraryDataFileListMessage.getRequestHops() : 100;
+
+                        ArbitraryFileListResponseInfo responseInfo = new ArbitraryFileListResponseInfo(hash58, signature58,
+                                peer, now, arbitraryDataFileListMessage.getRequestTime(), requestHops);
+
+                        ArbitraryDataFileManager.getInstance().arbitraryDataFileHashResponses.add(responseInfo);
+                    }
+                }
+
+                // Keep track of the source peer, for direct connections
+                if (arbitraryDataFileListMessage.getPeerAddress() != null) {
+                    ArbitraryDataFileManager.getInstance().addDirectConnectionInfoIfUnique(
+                            new ArbitraryDirectConnectionInfo(signature, arbitraryDataFileListMessage.getPeerAddress(), hashes, now));
+                }
             }
 
         } catch (DataException e) {
@@ -369,18 +511,37 @@ public class ArbitraryDataFileListManager {
             if (!isBlocked) {
                 Peer requestingPeer = request.getB();
                 if (requestingPeer != null) {
+                    Long requestTime = arbitraryDataFileListMessage.getRequestTime();
+                    Integer requestHops = arbitraryDataFileListMessage.getRequestHops();
+
                     // Add each hash to our local mapping so we know who to ask later
                     Long now = NTP.getTime();
                     for (byte[] hash : hashes) {
                         String hash58 = Base58.encode(hash);
-                        Triple<String, Peer, Long> value = new Triple<>(signature58, peer, now);
-                        arbitraryDataFileManager.arbitraryRelayMap.put(hash58, value);
-                        LOGGER.debug("Added {} to relay map: {}, {}, {}", hash58, signature58, peer, now);
+                        ArbitraryRelayInfo relayInfo = new ArbitraryRelayInfo(hash58, signature58, peer, now, requestTime, requestHops);
+                        ArbitraryDataFileManager.getInstance().addToRelayMap(relayInfo);
                     }
 
+                    // Bump requestHops if it exists
+                    if (requestHops != null) {
+                        requestHops++;
+                    }
+
+                    ArbitraryDataFileListMessage forwardArbitraryDataFileListMessage;
+
+                    // Remove optional parameters if the requesting peer doesn't support it yet
+                    // A message with less statistical data is better than no message at all
+                    if (!requestingPeer.isAtLeastVersion(MIN_PEER_VERSION_FOR_FILE_LIST_STATS)) {
+                        forwardArbitraryDataFileListMessage = new ArbitraryDataFileListMessage(signature, hashes);
+                    } else {
+                        forwardArbitraryDataFileListMessage = new ArbitraryDataFileListMessage(signature, hashes, requestTime, requestHops,
+                                arbitraryDataFileListMessage.getPeerAddress(), arbitraryDataFileListMessage.isRelayPossible());
+                    }
+                    forwardArbitraryDataFileListMessage.setId(message.getId());
+
                     // Forward to requesting peer
                     LOGGER.debug("Forwarding file list with {} hashes to requesting peer: {}", hashes.size(), requestingPeer);
-                    if (!requestingPeer.sendMessage(arbitraryDataFileListMessage)) {
+                    if (!requestingPeer.sendMessage(forwardArbitraryDataFileListMessage)) {
                         requestingPeer.disconnect("failed to forward arbitrary data file list");
                     }
                 }
@@ -404,14 +565,25 @@ public class ArbitraryDataFileListManager {
 
         // If we've seen this request recently, then ignore
         if (arbitraryDataFileListRequests.putIfAbsent(message.getId(), newEntry) != null) {
-            LOGGER.debug("Ignoring hash list request from peer {} for signature {}", peer, signature58);
+            LOGGER.trace("Ignoring hash list request from peer {} for signature {}", peer, signature58);
             return;
         }
 
-        LOGGER.debug("Received hash list request from peer {} for signature {}", peer, signature58);
+        List<byte[]> requestedHashes = getArbitraryDataFileListMessage.getHashes();
+        int hashCount = requestedHashes != null ? requestedHashes.size() : 0;
+        String requestingPeer = getArbitraryDataFileListMessage.getRequestingPeer();
+
+        if (requestingPeer != null) {
+            LOGGER.debug("Received hash list request with {} hashes from peer {} (requesting peer {}) for signature {}", hashCount, peer, requestingPeer, signature58);
+        }
+        else {
+            LOGGER.debug("Received hash list request with {} hashes from peer {} for signature {}", hashCount, peer, signature58);
+        }
 
         List<byte[]> hashes = new ArrayList<>();
         ArbitraryTransactionData transactionData = null;
+        boolean allChunksExist = false;
+        boolean hasMetadata = false;
 
         try (final Repository repository = RepositoryManager.getRepository()) {
 
@@ -427,26 +599,39 @@ public class ArbitraryDataFileListManager {
 
                     // Load file(s) and add any that exist to the list of hashes
                     ArbitraryDataFile arbitraryDataFile = ArbitraryDataFile.fromHash(hash, signature);
-                    if (metadataHash != null) {
-                        arbitraryDataFile.setMetadataHash(metadataHash);
+                    arbitraryDataFile.setMetadataHash(metadataHash);
 
-                        // If we have the metadata file, add its hash
-                        if (arbitraryDataFile.getMetadataFile().exists()) {
-                            hashes.add(arbitraryDataFile.getMetadataHash());
+                    // If the peer didn't supply a hash list, we need to return all hashes for this transaction
+                    if (requestedHashes == null || requestedHashes.isEmpty()) {
+                        requestedHashes = new ArrayList<>();
+
+                        // Add the metadata file
+                        if (arbitraryDataFile.getMetadataHash() != null) {
+                            requestedHashes.add(arbitraryDataFile.getMetadataHash());
+                            hasMetadata = true;
                         }
 
-                        for (ArbitraryDataFileChunk chunk : arbitraryDataFile.getChunks()) {
-                            if (chunk.exists()) {
-                                hashes.add(chunk.getHash());
-                                //LOGGER.trace("Added hash {}", chunk.getHash58());
-                            } else {
-                                LOGGER.debug("Couldn't add hash {} because it doesn't exist", chunk.getHash58());
-                            }
+                        // Add the chunk hashes
+                        if (arbitraryDataFile.getChunkHashes().size() > 0) {
+                            requestedHashes.addAll(arbitraryDataFile.getChunkHashes());
                         }
-                    } else {
-                        // This transaction has no chunks, so include the complete file if we have it
-                        if (arbitraryDataFile.exists()) {
-                            hashes.add(arbitraryDataFile.getHash());
+                        // Add complete file if there are no hashes
+                        else {
+                            requestedHashes.add(arbitraryDataFile.getHash());
+                        }
+                    }
+
+                    // Assume all chunks exists, unless one can't be found below
+                    allChunksExist = true;
+
+                    for (byte[] requestedHash : requestedHashes) {
+                        ArbitraryDataFileChunk chunk = ArbitraryDataFileChunk.fromHash(requestedHash, signature);
+                        if (chunk.exists()) {
+                            hashes.add(chunk.getHash());
+                            //LOGGER.trace("Added hash {}", chunk.getHash58());
+                        } else {
+                            LOGGER.trace("Couldn't add hash {} because it doesn't exist", chunk.getHash58());
+                            allChunksExist = false;
                         }
                     }
                 }
@@ -456,52 +641,86 @@ public class ArbitraryDataFileListManager {
             LOGGER.error(String.format("Repository issue while fetching arbitrary file list for peer %s", peer), e);
         }
 
+        // If the only file we have is the metadata then we shouldn't respond. Most nodes will already have that,
+        // or can use the separate metadata protocol to fetch it. This should greatly reduce network spam.
+        if (hasMetadata && hashes.size() == 1) {
+            hashes.clear();
+        }
+
         // We should only respond if we have at least one hash
         if (hashes.size() > 0) {
 
-            // Update requests map to reflect that we've sent it
-            newEntry = new Triple<>(signature58, null, now);
-            arbitraryDataFileListRequests.put(message.getId(), newEntry);
+            // Firstly we should keep track of the requesting peer, to allow for potential direct connections later
+            ArbitraryDataFileManager.getInstance().addRecentDataRequest(requestingPeer);
+
+            // We have all the chunks, so update requests map to reflect that we've sent it
+            // There is no need to keep track of the request, as we can serve all the chunks
+            if (allChunksExist) {
+                newEntry = new Triple<>(null, null, now);
+                arbitraryDataFileListRequests.put(message.getId(), newEntry);
+            }
+
+            String ourAddress = Network.getInstance().getOurExternalIpAddressAndPort();
+            ArbitraryDataFileListMessage arbitraryDataFileListMessage;
+
+            // Remove optional parameters if the requesting peer doesn't support it yet
+            // A message with less statistical data is better than no message at all
+            if (!peer.isAtLeastVersion(MIN_PEER_VERSION_FOR_FILE_LIST_STATS)) {
+                arbitraryDataFileListMessage = new ArbitraryDataFileListMessage(signature, hashes);
+            } else {
+                arbitraryDataFileListMessage = new ArbitraryDataFileListMessage(signature,
+                        hashes, NTP.getTime(), 0, ourAddress, true);
+            }
 
-            ArbitraryDataFileListMessage arbitraryDataFileListMessage = new ArbitraryDataFileListMessage(signature, hashes);
             arbitraryDataFileListMessage.setId(message.getId());
+
             if (!peer.sendMessage(arbitraryDataFileListMessage)) {
                 LOGGER.debug("Couldn't send list of hashes");
                 peer.disconnect("failed to send list of hashes");
+                return;
             }
             LOGGER.debug("Sent list of hashes (count: {})", hashes.size());
 
+            if (allChunksExist) {
+                // Nothing left to do, so return to prevent any unnecessary forwarding from occurring
+                LOGGER.debug("No need for any forwarding because file list request is fully served");
+                return;
+            }
+
         }
-        else {
-            boolean isBlocked = (transactionData == null || ArbitraryDataStorageManager.getInstance().isNameBlocked(transactionData.getName()));
-            if (Settings.getInstance().isRelayModeEnabled() && !isBlocked) {
-                // In relay mode - so ask our other peers if they have it
 
-                long requestTime = getArbitraryDataFileListMessage.getRequestTime();
-                int requestHops = getArbitraryDataFileListMessage.getRequestHops();
-                getArbitraryDataFileListMessage.setRequestHops(++requestHops);
-                long totalRequestTime = now - requestTime;
+        // We may need to forward this request on
+        boolean isBlocked = (transactionData == null || ArbitraryDataStorageManager.getInstance().isNameBlocked(transactionData.getName()));
+        if (Settings.getInstance().isRelayModeEnabled() && !isBlocked) {
+            // In relay mode - so ask our other peers if they have it
 
-                if (totalRequestTime < RELAY_REQUEST_MAX_DURATION) {
-                    // Relay request hasn't timed out yet, so can potentially be rebroadcast
-                    if (requestHops < RELAY_REQUEST_MAX_HOPS) {
-                        // Relay request hasn't reached the maximum number of hops yet, so can be rebroadcast
+            long requestTime = getArbitraryDataFileListMessage.getRequestTime();
+            int requestHops = getArbitraryDataFileListMessage.getRequestHops() + 1;
+            long totalRequestTime = now - requestTime;
 
-                        LOGGER.info("Rebroadcasting hash list request from peer {} for signature {} to our other peers... totalRequestTime: {}, requestHops: {}", peer, Base58.encode(signature), totalRequestTime, requestHops);
-                        Network.getInstance().broadcast(
-                                broadcastPeer -> broadcastPeer == peer ||
-                                        Objects.equals(broadcastPeer.getPeerData().getAddress().getHost(), peer.getPeerData().getAddress().getHost())
-                                        ? null : getArbitraryDataFileListMessage);
+            if (totalRequestTime < RELAY_REQUEST_MAX_DURATION) {
+                // Relay request hasn't timed out yet, so can potentially be rebroadcast
+                if (requestHops < RELAY_REQUEST_MAX_HOPS) {
+                    // Relay request hasn't reached the maximum number of hops yet, so can be rebroadcast
+
+                    Message relayGetArbitraryDataFileListMessage = new GetArbitraryDataFileListMessage(signature, hashes, requestTime, requestHops, requestingPeer);
+                    relayGetArbitraryDataFileListMessage.setId(message.getId());
+
+                    LOGGER.debug("Rebroadcasting hash list request from peer {} for signature {} to our other peers... totalRequestTime: {}, requestHops: {}", peer, Base58.encode(signature), totalRequestTime, requestHops);
+                    Network.getInstance().broadcast(
+                            broadcastPeer ->
+                                    !broadcastPeer.isAtLeastVersion(RELAY_MIN_PEER_VERSION) ? null :
+                                    broadcastPeer == peer || Objects.equals(broadcastPeer.getPeerData().getAddress().getHost(), peer.getPeerData().getAddress().getHost()) ? null : relayGetArbitraryDataFileListMessage
+                    );
 
-                    }
-                    else {
-                        // This relay request has reached the maximum number of allowed hops
-                    }
                 }
                 else {
-                    // This relay request has timed out
+                    // This relay request has reached the maximum number of allowed hops
                 }
             }
+            else {
+                // This relay request has timed out
+            }
         }
     }
 

src/main/java/org/qortal/controller/arbitrary/ArbitraryDataFileManager.java

@@ -1,13 +1,15 @@
 package org.qortal.controller.arbitrary;
 
+import com.google.common.net.InetAddresses;
 import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
 import org.qortal.arbitrary.ArbitraryDataFile;
 import org.qortal.controller.Controller;
-import org.qortal.data.network.ArbitraryPeerData;
+import org.qortal.data.arbitrary.ArbitraryDirectConnectionInfo;
+import org.qortal.data.arbitrary.ArbitraryFileListResponseInfo;
+import org.qortal.data.arbitrary.ArbitraryRelayInfo;
 import org.qortal.data.network.PeerData;
 import org.qortal.data.transaction.ArbitraryTransactionData;
-import org.qortal.data.transaction.TransactionData;
 import org.qortal.network.Network;
 import org.qortal.network.Peer;
 import org.qortal.network.message.*;
@@ -15,31 +17,53 @@ import org.qortal.repository.DataException;
 import org.qortal.repository.Repository;
 import org.qortal.repository.RepositoryManager;
 import org.qortal.settings.Settings;
+import org.qortal.utils.ArbitraryTransactionUtils;
 import org.qortal.utils.Base58;
 import org.qortal.utils.NTP;
-import org.qortal.utils.Triple;
 
 import java.security.SecureRandom;
 import java.util.*;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
 import java.util.stream.Collectors;
 
-public class ArbitraryDataFileManager {
+public class ArbitraryDataFileManager extends Thread {
 
     private static final Logger LOGGER = LogManager.getLogger(ArbitraryDataFileManager.class);
 
     private static ArbitraryDataFileManager instance;
+    private volatile boolean isStopping = false;
 
 
     /**
      * Map to keep track of our in progress (outgoing) arbitrary data file requests
      */
-    private Map<String, Long> arbitraryDataFileRequests = Collections.synchronizedMap(new HashMap<>());
+    public Map<String, Long> arbitraryDataFileRequests = Collections.synchronizedMap(new HashMap<>());
 
     /**
-     * Map to keep track of hashes that we might need to relay, keyed by the hash of the file (base58 encoded).
-     * Value is comprised of the base58-encoded signature, the peer that is hosting it, and the timestamp that it was added
+     * Map to keep track of hashes that we might need to relay
      */
-    public Map<String, Triple<String, Peer, Long>> arbitraryRelayMap = Collections.synchronizedMap(new HashMap<>());
+    public List<ArbitraryRelayInfo> arbitraryRelayMap = Collections.synchronizedList(new ArrayList<>());
+
+    /**
+     * List to keep track of any arbitrary data file hash responses
+     */
+    public final List<ArbitraryFileListResponseInfo> arbitraryDataFileHashResponses = Collections.synchronizedList(new ArrayList<>());
+
+    /**
+     * List to keep track of peers potentially available for direct connections, based on recent requests
+     */
+    private List<ArbitraryDirectConnectionInfo> directConnectionInfo = Collections.synchronizedList(new ArrayList<>());
+
+    /**
+     * Map to keep track of peers requesting QDN data that we hold.
+     * Key = peer address string, value = time of last request.
+     * This allows for additional "burst" connections beyond existing limits.
+     */
+    private Map<String, Long> recentDataRequests = Collections.synchronizedMap(new HashMap<>());
+
+
+    public static int MAX_FILE_HASH_RESPONSES = 1000;
 
 
     private ArbitraryDataFileManager() {
@@ -52,6 +76,32 @@ public class ArbitraryDataFileManager {
         return instance;
     }
 
+    @Override
+    public void run() {
+        Thread.currentThread().setName("Arbitrary Data File Manager");
+
+        try {
+            // Use a fixed thread pool to execute the arbitrary data file requests
+            int threadCount = 10;
+            ExecutorService arbitraryDataFileRequestExecutor = Executors.newFixedThreadPool(threadCount);
+            for (int i = 0; i < threadCount; i++) {
+                arbitraryDataFileRequestExecutor.execute(new ArbitraryDataFileRequestThread());
+            }
+
+            while (!isStopping) {
+                // Nothing to do yet
+                Thread.sleep(1000);
+            }
+        } catch (InterruptedException e) {
+            // Fall-through to exit thread...
+        }
+    }
+
+    public void shutdown() {
+        isStopping = true;
+        this.interrupt();
+    }
+
 
     public void cleanupRequestCache(Long now) {
         if (now == null) {
@@ -61,29 +111,20 @@ public class ArbitraryDataFileManager {
         arbitraryDataFileRequests.entrySet().removeIf(entry -> entry.getValue() == null || entry.getValue() < requestMinimumTimestamp);
 
         final long relayMinimumTimestamp = now - ArbitraryDataManager.getInstance().ARBITRARY_RELAY_TIMEOUT;
-        arbitraryRelayMap.entrySet().removeIf(entry -> entry.getValue().getC() == null || entry.getValue().getC() < relayMinimumTimestamp);
+        arbitraryRelayMap.removeIf(entry -> entry == null || entry.getTimestamp() == null || entry.getTimestamp() < relayMinimumTimestamp);
+        arbitraryDataFileHashResponses.removeIf(entry -> entry.getTimestamp() < relayMinimumTimestamp);
+
+        final long directConnectionInfoMinimumTimestamp = now - ArbitraryDataManager.getInstance().ARBITRARY_DIRECT_CONNECTION_INFO_TIMEOUT;
+        directConnectionInfo.removeIf(entry -> entry.getTimestamp() < directConnectionInfoMinimumTimestamp);
+
+        final long recentDataRequestMinimumTimestamp = now - ArbitraryDataManager.getInstance().ARBITRARY_RECENT_DATA_REQUESTS_TIMEOUT;
+        recentDataRequests.entrySet().removeIf(entry -> entry.getValue() < recentDataRequestMinimumTimestamp);
     }
 
 
 
     // Fetch data files by hash
 
-    public boolean fetchAllArbitraryDataFiles(Repository repository, Peer peer, byte[] signature) {
-        try {
-            TransactionData transactionData = repository.getTransactionRepository().fromSignature(signature);
-            if (!(transactionData instanceof ArbitraryTransactionData))
-                return false;
-
-            ArbitraryTransactionData arbitraryTransactionData = (ArbitraryTransactionData) transactionData;
-
-            // We use null to represent all hashes associated with this transaction
-            return this.fetchArbitraryDataFiles(repository, peer, signature, arbitraryTransactionData, null);
-
-        } catch (DataException e) {}
-
-        return false;
-    }
-
     public boolean fetchArbitraryDataFiles(Repository repository,
                                            Peer peer,
                                            byte[] signature,
@@ -94,73 +135,58 @@ public class ArbitraryDataFileManager {
         ArbitraryDataFile arbitraryDataFile = ArbitraryDataFile.fromHash(arbitraryTransactionData.getData(), signature);
         byte[] metadataHash = arbitraryTransactionData.getMetadataHash();
         arbitraryDataFile.setMetadataHash(metadataHash);
-
-        // If hashes are null, we will treat this to mean all data hashes associated with this file
-        if (hashes == null) {
-            if (metadataHash == null) {
-                // This transaction has no metadata/chunks, so use the main file hash
-                hashes = Arrays.asList(arbitraryDataFile.getHash());
-            }
-            else if (!arbitraryDataFile.getMetadataFile().exists()) {
-                // We don't have the metadata file yet, so request it
-                hashes = Arrays.asList(arbitraryDataFile.getMetadataFile().getHash());
-            }
-            else {
-                // Add the chunk hashes
-                hashes = arbitraryDataFile.getChunkHashes();
-            }
-        }
-
         boolean receivedAtLeastOneFile = false;
 
         // Now fetch actual data from this peer
         for (byte[] hash : hashes) {
+            if (isStopping) {
+                return false;
+            }
+            String hash58 = Base58.encode(hash);
             if (!arbitraryDataFile.chunkExists(hash)) {
                 // Only request the file if we aren't already requesting it from someone else
                 if (!arbitraryDataFileRequests.containsKey(Base58.encode(hash))) {
+                    LOGGER.debug("Requesting data file {} from peer {}", hash58, peer);
+                    Long startTime = NTP.getTime();
                     ArbitraryDataFileMessage receivedArbitraryDataFileMessage = fetchArbitraryDataFile(peer, null, signature, hash, null);
-                    if (receivedArbitraryDataFileMessage != null) {
-                        LOGGER.debug("Received data file {} from peer {}", receivedArbitraryDataFileMessage.getArbitraryDataFile().getHash58(), peer);
+                    Long endTime = NTP.getTime();
+                    if (receivedArbitraryDataFileMessage != null && receivedArbitraryDataFileMessage.getArbitraryDataFile() != null) {
+                        LOGGER.debug("Received data file {} from peer {}. Time taken: {} ms", receivedArbitraryDataFileMessage.getArbitraryDataFile().getHash58(), peer, (endTime-startTime));
                         receivedAtLeastOneFile = true;
+
+                        // Remove this hash from arbitraryDataFileHashResponses now that we have received it
+                        arbitraryDataFileHashResponses.remove(hash58);
                     }
                     else {
-                        LOGGER.debug("Peer {} didn't respond with data file {} for signature {}", peer, Base58.encode(hash), Base58.encode(signature));
+                        LOGGER.debug("Peer {} didn't respond with data file {} for signature {}. Time taken: {} ms", peer, Base58.encode(hash), Base58.encode(signature), (endTime-startTime));
+
+                        // Remove this hash from arbitraryDataFileHashResponses now that we have failed to receive it
+                        arbitraryDataFileHashResponses.remove(hash58);
+
+                        // Stop asking for files from this peer
+                        break;
                     }
                 }
                 else {
-                    LOGGER.debug("Already requesting data file {} for signature {}", arbitraryDataFile, Base58.encode(signature));
+                    LOGGER.trace("Already requesting data file {} for signature {} from peer {}", arbitraryDataFile, Base58.encode(signature), peer);
                 }
             }
+            else {
+                // Remove this hash from arbitraryDataFileHashResponses because we have a local copy
+                arbitraryDataFileHashResponses.remove(hash58);
+            }
         }
 
         if (receivedAtLeastOneFile) {
-            // Update our lookup table to indicate that this peer holds data for this signature
-            String peerAddress = peer.getPeerData().getAddress().toString();
-            ArbitraryPeerData arbitraryPeerData = new ArbitraryPeerData(signature, peer);
-            repository.discardChanges();
-            if (arbitraryPeerData.isPeerAddressValid()) {
-                LOGGER.debug("Adding arbitrary peer: {} for signature {}", peerAddress, Base58.encode(signature));
-                repository.getArbitraryRepository().save(arbitraryPeerData);
-                repository.saveChanges();
-            }
-
             // Invalidate the hosted transactions cache as we are now hosting something new
             ArbitraryDataStorageManager.getInstance().invalidateHostedTransactionsCache();
-        }
 
-        // Check if we have all the files we need for this transaction
-        if (arbitraryDataFile.allFilesExist()) {
+            // Check if we have all the files we need for this transaction
+            if (arbitraryDataFile.allFilesExist()) {
 
-            // We have all the chunks for this transaction, so we should invalidate the transaction's name's
-            // data cache so that it is rebuilt the next time we serve it
-            ArbitraryDataManager.getInstance().invalidateCache(arbitraryTransactionData);
-
-            // We may also need to broadcast to the network that we are now hosting files for this transaction,
-            // but only if these files are in accordance with our storage policy
-            if (ArbitraryDataStorageManager.getInstance().canStoreData(arbitraryTransactionData)) {
-                // Use a null peer address to indicate our own
-                Message newArbitrarySignatureMessage = new ArbitrarySignaturesMessage(null, 0, Arrays.asList(signature));
-                Network.getInstance().broadcast(broadcastPeer -> newArbitrarySignatureMessage);
+                // We have all the chunks for this transaction, so we should invalidate the transaction's name's
+                // data cache so that it is rebuilt the next time we serve it
+                ArbitraryDataManager.getInstance().invalidateCache(arbitraryTransactionData);
             }
         }
 
@@ -170,31 +196,45 @@ public class ArbitraryDataFileManager {
     private ArbitraryDataFileMessage fetchArbitraryDataFile(Peer peer, Peer requestingPeer, byte[] signature, byte[] hash, Message originalMessage) throws DataException {
         ArbitraryDataFile existingFile = ArbitraryDataFile.fromHash(hash, signature);
         boolean fileAlreadyExists = existingFile.exists();
-        Message message = null;
+        String hash58 = Base58.encode(hash);
+        ArbitraryDataFileMessage arbitraryDataFileMessage;
 
         // Fetch the file if it doesn't exist locally
         if (!fileAlreadyExists) {
-            String hash58 = Base58.encode(hash);
             LOGGER.debug(String.format("Fetching data file %.8s from peer %s", hash58, peer));
             arbitraryDataFileRequests.put(hash58, NTP.getTime());
             Message getArbitraryDataFileMessage = new GetArbitraryDataFileMessage(signature, hash);
 
+            Message response = null;
             try {
-                message = peer.getResponseWithTimeout(getArbitraryDataFileMessage, (int) ArbitraryDataManager.ARBITRARY_REQUEST_TIMEOUT);
+                response = peer.getResponseWithTimeout(getArbitraryDataFileMessage, (int) ArbitraryDataManager.ARBITRARY_REQUEST_TIMEOUT);
             } catch (InterruptedException e) {
-                // Will return below due to null message
+                // Will return below due to null response
             }
             arbitraryDataFileRequests.remove(hash58);
             LOGGER.trace(String.format("Removed hash %.8s from arbitraryDataFileRequests", hash58));
 
-            if (message == null || message.getType() != Message.MessageType.ARBITRARY_DATA_FILE) {
+            // We may need to remove the file list request, if we have all the files for this transaction
+            this.handleFileListRequests(signature);
+
+            if (response == null) {
+                LOGGER.debug("Received null response from peer {}", peer);
                 return null;
             }
+            if (response.getType() != MessageType.ARBITRARY_DATA_FILE) {
+                LOGGER.debug("Received response with invalid type: {} from peer {}", response.getType(), peer);
+                return null;
+            }
+
+            ArbitraryDataFileMessage peersArbitraryDataFileMessage = (ArbitraryDataFileMessage) response;
+            arbitraryDataFileMessage = new ArbitraryDataFileMessage(signature, peersArbitraryDataFileMessage.getArbitraryDataFile());
+        } else {
+            LOGGER.debug(String.format("File hash %s already exists, so skipping the request", hash58));
+            arbitraryDataFileMessage = new ArbitraryDataFileMessage(signature, existingFile);
         }
-        ArbitraryDataFileMessage arbitraryDataFileMessage = (ArbitraryDataFileMessage) message;
 
         // We might want to forward the request to the peer that originally requested it
-        this.handleArbitraryDataFileForwarding(requestingPeer, message, originalMessage);
+        this.handleArbitraryDataFileForwarding(requestingPeer, arbitraryDataFileMessage, originalMessage);
 
         boolean isRelayRequest = (requestingPeer != null);
         if (isRelayRequest) {
@@ -202,13 +242,35 @@ public class ArbitraryDataFileManager {
                 // File didn't exist locally before the request, and it's a forwarding request, so delete it
                 LOGGER.debug("Deleting file {} because it was needed for forwarding only", Base58.encode(hash));
                 ArbitraryDataFile dataFile = arbitraryDataFileMessage.getArbitraryDataFile();
-                dataFile.delete();
+
+                // Keep trying to delete the data until it is deleted, or we reach 10 attempts
+                dataFile.delete(10);
             }
         }
 
         return arbitraryDataFileMessage;
     }
 
+    private void handleFileListRequests(byte[] signature) {
+        try (final Repository repository = RepositoryManager.getRepository()) {
+
+            // Fetch the transaction data
+            ArbitraryTransactionData arbitraryTransactionData = ArbitraryTransactionUtils.fetchTransactionData(repository, signature);
+            if (arbitraryTransactionData == null) {
+                return;
+            }
+
+            boolean allChunksExist = ArbitraryTransactionUtils.allChunksExist(arbitraryTransactionData);
+
+            if (allChunksExist) {
+                // Update requests map to reflect that we've received all chunks
+                ArbitraryDataFileListManager.getInstance().deleteFileListRequestsForSignature(signature);
+            }
+
+        } catch (DataException e) {
+            LOGGER.debug("Unable to handle file list requests: {}", e.getMessage());
+        }
+    }
 
     public void handleArbitraryDataFileForwarding(Peer requestingPeer, Message message, Message originalMessage) {
         // Return if there is no originally requesting peer to forward to
@@ -238,89 +300,243 @@ public class ArbitraryDataFileManager {
 
     // Fetch data directly from peers
 
+    private List<ArbitraryDirectConnectionInfo> getDirectConnectionInfoForSignature(byte[] signature) {
+        synchronized (directConnectionInfo) {
+            return directConnectionInfo.stream().filter(i -> Arrays.equals(i.getSignature(), signature)).collect(Collectors.toList());
+        }
+    }
+
+    /**
+     * Add an ArbitraryDirectConnectionInfo item, but only if one with this peer-signature combination
+     * doesn't already exist.
+     * @param connectionInfo - the direct connection info to add
+     */
+    public void addDirectConnectionInfoIfUnique(ArbitraryDirectConnectionInfo connectionInfo) {
+        boolean peerAlreadyExists;
+        synchronized (directConnectionInfo) {
+            peerAlreadyExists = directConnectionInfo.stream()
+                    .anyMatch(i -> Arrays.equals(i.getSignature(), connectionInfo.getSignature())
+                            && Objects.equals(i.getPeerAddress(), connectionInfo.getPeerAddress()));
+        }
+        if (!peerAlreadyExists) {
+            directConnectionInfo.add(connectionInfo);
+        }
+    }
+
+    private void removeDirectConnectionInfo(ArbitraryDirectConnectionInfo connectionInfo) {
+        this.directConnectionInfo.remove(connectionInfo);
+    }
+
     public boolean fetchDataFilesFromPeersForSignature(byte[] signature) {
         String signature58 = Base58.encode(signature);
-        ArbitraryDataFileListManager.getInstance().addToSignatureRequests(signature58, false, true);
 
-        // Firstly fetch peers that claim to be hosting files for this signature
-        try (final Repository repository = RepositoryManager.getRepository()) {
+        boolean success = false;
 
-            List<ArbitraryPeerData> peers = repository.getArbitraryRepository().getArbitraryPeerDataForSignature(signature);
-            if (peers == null || peers.isEmpty()) {
-                LOGGER.debug("No peers found for signature {}", signature58);
-                return false;
-            }
-
-            LOGGER.debug("Attempting a direct peer connection for signature {}...", signature58);
-
-            // Peers found, so pick a random one and request data from it
-            int index = new SecureRandom().nextInt(peers.size());
-            ArbitraryPeerData arbitraryPeerData = peers.get(index);
-            String peerAddressString = arbitraryPeerData.getPeerAddress();
-            boolean success = Network.getInstance().requestDataFromPeer(peerAddressString, signature);
-
-            // Parse the peer address to find the host and port
-            String host = null;
-            int port = -1;
-            String[] parts = peerAddressString.split(":");
-            if (parts.length > 1) {
-                host = parts[0];
-                port = Integer.parseInt(parts[1]);
-            }
-
-            // If unsuccessful, and using a non-standard port, try a second connection with the default listen port,
-            // since almost all nodes use that. This is a workaround to account for any ephemeral ports that may
-            // have made it into the dataset.
-            if (!success) {
-                if (host != null && port > 0) {
-                    int defaultPort = Settings.getInstance().getDefaultListenPort();
-                    if (port != defaultPort) {
-                        String newPeerAddressString = String.format("%s:%d", host, defaultPort);
-                        success = Network.getInstance().requestDataFromPeer(newPeerAddressString, signature);
-                    }
+        try {
+            while (!success) {
+                if (isStopping) {
+                    return false;
                 }
-            }
+                Thread.sleep(500L);
 
-            // If _still_ unsuccessful, try matching the peer's IP address with some known peers, and then connect
-            // to each of those in turn until one succeeds.
-            if (!success) {
-                if (host != null) {
-                    final String finalHost = host;
-                    List<PeerData> knownPeers = Network.getInstance().getAllKnownPeers().stream()
-                            .filter(knownPeerData -> knownPeerData.getAddress().getHost().equals(finalHost))
-                            .collect(Collectors.toList());
-                    // Loop through each match and attempt a connection
-                    for (PeerData matchingPeer : knownPeers) {
-                        String matchingPeerAddress = matchingPeer.getAddress().toString();
-                        success = Network.getInstance().requestDataFromPeer(matchingPeerAddress, signature);
-                        if (success) {
-                            // Successfully connected, so stop making connections
-                            break;
+                // Firstly fetch peers that claim to be hosting files for this signature
+                List<ArbitraryDirectConnectionInfo> connectionInfoList = getDirectConnectionInfoForSignature(signature);
+                if (connectionInfoList == null || connectionInfoList.isEmpty()) {
+                    LOGGER.debug("No remaining direct connection peers found for signature {}", signature58);
+                    return false;
+                }
+
+                LOGGER.debug("Attempting a direct peer connection for signature {}...", signature58);
+
+                // Peers found, so pick one with the highest number of chunks
+                Comparator<ArbitraryDirectConnectionInfo> highestChunkCountFirstComparator =
+                        Comparator.comparingInt(ArbitraryDirectConnectionInfo::getHashCount).reversed();
+                ArbitraryDirectConnectionInfo directConnectionInfo = connectionInfoList.stream()
+                        .sorted(highestChunkCountFirstComparator).findFirst().orElse(null);
+
+                if (directConnectionInfo == null) {
+                    return false;
+                }
+
+                // Remove from the list so that a different peer is tried next time
+                removeDirectConnectionInfo(directConnectionInfo);
+
+                String peerAddressString = directConnectionInfo.getPeerAddress();
+
+                // Parse the peer address to find the host and port
+                String host = null;
+                int port = -1;
+                String[] parts = peerAddressString.split(":");
+                if (parts.length > 1) {
+                    host = parts[0];
+                    port = Integer.parseInt(parts[1]);
+                } else {
+                    // Assume no port included
+                    host = peerAddressString;
+                    // Use default listen port
+                    port = Settings.getInstance().getDefaultListenPort();
+                }
+
+                String peerAddressStringWithPort = String.format("%s:%d", host, port);
+                success = Network.getInstance().requestDataFromPeer(peerAddressStringWithPort, signature);
+
+                int defaultPort = Settings.getInstance().getDefaultListenPort();
+
+                // If unsuccessful, and using a non-standard port, try a second connection with the default listen port,
+                // since almost all nodes use that. This is a workaround to account for any ephemeral ports that may
+                // have made it into the dataset.
+                if (!success) {
+                    if (host != null && port > 0) {
+                        if (port != defaultPort) {
+                            String newPeerAddressString = String.format("%s:%d", host, defaultPort);
+                            success = Network.getInstance().requestDataFromPeer(newPeerAddressString, signature);
                         }
                     }
                 }
-            }
 
-            // Keep track of the success or failure
-            arbitraryPeerData.markAsAttempted();
-            if (success) {
-                arbitraryPeerData.markAsRetrieved();
-                arbitraryPeerData.incrementSuccesses();
-            }
-            else {
-                arbitraryPeerData.incrementFailures();
-            }
-            repository.discardChanges();
-            repository.getArbitraryRepository().save(arbitraryPeerData);
-            repository.saveChanges();
+                // If _still_ unsuccessful, try matching the peer's IP address with some known peers, and then connect
+                // to each of those in turn until one succeeds.
+                if (!success) {
+                    if (host != null) {
+                        final String finalHost = host;
+                        List<PeerData> knownPeers = Network.getInstance().getAllKnownPeers().stream()
+                                .filter(knownPeerData -> knownPeerData.getAddress().getHost().equals(finalHost))
+                                .collect(Collectors.toList());
+                        // Loop through each match and attempt a connection
+                        for (PeerData matchingPeer : knownPeers) {
+                            String matchingPeerAddress = matchingPeer.getAddress().toString();
+                            int matchingPeerPort = matchingPeer.getAddress().getPort();
+                            // Make sure that it's not a port we've already tried
+                            if (matchingPeerPort != port && matchingPeerPort != defaultPort) {
+                                success = Network.getInstance().requestDataFromPeer(matchingPeerAddress, signature);
+                                if (success) {
+                                    // Successfully connected, so stop making connections
+                                    break;
+                                }
+                            }
+                        }
+                    }
+                }
 
-            return success;
+                if (success) {
+                    // We were able to connect with a peer, so track the request
+                    ArbitraryDataFileListManager.getInstance().addToSignatureRequests(signature58, false, true);
+                }
 
-        } catch (DataException e) {
-            LOGGER.debug("Unable to fetch peer list from repository");
+            }
+        } catch (InterruptedException e) {
+            // Do nothing
         }
 
-        return false;
+        return success;
+    }
+
+
+    // Relays
+
+    private List<ArbitraryRelayInfo> getRelayInfoListForHash(String hash58) {
+        synchronized (arbitraryRelayMap) {
+            return arbitraryRelayMap.stream()
+                    .filter(relayInfo -> Objects.equals(relayInfo.getHash58(), hash58))
+                    .collect(Collectors.toList());
+        }
+    }
+
+    private ArbitraryRelayInfo getOptimalRelayInfoEntryForHash(String hash58) {
+        LOGGER.trace("Fetching relay info for hash: {}", hash58);
+        List<ArbitraryRelayInfo> relayInfoList = this.getRelayInfoListForHash(hash58);
+        if (relayInfoList != null && !relayInfoList.isEmpty()) {
+
+            // Remove any with null requestHops
+            relayInfoList.removeIf(r -> r.getRequestHops() == null);
+
+            // If list is now empty, then just return one at random
+            if (relayInfoList.isEmpty()) {
+                return this.getRandomRelayInfoEntryForHash(hash58);
+            }
+
+            // Sort by number of hops (lowest first)
+            relayInfoList.sort(Comparator.comparingInt(ArbitraryRelayInfo::getRequestHops));
+
+            // FUTURE: secondary sort by requestTime?
+
+            ArbitraryRelayInfo relayInfo = relayInfoList.get(0);
+
+            LOGGER.trace("Returning optimal relay info for hash: {} (requestHops {})", hash58, relayInfo.getRequestHops());
+            return relayInfo;
+        }
+        LOGGER.trace("No relay info exists for hash: {}", hash58);
+        return null;
+    }
+
+    private ArbitraryRelayInfo getRandomRelayInfoEntryForHash(String hash58) {
+        LOGGER.trace("Fetching random relay info for hash: {}", hash58);
+        List<ArbitraryRelayInfo> relayInfoList = this.getRelayInfoListForHash(hash58);
+        if (relayInfoList != null && !relayInfoList.isEmpty()) {
+
+            // Pick random item
+            int index = new SecureRandom().nextInt(relayInfoList.size());
+            LOGGER.trace("Returning random relay info for hash: {} (index {})", hash58, index);
+            return relayInfoList.get(index);
+        }
+        LOGGER.trace("No relay info exists for hash: {}", hash58);
+        return null;
+    }
+
+    public void addToRelayMap(ArbitraryRelayInfo newEntry) {
+        if (newEntry == null || !newEntry.isValid()) {
+            return;
+        }
+
+        // Remove existing entry for this peer if it exists, to renew the timestamp
+        this.removeFromRelayMap(newEntry);
+
+        // Re-add
+        arbitraryRelayMap.add(newEntry);
+        LOGGER.debug("Added entry to relay map: {}", newEntry);
+    }
+
+    private void removeFromRelayMap(ArbitraryRelayInfo entry) {
+        arbitraryRelayMap.removeIf(relayInfo -> relayInfo.equals(entry));
+    }
+
+
+    // Peers requesting QDN data from us
+
+    /**
+     * Add an address string of a peer that is trying to request data from us.
+     * @param peerAddress
+     */
+    public void addRecentDataRequest(String peerAddress) {
+        if (peerAddress == null) {
+            return;
+        }
+
+        Long now = NTP.getTime();
+        if (now == null) {
+            return;
+        }
+
+        // Make sure to remove the port, since it isn't guaranteed to match next time
+        String[] parts = peerAddress.split(":");
+        if (parts.length == 0) {
+            return;
+        }
+        String host = parts[0];
+        if (!InetAddresses.isInetAddress(host)) {
+            // Invalid host
+            return;
+        }
+
+        this.recentDataRequests.put(host, now);
+    }
+
+    public boolean isPeerRequestingData(String peerAddressWithoutPort) {
+        return this.recentDataRequests.containsKey(peerAddressWithoutPort);
+    }
+
+    public boolean hasPendingDataRequest() {
+        return !this.recentDataRequests.isEmpty();
     }
 
 
@@ -342,7 +558,7 @@ public class ArbitraryDataFileManager {
 
         try {
             ArbitraryDataFile arbitraryDataFile = ArbitraryDataFile.fromHash(hash, signature);
-            Triple<String, Peer, Long> relayInfo = this.arbitraryRelayMap.get(hash58);
+            ArbitraryRelayInfo relayInfo = this.getOptimalRelayInfoEntryForHash(hash58);
 
             if (arbitraryDataFile.exists()) {
                 LOGGER.trace("Hash {} exists", hash58);
@@ -359,15 +575,12 @@ public class ArbitraryDataFileManager {
             else if (relayInfo != null) {
                 LOGGER.debug("We have relay info for hash {}", Base58.encode(hash));
                 // We need to ask this peer for the file
-                Peer peerToAsk = relayInfo.getB();
+                Peer peerToAsk = relayInfo.getPeer();
                 if (peerToAsk != null) {
 
                     // Forward the message to this peer
                     LOGGER.debug("Asking peer {} for hash {}", peerToAsk, hash58);
                     this.fetchArbitraryDataFile(peerToAsk, peer, signature, hash, message);
-
-                    // Remove from the map regardless of outcome, as the relay attempt is now considered complete
-                    arbitraryRelayMap.remove(hash58);
                 }
                 else {
                     LOGGER.debug("Peer {} not found in relay info", peer);
@@ -382,9 +595,10 @@ public class ArbitraryDataFileManager {
                 // Send valid, yet unexpected message type in response, so peer's synchronizer doesn't have to wait for timeout
                 LOGGER.debug(String.format("Sending 'file unknown' response to peer %s for GET_FILE request for unknown file %s", peer, arbitraryDataFile));
 
-                // We'll send empty block summaries message as it's very short
-                // TODO: use a different message type here
-                Message fileUnknownMessage = new BlockSummariesMessage(Collections.emptyList());
+                // Send generic 'unknown' message as it's very short
+                Message fileUnknownMessage = peer.getPeersVersion() >= GenericUnknownMessage.MINIMUM_PEER_VERSION
+                        ? new GenericUnknownMessage()
+                        : new BlockSummariesMessage(Collections.emptyList());
                 fileUnknownMessage.setId(message.getId());
                 if (!peer.sendMessage(fileUnknownMessage)) {
                     LOGGER.debug("Couldn't sent file-unknown response");

src/main/java/org/qortal/controller/arbitrary/ArbitraryDataFileRequestThread.java

@@ -0,0 +1,124 @@
+package org.qortal.controller.arbitrary;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.qortal.controller.Controller;
+import org.qortal.data.arbitrary.ArbitraryFileListResponseInfo;
+import org.qortal.data.transaction.ArbitraryTransactionData;
+import org.qortal.network.Peer;
+import org.qortal.repository.DataException;
+import org.qortal.repository.Repository;
+import org.qortal.repository.RepositoryManager;
+import org.qortal.utils.ArbitraryTransactionUtils;
+import org.qortal.utils.Base58;
+import org.qortal.utils.NTP;
+
+import java.util.*;
+import java.util.stream.Collectors;
+
+public class ArbitraryDataFileRequestThread implements Runnable {
+
+    private static final Logger LOGGER = LogManager.getLogger(ArbitraryDataFileRequestThread.class);
+
+    public ArbitraryDataFileRequestThread() {
+
+    }
+
+    @Override
+    public void run() {
+        Thread.currentThread().setName("Arbitrary Data File Request Thread");
+
+        try {
+            while (!Controller.isStopping()) {
+                Long now = NTP.getTime();
+                this.processFileHashes(now);
+            }
+        } catch (InterruptedException e) {
+            // Fall-through to exit thread...
+        }
+    }
+
+    private void processFileHashes(Long now) throws InterruptedException {
+		if (Controller.isStopping()) {
+            return;
+        }
+
+        ArbitraryDataFileManager arbitraryDataFileManager = ArbitraryDataFileManager.getInstance();
+        String signature58 = null;
+        String hash58 = null;
+        Peer peer = null;
+        boolean shouldProcess = false;
+
+        synchronized (arbitraryDataFileManager.arbitraryDataFileHashResponses) {
+            if (!arbitraryDataFileManager.arbitraryDataFileHashResponses.isEmpty()) {
+
+                // Sort by lowest number of node hops first
+                Comparator<ArbitraryFileListResponseInfo> lowestHopsFirstComparator =
+                        Comparator.comparingInt(ArbitraryFileListResponseInfo::getRequestHops);
+                arbitraryDataFileManager.arbitraryDataFileHashResponses.sort(lowestHopsFirstComparator);
+
+                Iterator iterator = arbitraryDataFileManager.arbitraryDataFileHashResponses.iterator();
+                while (iterator.hasNext()) {
+                    if (Controller.isStopping()) {
+                        return;
+                    }
+
+                    ArbitraryFileListResponseInfo responseInfo = (ArbitraryFileListResponseInfo) iterator.next();
+                    if (responseInfo == null) {
+                        iterator.remove();
+                        continue;
+                    }
+
+                    hash58 = responseInfo.getHash58();
+                    peer = responseInfo.getPeer();
+                    signature58 = responseInfo.getSignature58();
+                    Long timestamp = responseInfo.getTimestamp();
+
+                    if (now - timestamp >= ArbitraryDataManager.ARBITRARY_RELAY_TIMEOUT || signature58 == null || peer == null) {
+                        // Ignore - to be deleted
+                        iterator.remove();
+                        continue;
+                    }
+
+                    // Skip if already requesting, but don't remove, as we might want to retry later
+                    if (arbitraryDataFileManager.arbitraryDataFileRequests.containsKey(hash58)) {
+                        // Already requesting - leave this attempt for later
+                        continue;
+                    }
+
+                    // We want to process this file
+                    shouldProcess = true;
+                    iterator.remove();
+                    break;
+                }
+            }
+        }
+
+        if (!shouldProcess) {
+            // Nothing to do
+            Thread.sleep(1000L);
+            return;
+        }
+
+        byte[] hash = Base58.decode(hash58);
+        byte[] signature = Base58.decode(signature58);
+
+        // Fetch the transaction data
+        try (final Repository repository = RepositoryManager.getRepository()) {
+            ArbitraryTransactionData arbitraryTransactionData = ArbitraryTransactionUtils.fetchTransactionData(repository, signature);
+            if (arbitraryTransactionData == null) {
+                return;
+            }
+
+            if (signature == null || hash == null || peer == null || arbitraryTransactionData == null) {
+                return;
+            }
+
+            LOGGER.debug("Fetching file {} from peer {} via request thread...", hash58, peer);
+            arbitraryDataFileManager.fetchArbitraryDataFiles(repository, peer, signature, arbitraryTransactionData, Arrays.asList(hash));
+
+        } catch (DataException e) {
+            LOGGER.debug("Unable to process file hashes: {}", e.getMessage());
+        }
+    }
+}

src/main/java/org/qortal/controller/arbitrary/ArbitraryDataManager.java

@@ -1,23 +1,24 @@
 package org.qortal.controller.arbitrary;
 
 import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
 import java.util.*;
-import java.util.stream.Collectors;
 
 import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
 import org.qortal.api.resource.TransactionsResource.ConfirmationStatus;
 import org.qortal.arbitrary.ArbitraryDataFile;
 import org.qortal.arbitrary.ArbitraryDataResource;
+import org.qortal.arbitrary.metadata.ArbitraryDataTransactionMetadata;
 import org.qortal.arbitrary.misc.Service;
 import org.qortal.controller.Controller;
-import org.qortal.data.network.ArbitraryPeerData;
 import org.qortal.data.transaction.ArbitraryTransactionData;
 import org.qortal.data.transaction.TransactionData;
 import org.qortal.list.ResourceListManager;
 import org.qortal.network.Network;
 import org.qortal.network.Peer;
-import org.qortal.network.message.*;
 import org.qortal.repository.DataException;
 import org.qortal.repository.Repository;
 import org.qortal.repository.RepositoryManager;
@@ -38,14 +39,26 @@ public class ArbitraryDataManager extends Thread {
 	private int powDifficulty = 14; // Must not be final, as unit tests need to reduce this value
 
 	/** Request timeout when transferring arbitrary data */
-	public static final long ARBITRARY_REQUEST_TIMEOUT = 10 * 1000L; // ms
+	public static final long ARBITRARY_REQUEST_TIMEOUT = 12 * 1000L; // ms
 
 	/** Maximum time to hold information about an in-progress relay */
-	public static final long ARBITRARY_RELAY_TIMEOUT = 30 * 1000L; // ms
+	public static final long ARBITRARY_RELAY_TIMEOUT = 60 * 1000L; // ms
+
+	/** Maximum time to hold direct peer connection information */
+	public static final long ARBITRARY_DIRECT_CONNECTION_INFO_TIMEOUT = 2 * 60 * 1000L; // ms
+
+	/** Maximum time to hold information about recent data requests that we can fulfil */
+	public static final long ARBITRARY_RECENT_DATA_REQUESTS_TIMEOUT = 2 * 60 * 1000L; // ms
 
 	/** Maximum number of hops that an arbitrary signatures request is allowed to make */
 	private static int ARBITRARY_SIGNATURES_REQUEST_MAX_HOPS = 3;
 
+	private long lastMetadataFetchTime = 0L;
+	private static long METADATA_FETCH_INTERVAL = 5 * 60 * 1000L;
+
+	private long lastDataFetchTime = 0L;
+	private static long DATA_FETCH_INTERVAL = 1 * 60 * 1000L;
+
 	private static ArbitraryDataManager instance;
 	private final Object peerDataLock = new Object();
 
@@ -79,7 +92,13 @@ public class ArbitraryDataManager extends Thread {
 	public void run() {
 		Thread.currentThread().setName("Arbitrary Data Manager");
 
+		// Create data directory in case it doesn't exist yet
+		this.createDataDirectory();
+
 		try {
+			// Wait for node to finish starting up and making connections
+			Thread.sleep(2 * 60 * 1000L);
+
 			while (!isStopping) {
 				Thread.sleep(2000);
 
@@ -89,7 +108,13 @@ public class ArbitraryDataManager extends Thread {
 					continue;
 				}
 
-				List<Peer> peers = Network.getInstance().getHandshakedPeers();
+				Long now = NTP.getTime();
+				if (now == null) {
+					continue;
+				}
+
+				// Needs a mutable copy of the unmodifiableList
+				List<Peer> peers = new ArrayList<>(Network.getInstance().getImmutableHandshakedPeers());
 
 				// Disregard peers that have "misbehaved" recently
 				peers.removeIf(Controller.hasMisbehaved);
@@ -99,6 +124,21 @@ public class ArbitraryDataManager extends Thread {
 					continue;
 				}
 
+				// Fetch metadata
+				if (NTP.getTime() - lastMetadataFetchTime >= METADATA_FETCH_INTERVAL) {
+					this.fetchAllMetadata();
+					lastMetadataFetchTime = NTP.getTime();
+				}
+
+				// Check if we need to fetch any data
+				if (NTP.getTime() - lastDataFetchTime < DATA_FETCH_INTERVAL) {
+					// Nothing to do yet
+					continue;
+				}
+
+				// In case the data directory has been deleted...
+				this.createDataDirectory();
+
 				// Fetch data according to storage policy
 				switch (Settings.getInstance().getStoragePolicy()) {
 					case FOLLOWED:
@@ -108,6 +148,7 @@ public class ArbitraryDataManager extends Thread {
 
 					case ALL:
 						this.processAll();
+						break;
 
 					case NONE:
 					case VIEWED:
@@ -116,6 +157,8 @@ public class ArbitraryDataManager extends Thread {
 						Thread.sleep(60000);
 						break;
 				}
+
+				lastDataFetchTime = NTP.getTime();
 			}
 		} catch (InterruptedException e) {
 			// Fall-through to exit thread...
@@ -127,7 +170,7 @@ public class ArbitraryDataManager extends Thread {
 		this.interrupt();
 	}
 
-	private void processNames() {
+	private void processNames() throws InterruptedException {
 		// Fetch latest list of followed names
 		List<String> followedNames = ResourceListManager.getInstance().getStringsInList("followedNames");
 		if (followedNames == null || followedNames.isEmpty()) {
@@ -140,11 +183,11 @@ public class ArbitraryDataManager extends Thread {
 		}
 	}
 
-	private void processAll() {
+	private void processAll() throws InterruptedException {
 		this.fetchAndProcessTransactions(null);
 	}
 
-	private void fetchAndProcessTransactions(String name) {
+	private void fetchAndProcessTransactions(String name) throws InterruptedException {
 		ArbitraryDataStorageManager storageManager = ArbitraryDataStorageManager.getInstance();
 
 		// Paginate queries when fetching arbitrary transactions
@@ -152,6 +195,7 @@ public class ArbitraryDataManager extends Thread {
 		int offset = 0;
 
 		while (!isStopping) {
+			Thread.sleep(1000L);
 
 			// Any arbitrary transactions we want to fetch data for?
 			try (final Repository repository = RepositoryManager.getRepository()) {
@@ -166,6 +210,7 @@ public class ArbitraryDataManager extends Thread {
 				// Loop through signatures and remove ones we don't need to process
 				Iterator iterator = signatures.iterator();
 				while (iterator.hasNext()) {
+					Thread.sleep(25L); // Reduce CPU usage
 					byte[] signature = (byte[]) iterator.next();
 
 					ArbitraryTransaction arbitraryTransaction = fetchTransaction(repository, signature);
@@ -222,6 +267,85 @@ public class ArbitraryDataManager extends Thread {
 		}
 	}
 
+	private void fetchAllMetadata() throws InterruptedException {
+		ArbitraryDataStorageManager storageManager = ArbitraryDataStorageManager.getInstance();
+
+		// Paginate queries when fetching arbitrary transactions
+		final int limit = 100;
+		int offset = 0;
+
+		while (!isStopping) {
+			Thread.sleep(1000L);
+
+			// Any arbitrary transactions we want to fetch data for?
+			try (final Repository repository = RepositoryManager.getRepository()) {
+				List<byte[]> signatures = repository.getTransactionRepository().getSignaturesMatchingCriteria(null, null, null, ARBITRARY_TX_TYPE, null, null, null, ConfirmationStatus.BOTH, limit, offset, true);
+				// LOGGER.trace("Found {} arbitrary transactions at offset: {}, limit: {}", signatures.size(), offset, limit);
+				if (signatures == null || signatures.isEmpty()) {
+					offset = 0;
+					break;
+				}
+				offset += limit;
+
+				// Loop through signatures and remove ones we don't need to process
+				Iterator iterator = signatures.iterator();
+				while (iterator.hasNext()) {
+					Thread.sleep(25L); // Reduce CPU usage
+					byte[] signature = (byte[]) iterator.next();
+
+					ArbitraryTransaction arbitraryTransaction = fetchTransaction(repository, signature);
+					if (arbitraryTransaction == null) {
+						// Best not to process this one
+						iterator.remove();
+						continue;
+					}
+					ArbitraryTransactionData arbitraryTransactionData = (ArbitraryTransactionData) arbitraryTransaction.getTransactionData();
+
+					// Skip transactions that are blocked
+					if (storageManager.isBlocked(arbitraryTransactionData)) {
+						iterator.remove();
+						continue;
+					}
+
+					// Remove transactions that we already have local data for
+					if (hasLocalMetadata(arbitraryTransaction)) {
+						iterator.remove();
+						continue;
+					}
+				}
+
+				if (signatures.isEmpty()) {
+					continue;
+				}
+
+				// Pick one at random
+				final int index = new Random().nextInt(signatures.size());
+				byte[] signature = signatures.get(index);
+
+				if (signature == null) {
+					continue;
+				}
+
+				// Check to see if we have had a more recent PUT
+				ArbitraryTransactionData arbitraryTransactionData = ArbitraryTransactionUtils.fetchTransactionData(repository, signature);
+				boolean hasMoreRecentPutTransaction = ArbitraryTransactionUtils.hasMoreRecentPutTransaction(repository, arbitraryTransactionData);
+				if (hasMoreRecentPutTransaction) {
+					// There is a more recent PUT transaction than the one we are currently processing.
+					// When a PUT is issued, it replaces any layers that would have been there before.
+					// Therefore any data relating to this older transaction is no longer needed and we
+					// shouldn't fetch it from the network.
+					continue;
+				}
+
+				// Ask our connected peers if they have metadata for this signature
+				fetchMetadata(arbitraryTransactionData);
+
+			} catch (DataException e) {
+				LOGGER.error("Repository issue when fetching arbitrary transaction data", e);
+			}
+		}
+	}
+
 	private ArbitraryTransaction fetchTransaction(final Repository repository, byte[] signature) {
 		try {
 			TransactionData transactionData = repository.getTransactionRepository().fromSignature(signature);
@@ -241,16 +365,48 @@ public class ArbitraryDataManager extends Thread {
 
 		} catch (DataException e) {
 			LOGGER.error("Repository issue when checking arbitrary transaction's data is local", e);
-			return true;
+			return true; // Assume true for now, to avoid network spam on error
 		}
 	}
 
+	private boolean hasLocalMetadata(ArbitraryTransaction arbitraryTransaction) {
+		try {
+			ArbitraryTransactionData arbitraryTransactionData = (ArbitraryTransactionData) arbitraryTransaction.getTransactionData();
+			byte[] signature = arbitraryTransactionData.getSignature();
+			byte[] metadataHash = arbitraryTransactionData.getMetadataHash();
+
+			if (metadataHash == null) {
+				// This transaction doesn't have metadata associated with it, so return true to indicate that we have everything
+				return true;
+			}
+
+			ArbitraryDataFile metadataFile = ArbitraryDataFile.fromHash(metadataHash, signature);
+
+			return metadataFile.exists();
+
+		} catch (DataException e) {
+			LOGGER.error("Repository issue when checking arbitrary transaction's metadata is local", e);
+			return true; // Assume true for now, to avoid network spam on error
+		}
+	}
 
 	// Entrypoint to request new data from peers
 	public boolean fetchData(ArbitraryTransactionData arbitraryTransactionData) {
 		return ArbitraryDataFileListManager.getInstance().fetchArbitraryDataFileList(arbitraryTransactionData);
 	}
 
+	// Entrypoint to request new metadata from peers
+	public ArbitraryDataTransactionMetadata fetchMetadata(ArbitraryTransactionData arbitraryTransactionData) {
+
+		ArbitraryDataResource resource = new ArbitraryDataResource(
+				arbitraryTransactionData.getName(),
+				ArbitraryDataFile.ResourceIdType.NAME,
+				arbitraryTransactionData.getService(),
+				arbitraryTransactionData.getIdentifier()
+		);
+		return ArbitraryMetadataManager.getInstance().fetchMetadata(resource, true);
+	}
+
 
 	// Useful methods used by other parts of the app
 
@@ -275,6 +431,9 @@ public class ArbitraryDataManager extends Thread {
 
 		// Cleanup file request caches
 		ArbitraryDataFileManager.getInstance().cleanupRequestCache(now);
+
+		// Clean up metadata request caches
+		ArbitraryMetadataManager.getInstance().cleanupRequestCache(now);
 	}
 
 	public boolean isResourceCached(ArbitraryDataResource resource) {
@@ -338,7 +497,7 @@ public class ArbitraryDataManager extends Thread {
 			ArbitraryDataResource resource =
 					new ArbitraryDataResource(resourceId, ArbitraryDataFile.ResourceIdType.NAME, service, identifier);
 			String key = resource.getUniqueKey();
-			LOGGER.info("Clearing cache for {}...", resource);
+			LOGGER.trace("Clearing cache for {}...", resource);
 
 			if (this.arbitraryDataCachedResources.containsKey(key)) {
 				this.arbitraryDataCachedResources.remove(key);
@@ -362,95 +521,19 @@ public class ArbitraryDataManager extends Thread {
 		}
 	}
 
-
-	// Broadcast list of hosted signatures
-
-	public void broadcastHostedSignatureList() {
-		try (final Repository repository = RepositoryManager.getRepository()) {
-			List<ArbitraryTransactionData> hostedTransactions = ArbitraryDataStorageManager.getInstance().listAllHostedTransactions(repository);
-			List<byte[]> hostedSignatures = hostedTransactions.stream().map(ArbitraryTransactionData::getSignature).collect(Collectors.toList());
-			if (!hostedSignatures.isEmpty()) {
-				// Broadcast the list, using null to represent our peer address
-				LOGGER.info("Broadcasting list of hosted signatures...");
-				Message arbitrarySignatureMessage = new ArbitrarySignaturesMessage(null, 0, hostedSignatures);
-				Network.getInstance().broadcast(broadcastPeer -> arbitrarySignatureMessage);
-			}
-		} catch (DataException e) {
-			LOGGER.error("Repository issue when fetching arbitrary transaction data for broadcast", e);
+	private boolean createDataDirectory() {
+		// Create the data directory if it doesn't exist
+		String dataPath = Settings.getInstance().getDataPath();
+		Path dataDirectory = Paths.get(dataPath);
+		try {
+			Files.createDirectories(dataDirectory);
+		} catch (IOException e) {
+			LOGGER.error("Unable to create data directory");
+			return false;
 		}
+		return true;
 	}
 
-
-	// Handle incoming arbitrary signatures messages
-
-	public void onNetworkArbitrarySignaturesMessage(Peer peer, Message message) {
-		// Don't process if QDN is disabled
-		if (!Settings.getInstance().isQdnEnabled()) {
-			return;
-		}
-
-		LOGGER.debug("Received arbitrary signature list from peer {}", peer);
-
-		ArbitrarySignaturesMessage arbitrarySignaturesMessage = (ArbitrarySignaturesMessage) message;
-		List<byte[]> signatures = arbitrarySignaturesMessage.getSignatures();
-
-		String peerAddress = peer.getPeerData().getAddress().toString();
-		if (arbitrarySignaturesMessage.getPeerAddress() != null && !arbitrarySignaturesMessage.getPeerAddress().isEmpty()) {
-			// This message is about a different peer than the one that sent it
-			peerAddress = arbitrarySignaturesMessage.getPeerAddress();
-		}
-
-		boolean containsNewEntry = false;
-
-		// Synchronize peer data lookups to make this process thread safe. Otherwise we could broadcast
-		// the same data multiple times, due to more than one thread processing the same message from different peers
-		synchronized (this.peerDataLock) {
-			try (final Repository repository = RepositoryManager.getRepository()) {
-				for (byte[] signature : signatures) {
-
-					// Check if a record already exists for this hash/host combination
-					// The port is not checked here - only the host/ip - in order to avoid duplicates
-					// from filling up the db due to dynamic/ephemeral ports
-					ArbitraryPeerData existingEntry = repository.getArbitraryRepository()
-							.getArbitraryPeerDataForSignatureAndHost(signature, peer.getPeerData().getAddress().getHost());
-
-					if (existingEntry == null) {
-						// We haven't got a record of this mapping yet, so add it
-						ArbitraryPeerData arbitraryPeerData = new ArbitraryPeerData(signature, peerAddress);
-						repository.discardChanges();
-						if (arbitraryPeerData.isPeerAddressValid()) {
-							LOGGER.debug("Adding arbitrary peer: {} for signature {}", peerAddress, Base58.encode(signature));
-							repository.getArbitraryRepository().save(arbitraryPeerData);
-							repository.saveChanges();
-
-							// Remember that this data is new, so that it can be rebroadcast later
-							containsNewEntry = true;
-						}
-					}
-				}
-
-				// If at least one signature in this batch was new to us, we should rebroadcast the message to the
-				// network in case some peers haven't received it yet
-				if (containsNewEntry) {
-					int requestHops = arbitrarySignaturesMessage.getRequestHops();
-					arbitrarySignaturesMessage.setRequestHops(++requestHops);
-					if (requestHops < ARBITRARY_SIGNATURES_REQUEST_MAX_HOPS) {
-						LOGGER.debug("Rebroadcasting arbitrary signature list for peer {}. requestHops: {}", peerAddress, requestHops);
-						Network.getInstance().broadcast(broadcastPeer -> broadcastPeer == peer ? null : arbitrarySignaturesMessage);
-					}
-				} else {
-					// Don't rebroadcast as otherwise we could get into a loop
-				}
-
-				// If anything needed saving, it would already have called saveChanges() above
-				repository.discardChanges();
-			} catch (DataException e) {
-				LOGGER.error(String.format("Repository issue while processing arbitrary transaction signature list from peer %s", peer), e);
-			}
-		}
-	}
-
-
 	public int getPowDifficulty() {
 		return this.powDifficulty;
 	}

src/main/java/org/qortal/controller/arbitrary/ArbitraryDataRenderManager.java

@@ -32,7 +32,7 @@ public class ArbitraryDataRenderManager extends Thread {
 
     @Override
     public void run() {
-        Thread.currentThread().setName("Arbitrary Data Manager");
+        Thread.currentThread().setName("Arbitrary Data Render Manager");
 
         try {
             while (!isStopping) {

src/main/java/org/qortal/controller/arbitrary/ArbitraryDataStorageManager.java

@@ -10,15 +10,18 @@ import org.qortal.repository.DataException;
 import org.qortal.repository.Repository;
 import org.qortal.settings.Settings;
 import org.qortal.transaction.Transaction;
+import org.qortal.utils.ArbitraryTransactionUtils;
 import org.qortal.utils.Base58;
 import org.qortal.utils.FilesystemUtils;
 import org.qortal.utils.NTP;
 
 import java.io.IOException;
+import java.io.UncheckedIOException;
 import java.nio.file.Files;
 import java.nio.file.Path;
 import java.nio.file.Paths;
 import java.util.ArrayList;
+import java.util.Comparator;
 import java.util.List;
 import java.util.Objects;
 import java.util.stream.Collectors;
@@ -44,6 +47,9 @@ public class ArbitraryDataStorageManager extends Thread {
 
     private List<ArbitraryTransactionData> hostedTransactions;
 
+    private String searchQuery;
+    private List<ArbitraryTransactionData> searchResultsTransactions;
+
     private static final long DIRECTORY_SIZE_CHECK_INTERVAL = 10 * 60 * 1000L; // 10 minutes
 
     /** Treat storage as full at 90% usage, to reduce risk of going over the limit.
@@ -223,6 +229,16 @@ public class ArbitraryDataStorageManager extends Thread {
         }
     }
 
+    /**
+     * Check if data relating to a transaction is blocked by this node.
+     *
+     * @param arbitraryTransactionData - the transaction
+     * @return boolean - whether the resource is blocked or not
+     */
+    public boolean isBlocked(ArbitraryTransactionData arbitraryTransactionData) {
+        return isNameBlocked(arbitraryTransactionData.getName());
+    }
+
     private boolean isDataTypeAllowed(ArbitraryTransactionData arbitraryTransactionData) {
         byte[] secret = arbitraryTransactionData.getSecret();
         boolean hasSecret = (secret != null && secret.length == 32);
@@ -255,14 +271,8 @@ public class ArbitraryDataStorageManager extends Thread {
     }
 
 
-    // Hosted data
-
-    public List<ArbitraryTransactionData> listAllHostedTransactions(Repository repository) {
-        // Load from cache if we can, to avoid disk reads
-        if (this.hostedTransactions != null) {
-            return this.hostedTransactions;
-        }
-
+    public List<ArbitraryTransactionData> loadAllHostedTransactions(Repository repository) {
+        
         List<ArbitraryTransactionData> arbitraryTransactionDataList = new ArrayList<>();
 
         // Find all hosted paths
@@ -283,18 +293,94 @@ public class ArbitraryDataStorageManager extends Thread {
                 if (transactionData == null || transactionData.getType() != Transaction.TransactionType.ARBITRARY) {
                     continue;
                 }
-                arbitraryTransactionDataList.add((ArbitraryTransactionData) transactionData);
+                ArbitraryTransactionData arbitraryTransactionData = (ArbitraryTransactionData) transactionData;
+
+                // Make sure to exclude metadata-only resources
+                if (arbitraryTransactionData.getMetadataHash() != null) {
+                    if (contents.length == 1) {
+                        String metadataHash58 = Base58.encode(arbitraryTransactionData.getMetadataHash());
+                        if (Objects.equals(metadataHash58, contents[0])) {
+                            // We only have the metadata file for this resource, not the actual data, so exclude it
+                            continue;
+                        }
+                    }
+                }
+
+                // Found some data matching a transaction, so add it to the list
+                arbitraryTransactionDataList.add(arbitraryTransactionData);
 
             } catch (DataException e) {
                 continue;
             }
         }
 
-        // Update cache
-        this.hostedTransactions = arbitraryTransactionDataList;
+        // Sort by newest first
+        arbitraryTransactionDataList.sort(Comparator.comparingLong(ArbitraryTransactionData::getTimestamp).reversed());
 
         return arbitraryTransactionDataList;
     }
+    // Hosted data
+
+    public List<ArbitraryTransactionData> listAllHostedTransactions(Repository repository, Integer limit, Integer offset) {
+        // Load from cache if we can, to avoid disk reads
+
+        if (this.hostedTransactions != null) {
+            return ArbitraryTransactionUtils.limitOffsetTransactions(this.hostedTransactions, limit, offset);
+        }
+
+        this.hostedTransactions = this.loadAllHostedTransactions(repository);
+
+        return ArbitraryTransactionUtils.limitOffsetTransactions(this.hostedTransactions, limit, offset);
+    }
+    
+    /**
+     * searchHostedTransactions
+     * Allow to run a query against hosted data names and return matches if there are any
+     * @param repository
+     * @param query
+     * @param limit
+     * @param offset
+     * @return
+     */
+
+    public List<ArbitraryTransactionData> searchHostedTransactions(Repository repository, String query, Integer limit, Integer offset) {
+        // Load from results cache if we can (results that exists for the same query), to avoid disk reads
+        if (this.searchResultsTransactions != null && this.searchQuery.equals(query.toLowerCase())) {
+            return ArbitraryTransactionUtils.limitOffsetTransactions(this.searchResultsTransactions, limit, offset);
+        }
+
+        // Using cache if we can, to avoid disk reads
+        if (this.hostedTransactions == null) {
+            this.hostedTransactions = this.loadAllHostedTransactions(repository);
+        }
+
+        this.searchQuery = query.toLowerCase(); //set the searchQuery so that it can be checked on the next call
+
+        List<ArbitraryTransactionData> searchResultsList = new ArrayList<>();
+
+        // Loop through cached hostedTransactions
+        for (ArbitraryTransactionData atd : this.hostedTransactions) {
+            try {
+               if (atd.getName() != null && atd.getName().toLowerCase().contains(this.searchQuery)) {
+                   searchResultsList.add(atd);
+               }
+               else if (atd.getIdentifier() != null && atd.getIdentifier().toLowerCase().contains(this.searchQuery)) {
+                   searchResultsList.add(atd);
+               }
+
+            } catch (Exception e) {
+                continue;
+            }
+        }
+
+        // Sort by newest first
+        searchResultsList.sort(Comparator.comparingLong(ArbitraryTransactionData::getTimestamp).reversed());
+
+        // Update cache
+        this.searchResultsTransactions = searchResultsList;
+
+        return ArbitraryTransactionUtils.limitOffsetTransactions(this.searchResultsTransactions, limit, offset);
+    }
 
     /**
      * Warning: this method will walk through the entire data directory
@@ -317,7 +403,7 @@ public class ArbitraryDataStorageManager extends Thread {
                             && path.getFileName().toString().length() > 32)
                     .collect(Collectors.toList());
         }
-        catch (IOException e) {
+        catch (IOException | UncheckedIOException e) {
             LOGGER.info("Unable to walk through hosted data: {}", e.getMessage());
         }
 
@@ -446,7 +532,7 @@ public class ArbitraryDataStorageManager extends Thread {
         long maxStoragePerName = this.storageCapacityPerName(threshold);
 
         // Fetch all hosted transactions
-        List<ArbitraryTransactionData> hostedTransactions = this.listAllHostedTransactions(repository);
+        List<ArbitraryTransactionData> hostedTransactions = this.listAllHostedTransactions(repository, null, null);
         for (ArbitraryTransactionData transactionData : hostedTransactions) {
             String transactionName = transactionData.getName();
             if (!Objects.equals(name, transactionName)) {

src/main/java/org/qortal/controller/arbitrary/ArbitraryMetadataManager.java

@@ -0,0 +1,456 @@
+package org.qortal.controller.arbitrary;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.qortal.arbitrary.ArbitraryDataFile;
+import org.qortal.arbitrary.ArbitraryDataResource;
+import org.qortal.arbitrary.metadata.ArbitraryDataTransactionMetadata;
+import org.qortal.controller.Controller;
+import org.qortal.data.transaction.ArbitraryTransactionData;
+import org.qortal.data.transaction.TransactionData;
+import org.qortal.network.Network;
+import org.qortal.network.Peer;
+import org.qortal.network.message.*;
+import org.qortal.repository.DataException;
+import org.qortal.repository.Repository;
+import org.qortal.repository.RepositoryManager;
+import org.qortal.settings.Settings;
+import org.qortal.utils.Base58;
+import org.qortal.utils.NTP;
+import org.qortal.utils.Triple;
+
+import java.io.IOException;
+import java.util.*;
+
+import static org.qortal.controller.arbitrary.ArbitraryDataFileListManager.*;
+
+public class ArbitraryMetadataManager {
+
+    private static final Logger LOGGER = LogManager.getLogger(ArbitraryMetadataManager.class);
+
+    private static ArbitraryMetadataManager instance;
+
+    /**
+     * Map of recent incoming requests for ARBITRARY transaction metadata.
+     * <p>
+     * Key is original request's message ID<br>
+     * Value is Triple&lt;transaction signature in base58, first requesting peer, first request's timestamp&gt;
+     * <p>
+     * If peer is null then either:<br>
+     * <ul>
+     * <li>we are the original requesting peer</li>
+     * <li>we have already sent data payload to original requesting peer.</li>
+     * </ul>
+     * If signature is null then we have already received the file list and either:<br>
+     * <ul>
+     * <li>we are the original requesting peer and have processed it</li>
+     * <li>we have forwarded the metadata</li>
+     * </ul>
+     */
+    public Map<Integer, Triple<String, Peer, Long>> arbitraryMetadataRequests = Collections.synchronizedMap(new HashMap<>());
+
+    /**
+     * Map to keep track of in progress arbitrary metadata requests
+     * Key: string - the signature encoded in base58
+     * Value: Triple<networkBroadcastCount, directPeerRequestCount, lastAttemptTimestamp>
+     */
+    private Map<String, Triple<Integer, Integer, Long>> arbitraryMetadataSignatureRequests = Collections.synchronizedMap(new HashMap<>());
+
+
+    private ArbitraryMetadataManager() {
+    }
+
+    public static ArbitraryMetadataManager getInstance() {
+        if (instance == null)
+            instance = new ArbitraryMetadataManager();
+
+        return instance;
+    }
+
+    public void cleanupRequestCache(Long now) {
+        if (now == null) {
+            return;
+        }
+        final long requestMinimumTimestamp = now - ArbitraryDataManager.ARBITRARY_REQUEST_TIMEOUT;
+        arbitraryMetadataRequests.entrySet().removeIf(entry -> entry.getValue().getC() == null || entry.getValue().getC() < requestMinimumTimestamp);
+    }
+
+
+    public ArbitraryDataTransactionMetadata fetchMetadata(ArbitraryDataResource arbitraryDataResource, boolean useRateLimiter) {
+        try (final Repository repository = RepositoryManager.getRepository()) {
+            // Find latest transaction
+            ArbitraryTransactionData latestTransaction = repository.getArbitraryRepository()
+                    .getLatestTransaction(arbitraryDataResource.getResourceId(), arbitraryDataResource.getService(),
+                            null, arbitraryDataResource.getIdentifier());
+
+            if (latestTransaction != null) {
+                byte[] signature = latestTransaction.getSignature();
+                byte[] metadataHash = latestTransaction.getMetadataHash();
+                if (metadataHash == null) {
+                    // This resource doesn't have metadata
+                    throw new IllegalArgumentException("This resource doesn't have metadata");
+                }
+
+                ArbitraryDataFile metadataFile = ArbitraryDataFile.fromHash(metadataHash, signature);
+                if (!metadataFile.exists()) {
+                    // Request from network
+                    this.fetchArbitraryMetadata(latestTransaction, useRateLimiter);
+                }
+
+                // Now check again as it may have been downloaded above
+                if (metadataFile.exists()) {
+                    // Use local copy
+                    ArbitraryDataTransactionMetadata transactionMetadata = new ArbitraryDataTransactionMetadata(metadataFile.getFilePath());
+                    transactionMetadata.read();
+                    return transactionMetadata;
+                }
+            }
+
+        } catch (DataException | IOException e) {
+            LOGGER.error("Repository issue when fetching arbitrary transaction metadata", e);
+        }
+
+        return null;
+    }
+
+
+    // Request metadata from network
+
+    public byte[] fetchArbitraryMetadata(ArbitraryTransactionData arbitraryTransactionData, boolean useRateLimiter) {
+        byte[] metadataHash = arbitraryTransactionData.getMetadataHash();
+        if (metadataHash == null) {
+            return null;
+        }
+
+        byte[] signature = arbitraryTransactionData.getSignature();
+        String signature58 = Base58.encode(signature);
+
+        // Require an NTP sync
+        Long now = NTP.getTime();
+        if (now == null) {
+            return null;
+        }
+
+        // If we've already tried too many times in a short space of time, make sure to give up
+        if (useRateLimiter && !this.shouldMakeMetadataRequestForSignature(signature58)) {
+            LOGGER.trace("Skipping metadata request for signature {} due to rate limit", signature58);
+            return null;
+        }
+        this.addToSignatureRequests(signature58, true, false);
+
+        List<Peer> handshakedPeers = Network.getInstance().getImmutableHandshakedPeers();
+        LOGGER.debug(String.format("Sending metadata request for signature %s to %d peers...", signature58, handshakedPeers.size()));
+
+        // Build request
+        Message getArbitraryMetadataMessage = new GetArbitraryMetadataMessage(signature, now, 0);
+
+        // Save our request into requests map
+        Triple<String, Peer, Long> requestEntry = new Triple<>(signature58, null, NTP.getTime());
+
+        // Assign random ID to this message
+        int id;
+        do {
+            id = new Random().nextInt(Integer.MAX_VALUE - 1) + 1;
+
+            // Put queue into map (keyed by message ID) so we can poll for a response
+            // If putIfAbsent() doesn't return null, then this ID is already taken
+        } while (arbitraryMetadataRequests.put(id, requestEntry) != null);
+        getArbitraryMetadataMessage.setId(id);
+
+        // Broadcast request
+        Network.getInstance().broadcast(peer -> getArbitraryMetadataMessage);
+
+        // Poll to see if data has arrived
+        final long singleWait = 100;
+        long totalWait = 0;
+        while (totalWait < ArbitraryDataManager.ARBITRARY_REQUEST_TIMEOUT) {
+            try {
+                Thread.sleep(singleWait);
+            } catch (InterruptedException e) {
+                break;
+            }
+
+            requestEntry = arbitraryMetadataRequests.get(id);
+            if (requestEntry == null)
+                return null;
+
+            if (requestEntry.getA() == null)
+                break;
+
+            totalWait += singleWait;
+        }
+
+        try {
+            ArbitraryDataFile metadataFile = ArbitraryDataFile.fromHash(metadataHash, signature);
+            if (metadataFile.exists()) {
+                return metadataFile.getBytes();
+            }
+        } catch (DataException e) {
+            // Do nothing
+        }
+
+        return null;
+    }
+
+
+    // Track metadata lookups by signature
+
+    private boolean shouldMakeMetadataRequestForSignature(String signature58) {
+        Triple<Integer, Integer, Long> request = arbitraryMetadataSignatureRequests.get(signature58);
+
+        if (request == null) {
+            // Not attempted yet
+            return true;
+        }
+
+        // Extract the components
+        Integer networkBroadcastCount = request.getA();
+        // Integer directPeerRequestCount = request.getB();
+        Long lastAttemptTimestamp = request.getC();
+
+        if (lastAttemptTimestamp == null) {
+            // Not attempted yet
+            return true;
+        }
+
+        long timeSinceLastAttempt = NTP.getTime() - lastAttemptTimestamp;
+
+        // Allow a second attempt after 60 seconds
+        if (timeSinceLastAttempt > 60 * 1000L) {
+            // We haven't tried for at least 60 seconds
+
+            if (networkBroadcastCount < 2) {
+                // We've made less than 2 total attempts
+                return true;
+            }
+        }
+
+        // Then allow another attempt after 60 minutes
+        if (timeSinceLastAttempt > 60 * 60 * 1000L) {
+            // We haven't tried for at least 60 minutes
+
+            if (networkBroadcastCount < 3) {
+                // We've made less than 3 total attempts
+                return true;
+            }
+        }
+
+        return false;
+    }
+
+    public boolean isSignatureRateLimited(byte[] signature) {
+        String signature58 = Base58.encode(signature);
+        return !this.shouldMakeMetadataRequestForSignature(signature58);
+    }
+
+    public long lastRequestForSignature(byte[] signature) {
+        String signature58 = Base58.encode(signature);
+        Triple<Integer, Integer, Long> request = arbitraryMetadataSignatureRequests.get(signature58);
+
+        if (request == null) {
+            // Not attempted yet
+            return 0;
+        }
+
+        // Extract the components
+        Long lastAttemptTimestamp = request.getC();
+        if (lastAttemptTimestamp != null) {
+            return  lastAttemptTimestamp;
+        }
+        return 0;
+    }
+
+    public void addToSignatureRequests(String signature58, boolean incrementNetworkRequests, boolean incrementPeerRequests) {
+        Triple<Integer, Integer, Long> request  = arbitraryMetadataSignatureRequests.get(signature58);
+        Long now = NTP.getTime();
+
+        if (request == null) {
+            // No entry yet
+            Triple<Integer, Integer, Long> newRequest = new Triple<>(0, 0, now);
+            arbitraryMetadataSignatureRequests.put(signature58, newRequest);
+        }
+        else {
+            // There is an existing entry
+            if (incrementNetworkRequests) {
+                request.setA(request.getA() + 1);
+            }
+            if (incrementPeerRequests) {
+                request.setB(request.getB() + 1);
+            }
+            request.setC(now);
+            arbitraryMetadataSignatureRequests.put(signature58, request);
+        }
+    }
+
+    public void removeFromSignatureRequests(String signature58) {
+        arbitraryMetadataSignatureRequests.remove(signature58);
+    }
+
+
+    // Network handlers
+
+    public void onNetworkArbitraryMetadataMessage(Peer peer, Message message) {
+        // Don't process if QDN is disabled
+        if (!Settings.getInstance().isQdnEnabled()) {
+            return;
+        }
+
+        ArbitraryMetadataMessage arbitraryMetadataMessage = (ArbitraryMetadataMessage) message;
+        LOGGER.debug("Received metadata from peer {}", peer);
+
+        // Do we have a pending request for this data?
+        Triple<String, Peer, Long> request = arbitraryMetadataRequests.get(message.getId());
+        if (request == null || request.getA() == null) {
+            return;
+        }
+        boolean isRelayRequest = (request.getB() != null);
+
+        // Does this message's signature match what we're expecting?
+        byte[] signature = arbitraryMetadataMessage.getSignature();
+        String signature58 = Base58.encode(signature);
+        if (!request.getA().equals(signature58)) {
+            return;
+        }
+
+        // Update requests map to reflect that we've received all chunks
+        Triple<String, Peer, Long> newEntry = new Triple<>(null, null, request.getC());
+        arbitraryMetadataRequests.put(message.getId(), newEntry);
+
+        ArbitraryTransactionData arbitraryTransactionData = null;
+
+        // Forwarding
+        if (isRelayRequest && Settings.getInstance().isRelayModeEnabled()) {
+
+            // Get transaction info
+            try (final Repository repository = RepositoryManager.getRepository()) {
+                TransactionData transactionData = repository.getTransactionRepository().fromSignature(signature);
+                if (!(transactionData instanceof ArbitraryTransactionData))
+                    return;
+                arbitraryTransactionData = (ArbitraryTransactionData) transactionData;
+            } catch (DataException e) {
+                LOGGER.error(String.format("Repository issue while finding arbitrary transaction metadata for peer %s", peer), e);
+            }
+
+            // Check if the name is blocked
+            boolean isBlocked = (arbitraryTransactionData == null || ArbitraryDataStorageManager.getInstance().isNameBlocked(arbitraryTransactionData.getName()));
+            if (!isBlocked) {
+                Peer requestingPeer = request.getB();
+                if (requestingPeer != null) {
+
+                    ArbitraryMetadataMessage forwardArbitraryMetadataMessage = new ArbitraryMetadataMessage(signature, arbitraryMetadataMessage.getArbitraryMetadataFile());
+                    forwardArbitraryMetadataMessage.setId(arbitraryMetadataMessage.getId());
+
+                    // Forward to requesting peer
+                    LOGGER.debug("Forwarding metadata to requesting peer: {}", requestingPeer);
+                    if (!requestingPeer.sendMessage(forwardArbitraryMetadataMessage)) {
+                        requestingPeer.disconnect("failed to forward arbitrary metadata");
+                    }
+                }
+            }
+        }
+    }
+
+    public void onNetworkGetArbitraryMetadataMessage(Peer peer, Message message) {
+        // Don't respond if QDN is disabled
+        if (!Settings.getInstance().isQdnEnabled()) {
+            return;
+        }
+
+        Controller.getInstance().stats.getArbitraryMetadataMessageStats.requests.incrementAndGet();
+
+        GetArbitraryMetadataMessage getArbitraryMetadataMessage = (GetArbitraryMetadataMessage) message;
+        byte[] signature = getArbitraryMetadataMessage.getSignature();
+        String signature58 = Base58.encode(signature);
+        Long now = NTP.getTime();
+        Triple<String, Peer, Long> newEntry = new Triple<>(signature58, peer, now);
+
+        // If we've seen this request recently, then ignore
+        if (arbitraryMetadataRequests.putIfAbsent(message.getId(), newEntry) != null) {
+            LOGGER.debug("Ignoring metadata request from peer {} for signature {}", peer, signature58);
+            return;
+        }
+
+        LOGGER.debug("Received metadata request from peer {} for signature {}", peer, signature58);
+
+        ArbitraryTransactionData transactionData = null;
+        ArbitraryDataFile metadataFile = null;
+
+        try (final Repository repository = RepositoryManager.getRepository()) {
+
+            // Firstly we need to lookup this file on chain to get its metadata hash
+            transactionData = (ArbitraryTransactionData)repository.getTransactionRepository().fromSignature(signature);
+            if (transactionData instanceof ArbitraryTransactionData) {
+
+                // Check if we're even allowed to serve metadata for this transaction
+                if (ArbitraryDataStorageManager.getInstance().canStoreData(transactionData)) {
+
+                    byte[] metadataHash = transactionData.getMetadataHash();
+                    if (metadataHash != null) {
+
+                        // Load metadata file
+                        metadataFile = ArbitraryDataFile.fromHash(metadataHash, signature);
+                    }
+                }
+            }
+
+        } catch (DataException e) {
+            LOGGER.error(String.format("Repository issue while fetching arbitrary metadata for peer %s", peer), e);
+        }
+
+        // We should only respond if we have the metadata file
+        if (metadataFile != null && metadataFile.exists()) {
+
+            // We have the metadata file, so update requests map to reflect that we've sent it
+            newEntry = new Triple<>(null, null, now);
+            arbitraryMetadataRequests.put(message.getId(), newEntry);
+
+            ArbitraryMetadataMessage arbitraryMetadataMessage = new ArbitraryMetadataMessage(signature, metadataFile);
+            arbitraryMetadataMessage.setId(message.getId());
+            if (!peer.sendMessage(arbitraryMetadataMessage)) {
+                LOGGER.debug("Couldn't send metadata");
+                peer.disconnect("failed to send metadata");
+                return;
+            }
+            LOGGER.debug("Sent metadata");
+
+            // Nothing left to do, so return to prevent any unnecessary forwarding from occurring
+            LOGGER.debug("No need for any forwarding because metadata request is fully served");
+            return;
+
+        }
+
+        // We may need to forward this request on
+        boolean isBlocked = (transactionData == null || ArbitraryDataStorageManager.getInstance().isNameBlocked(transactionData.getName()));
+        if (Settings.getInstance().isRelayModeEnabled() && !isBlocked) {
+            // In relay mode - so ask our other peers if they have it
+
+            long requestTime = getArbitraryMetadataMessage.getRequestTime();
+            int requestHops = getArbitraryMetadataMessage.getRequestHops() + 1;
+            long totalRequestTime = now - requestTime;
+
+            if (totalRequestTime < RELAY_REQUEST_MAX_DURATION) {
+                // Relay request hasn't timed out yet, so can potentially be rebroadcast
+                if (requestHops < RELAY_REQUEST_MAX_HOPS) {
+                    // Relay request hasn't reached the maximum number of hops yet, so can be rebroadcast
+
+                    Message relayGetArbitraryMetadataMessage = new GetArbitraryMetadataMessage(signature, requestTime, requestHops);
+                    relayGetArbitraryMetadataMessage.setId(message.getId());
+
+                    LOGGER.debug("Rebroadcasting metadata request from peer {} for signature {} to our other peers... totalRequestTime: {}, requestHops: {}", peer, Base58.encode(signature), totalRequestTime, requestHops);
+                    Network.getInstance().broadcast(
+                            broadcastPeer ->
+                                    !broadcastPeer.isAtLeastVersion(RELAY_MIN_PEER_VERSION) ? null :
+                                    broadcastPeer == peer || Objects.equals(broadcastPeer.getPeerData().getAddress().getHost(), peer.getPeerData().getAddress().getHost()) ? null : relayGetArbitraryMetadataMessage);
+
+                }
+                else {
+                    // This relay request has reached the maximum number of allowed hops
+                }
+            }
+            else {
+                // This relay request has timed out
+            }
+        }
+    }
+
+}

src/main/java/org/qortal/controller/repository/AtStatesPruner.java

@@ -3,6 +3,7 @@ package org.qortal.controller.repository;
 import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
 import org.qortal.controller.Controller;
+import org.qortal.controller.Synchronizer;
 import org.qortal.data.block.BlockData;
 import org.qortal.repository.DataException;
 import org.qortal.repository.Repository;
@@ -18,6 +19,11 @@ public class AtStatesPruner implements Runnable {
 	public void run() {
 		Thread.currentThread().setName("AT States pruner");
 
+		if (Settings.getInstance().isLite()) {
+			// Nothing to prune in lite mode
+			return;
+		}
+
 		boolean archiveMode = false;
 		if (!Settings.getInstance().isTopOnly()) {
 			// Top-only mode isn't enabled, but we might want to prune for the purposes of archiving
@@ -47,7 +53,7 @@ public class AtStatesPruner implements Runnable {
 					continue;
 
 				// Don't even attempt if we're mid-sync as our repository requests will be delayed for ages
-				if (Controller.getInstance().isSynchronizing())
+				if (Synchronizer.getInstance().isSynchronizing())
 					continue;
 
 				// Prune AT states for all blocks up until our latest minus pruneBlockLimit

src/main/java/org/qortal/controller/repository/AtStatesTrimmer.java

@@ -3,6 +3,7 @@ package org.qortal.controller.repository;
 import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
 import org.qortal.controller.Controller;
+import org.qortal.controller.Synchronizer;
 import org.qortal.data.block.BlockData;
 import org.qortal.repository.DataException;
 import org.qortal.repository.Repository;
@@ -18,6 +19,11 @@ public class AtStatesTrimmer implements Runnable {
 	public void run() {
 		Thread.currentThread().setName("AT States trimmer");
 
+		if (Settings.getInstance().isLite()) {
+			// Nothing to trim in lite mode
+			return;
+		}
+
 		try (final Repository repository = RepositoryManager.getRepository()) {
 			int trimStartHeight = repository.getATRepository().getAtTrimHeight();
 
@@ -34,7 +40,7 @@ public class AtStatesTrimmer implements Runnable {
 					continue;
 
 				// Don't even attempt if we're mid-sync as our repository requests will be delayed for ages
-				if (Controller.getInstance().isSynchronizing())
+				if (Synchronizer.getInstance().isSynchronizing())
 					continue;
 
 				long currentTrimmableTimestamp = NTP.getTime() - Settings.getInstance().getAtStatesMaxLifetime();

src/main/java/org/qortal/controller/repository/BlockArchiver.java

@@ -3,6 +3,7 @@ package org.qortal.controller.repository;
 import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
 import org.qortal.controller.Controller;
+import org.qortal.controller.Synchronizer;
 import org.qortal.data.block.BlockData;
 import org.qortal.repository.*;
 import org.qortal.settings.Settings;
@@ -20,7 +21,7 @@ public class BlockArchiver implements Runnable {
 	public void run() {
 		Thread.currentThread().setName("Block archiver");
 
-		if (!Settings.getInstance().isArchiveEnabled()) {
+		if (!Settings.getInstance().isArchiveEnabled() || Settings.getInstance().isLite()) {
 			return;
 		}
 
@@ -51,7 +52,7 @@ public class BlockArchiver implements Runnable {
 				}
 
 				// Don't even attempt if we're mid-sync as our repository requests will be delayed for ages
-				if (Controller.getInstance().isSynchronizing()) {
+				if (Synchronizer.getInstance().isSynchronizing()) {
 					continue;
 				}
 

src/main/java/org/qortal/controller/repository/BlockPruner.java

@@ -3,6 +3,7 @@ package org.qortal.controller.repository;
 import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
 import org.qortal.controller.Controller;
+import org.qortal.controller.Synchronizer;
 import org.qortal.data.block.BlockData;
 import org.qortal.repository.DataException;
 import org.qortal.repository.Repository;
@@ -18,6 +19,11 @@ public class BlockPruner implements Runnable {
 	public void run() {
 		Thread.currentThread().setName("Block pruner");
 
+		if (Settings.getInstance().isLite()) {
+			// Nothing to prune in lite mode
+			return;
+		}
+
 		boolean archiveMode = false;
 		if (!Settings.getInstance().isTopOnly()) {
 			// Top-only mode isn't enabled, but we might want to prune for the purposes of archiving
@@ -51,7 +57,7 @@ public class BlockPruner implements Runnable {
 					continue;
 
 				// Don't even attempt if we're mid-sync as our repository requests will be delayed for ages
-				if (Controller.getInstance().isSynchronizing()) {
+				if (Synchronizer.getInstance().isSynchronizing()) {
 					continue;
 				}
 

src/main/java/org/qortal/controller/repository/NamesDatabaseIntegrityCheck.java

@@ -29,6 +29,15 @@ public class NamesDatabaseIntegrityCheck {
     private List<TransactionData> nameTransactions = new ArrayList<>();
 
     public int rebuildName(String name, Repository repository) {
+        return this.rebuildName(name, repository, null);
+    }
+
+    public int rebuildName(String name, Repository repository, List<String> referenceNames) {
+        // "referenceNames" tracks the linked names that have already been rebuilt, to prevent circular dependencies
+        if (referenceNames == null) {
+            referenceNames = new ArrayList<>();
+        }
+
         int modificationCount = 0;
         try {
             List<TransactionData> transactions = this.fetchAllTransactionsInvolvingName(name, repository);
@@ -56,7 +65,14 @@ public class NamesDatabaseIntegrityCheck {
                     if (Objects.equals(updateNameTransactionData.getNewName(), name) &&
                             !Objects.equals(updateNameTransactionData.getName(), updateNameTransactionData.getNewName())) {
                         // This renames an existing name, so we need to process that instead
-                        this.rebuildName(updateNameTransactionData.getName(), repository);
+
+                        if (!referenceNames.contains(name)) {
+                            referenceNames.add(name);
+                            this.rebuildName(updateNameTransactionData.getName(), repository, referenceNames);
+                        }
+                        else {
+                            // We've already processed this name so there's nothing more to do
+                        }
                     }
                     else {
                         Name nameObj = new Name(repository, name);
@@ -91,7 +107,7 @@ public class NamesDatabaseIntegrityCheck {
                     BuyNameTransactionData buyNameTransactionData = (BuyNameTransactionData) currentTransaction;
                     Name nameObj = new Name(repository, buyNameTransactionData.getName());
                     if (nameObj != null && nameObj.getNameData() != null) {
-                        nameObj.buy(buyNameTransactionData);
+                        nameObj.buy(buyNameTransactionData, false);
                         modificationCount++;
                         LOGGER.trace("Processed BUY_NAME transaction for name {}", name);
                     }
@@ -193,7 +209,12 @@ public class NamesDatabaseIntegrityCheck {
                                 newName = registeredName;
                             }
                             NameData newNameData = repository.getNameRepository().fromName(newName);
-                            if (!Objects.equals(creator.getAddress(), newNameData.getOwner())) {
+                            if (newNameData == null) {
+                                LOGGER.info("Error: registered name {} has no new name data. This is likely due to account {} " +
+                                                "being renamed another time, which is a scenario that is not yet checked automatically.",
+                                        updateNameTransactionData.getNewName(), creator.getAddress());
+                            }
+                            else if (!Objects.equals(creator.getAddress(), newNameData.getOwner())) {
                                 LOGGER.info("Error: registered name {} is owned by {}, but it should be {}",
                                         updateNameTransactionData.getNewName(), newNameData.getOwner(), creator.getAddress());
                                 integrityCheckFailed = true;
@@ -268,42 +289,6 @@ public class NamesDatabaseIntegrityCheck {
         return registerNameTransactions;
     }
 
-    private List<UpdateNameTransactionData> fetchUpdateNameTransactions() {
-        List<UpdateNameTransactionData> updateNameTransactions = new ArrayList<>();
-
-        for (TransactionData transactionData : this.nameTransactions) {
-            if (transactionData.getType() == TransactionType.UPDATE_NAME) {
-                UpdateNameTransactionData updateNameTransactionData = (UpdateNameTransactionData) transactionData;
-                updateNameTransactions.add(updateNameTransactionData);
-            }
-        }
-        return updateNameTransactions;
-    }
-
-    private List<SellNameTransactionData> fetchSellNameTransactions() {
-        List<SellNameTransactionData> sellNameTransactions = new ArrayList<>();
-
-        for (TransactionData transactionData : this.nameTransactions) {
-            if (transactionData.getType() == TransactionType.SELL_NAME) {
-                SellNameTransactionData sellNameTransactionData = (SellNameTransactionData) transactionData;
-                sellNameTransactions.add(sellNameTransactionData);
-            }
-        }
-        return sellNameTransactions;
-    }
-
-    private List<BuyNameTransactionData> fetchBuyNameTransactions() {
-        List<BuyNameTransactionData> buyNameTransactions = new ArrayList<>();
-
-        for (TransactionData transactionData : this.nameTransactions) {
-            if (transactionData.getType() == TransactionType.BUY_NAME) {
-                BuyNameTransactionData buyNameTransactionData = (BuyNameTransactionData) transactionData;
-                buyNameTransactions.add(buyNameTransactionData);
-            }
-        }
-        return buyNameTransactions;
-    }
-
     private void fetchAllNameTransactions(Repository repository) throws DataException {
         List<TransactionData> nameTransactions = new ArrayList<>();
 
@@ -319,43 +304,40 @@ public class NamesDatabaseIntegrityCheck {
         this.nameTransactions = nameTransactions;
     }
 
-    private List<TransactionData> fetchAllTransactionsInvolvingName(String name, Repository repository) throws DataException {
-        List<TransactionData> transactions = new ArrayList<>();
+    public List<TransactionData> fetchAllTransactionsInvolvingName(String name, Repository repository) throws DataException {
+        List<byte[]> signatures = new ArrayList<>();
         String reducedName = Unicode.sanitize(name);
 
-        // Fetch all the confirmed name-modification transactions
-        if (this.nameTransactions.isEmpty()) {
-            this.fetchAllNameTransactions(repository);
+        List<byte[]> registerNameTransactions = repository.getTransactionRepository().getSignaturesMatchingCustomCriteria(
+                TransactionType.REGISTER_NAME, Arrays.asList("(name = ? OR reduced_name = ?)"), Arrays.asList(name, reducedName));
+        signatures.addAll(registerNameTransactions);
+
+        List<byte[]> updateNameTransactions = repository.getTransactionRepository().getSignaturesMatchingCustomCriteria(
+                TransactionType.UPDATE_NAME,
+                Arrays.asList("(name = ? OR new_name = ? OR (reduced_new_name != '' AND reduced_new_name = ?))"),
+                Arrays.asList(name, name, reducedName));
+        signatures.addAll(updateNameTransactions);
+
+        List<byte[]> sellNameTransactions = repository.getTransactionRepository().getSignaturesMatchingCustomCriteria(
+                TransactionType.SELL_NAME, Arrays.asList("name = ?"), Arrays.asList(name));
+        signatures.addAll(sellNameTransactions);
+
+        List<byte[]> buyNameTransactions = repository.getTransactionRepository().getSignaturesMatchingCustomCriteria(
+                TransactionType.BUY_NAME, Arrays.asList("name = ?"), Arrays.asList(name));
+        signatures.addAll(buyNameTransactions);
+
+        List<TransactionData> transactions = new ArrayList<>();
+        for (byte[] signature : signatures) {
+            TransactionData transactionData = repository.getTransactionRepository().fromSignature(signature);
+            // Filter out any unconfirmed transactions
+            if (transactionData.getBlockHeight() != null && transactionData.getBlockHeight() > 0) {
+                transactions.add(transactionData);
+            }
         }
 
-        for (TransactionData transactionData : this.nameTransactions) {
+        // Sort by lowest timestamp first
+        transactions.sort(Comparator.comparingLong(TransactionData::getTimestamp));
 
-            if ((transactionData instanceof RegisterNameTransactionData)) {
-                RegisterNameTransactionData registerNameTransactionData = (RegisterNameTransactionData) transactionData;
-                if (Objects.equals(registerNameTransactionData.getReducedName(), reducedName)) {
-                    transactions.add(transactionData);
-                }
-            }
-            if ((transactionData instanceof UpdateNameTransactionData)) {
-                UpdateNameTransactionData updateNameTransactionData = (UpdateNameTransactionData) transactionData;
-                if (Objects.equals(updateNameTransactionData.getName(), name) ||
-                        Objects.equals(updateNameTransactionData.getReducedNewName(), reducedName)) {
-                    transactions.add(transactionData);
-                }
-            }
-            if ((transactionData instanceof BuyNameTransactionData)) {
-                BuyNameTransactionData buyNameTransactionData = (BuyNameTransactionData) transactionData;
-                if (Objects.equals(buyNameTransactionData.getName(), name)) {
-                    transactions.add(transactionData);
-                }
-            }
-            if ((transactionData instanceof SellNameTransactionData)) {
-                SellNameTransactionData sellNameTransactionData = (SellNameTransactionData) transactionData;
-                if (Objects.equals(sellNameTransactionData.getName(), name)) {
-                    transactions.add(transactionData);
-                }
-            }
-        }
         return transactions;
     }
 

src/main/java/org/qortal/controller/repository/OnlineAccountsSignaturesTrimmer.java

@@ -4,6 +4,7 @@ import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
 import org.qortal.block.BlockChain;
 import org.qortal.controller.Controller;
+import org.qortal.controller.Synchronizer;
 import org.qortal.data.block.BlockData;
 import org.qortal.repository.DataException;
 import org.qortal.repository.Repository;
@@ -20,6 +21,11 @@ public class OnlineAccountsSignaturesTrimmer implements Runnable {
 	public void run() {
 		Thread.currentThread().setName("Online Accounts trimmer");
 
+		if (Settings.getInstance().isLite()) {
+			// Nothing to trim in lite mode
+			return;
+		}
+
 		try (final Repository repository = RepositoryManager.getRepository()) {
 			// Don't even start trimming until initial rush has ended
 			Thread.sleep(INITIAL_SLEEP_PERIOD);
@@ -36,7 +42,7 @@ public class OnlineAccountsSignaturesTrimmer implements Runnable {
 					continue;
 
 				// Don't even attempt if we're mid-sync as our repository requests will be delayed for ages
-				if (Controller.getInstance().isSynchronizing())
+				if (Synchronizer.getInstance().isSynchronizing())
 					continue;
 
 				// Trim blockchain by removing 'old' online accounts signatures

src/main/java/org/qortal/controller/tradebot/BitcoinACCTv3TradeBot.java

@@ -45,9 +45,9 @@ import static java.util.stream.Collectors.toMap;
  * 	<li>Trade-bot entries</li>
  * </ul>
  */
-public class DogecoinACCTv2TradeBot implements AcctTradeBot {
+public class BitcoinACCTv3TradeBot implements AcctTradeBot {
 
-	private static final Logger LOGGER = LogManager.getLogger(DogecoinACCTv2TradeBot.class);
+	private static final Logger LOGGER = LogManager.getLogger(BitcoinACCTv3TradeBot.class);
 
 	public enum State implements TradeBot.StateNameAndValueSupplier {
 		BOB_WAITING_FOR_AT_CONFIRM(10, false, false),
@@ -91,18 +91,18 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 	/** Maximum time Bob waits for his AT creation transaction to be confirmed into a block. (milliseconds) */
 	private static final long MAX_AT_CONFIRMATION_PERIOD = 24 * 60 * 60 * 1000L; // ms
 
-	private static DogecoinACCTv2TradeBot instance;
+	private static BitcoinACCTv3TradeBot instance;
 
 	private final List<String> endStates = Arrays.asList(State.BOB_DONE, State.BOB_REFUNDED, State.ALICE_DONE, State.ALICE_REFUNDING_A, State.ALICE_REFUNDED).stream()
 			.map(State::name)
 			.collect(Collectors.toUnmodifiableList());
 
-	private DogecoinACCTv2TradeBot() {
+	private BitcoinACCTv3TradeBot() {
 	}
 
-	public static synchronized DogecoinACCTv2TradeBot getInstance() {
+	public static synchronized BitcoinACCTv3TradeBot getInstance() {
 		if (instance == null)
-			instance = new DogecoinACCTv2TradeBot();
+			instance = new BitcoinACCTv3TradeBot();
 
 		return instance;
 	}
@@ -113,7 +113,7 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 	}
 
 	/**
-	 * Creates a new trade-bot entry from the "Bob" viewpoint, i.e. OFFERing QORT in exchange for DOGE.
+	 * Creates a new trade-bot entry from the "Bob" viewpoint, i.e. OFFERing QORT in exchange for BTC.
 	 * <p>
 	 * Generates:
 	 * <ul>
@@ -122,14 +122,14 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 	 * Derives:
 	 * <ul>
 	 * 	<li>'native' (as in Qortal) public key, public key hash, address (starting with Q)</li>
-	 * 	<li>'foreign' (as in Dogecoin) public key, public key hash</li>
+	 * 	<li>'foreign' (as in Bitcoin) public key, public key hash</li>
 	 * </ul>
 	 * A Qortal AT is then constructed including the following as constants in the 'data segment':
 	 * <ul>
 	 * 	<li>'native'/Qortal 'trade' address - used as a MESSAGE contact</li>
-	 * 	<li>'foreign'/Dogecoin public key hash - used by Alice's P2SH scripts to allow redeem</li>
+	 * 	<li>'foreign'/Bitcoin public key hash - used by Alice's P2SH scripts to allow redeem</li>
 	 * 	<li>QORT amount on offer by Bob</li>
-	 * 	<li>DOGE amount expected in return by Bob (from Alice)</li>
+	 * 	<li>BTC amount expected in return by Bob (from Alice)</li>
 	 * 	<li>trading timeout, in case things go wrong and everyone needs to refund</li>
 	 * </ul>
 	 * Returns a DEPLOY_AT transaction that needs to be signed and broadcast to the Qortal network.
@@ -151,17 +151,17 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 		byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
 		byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
 
-		// Convert Dogecoin receiving address into public key hash (we only support P2PKH at this time)
-		Address dogecoinReceivingAddress;
+		// Convert Bitcoin receiving address into public key hash (we only support P2PKH at this time)
+		Address bitcoinReceivingAddress;
 		try {
-			dogecoinReceivingAddress = Address.fromString(Dogecoin.getInstance().getNetworkParameters(), tradeBotCreateRequest.receivingAddress);
+			bitcoinReceivingAddress = Address.fromString(Bitcoin.getInstance().getNetworkParameters(), tradeBotCreateRequest.receivingAddress);
 		} catch (AddressFormatException e) {
-			throw new DataException("Unsupported Dogecoin receiving address: " + tradeBotCreateRequest.receivingAddress);
+			throw new DataException("Unsupported Bitcoin receiving address: " + tradeBotCreateRequest.receivingAddress);
 		}
-		if (dogecoinReceivingAddress.getOutputScriptType() != ScriptType.P2PKH)
-			throw new DataException("Unsupported Dogecoin receiving address: " + tradeBotCreateRequest.receivingAddress);
+		if (bitcoinReceivingAddress.getOutputScriptType() != ScriptType.P2PKH)
+			throw new DataException("Unsupported Bitcoin receiving address: " + tradeBotCreateRequest.receivingAddress);
 
-		byte[] dogecoinReceivingAccountInfo = dogecoinReceivingAddress.getHash();
+		byte[] bitcoinReceivingAccountInfo = bitcoinReceivingAddress.getHash();
 
 		PublicKeyAccount creator = new PublicKeyAccount(repository, tradeBotCreateRequest.creatorPublicKey);
 
@@ -172,11 +172,11 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 		byte[] signature = null;
 		BaseTransactionData baseTransactionData = new BaseTransactionData(timestamp, Group.NO_GROUP, reference, creator.getPublicKey(), fee, signature);
 
-		String name = "QORT/DOGE ACCT";
-		String description = "QORT/DOGE cross-chain trade";
+		String name = "QORT/BTC ACCT";
+		String description = "QORT/BTC cross-chain trade";
 		String aTType = "ACCT";
-		String tags = "ACCT QORT DOGE";
-		byte[] creationBytes = DogecoinACCTv2.buildQortalAT(tradeNativeAddress, tradeForeignPublicKeyHash, tradeBotCreateRequest.qortAmount,
+		String tags = "ACCT QORT BTC";
+		byte[] creationBytes = BitcoinACCTv3.buildQortalAT(tradeNativeAddress, tradeForeignPublicKeyHash, tradeBotCreateRequest.qortAmount,
 				tradeBotCreateRequest.foreignAmount, tradeBotCreateRequest.tradeTimeout);
 		long amount = tradeBotCreateRequest.fundingQortAmount;
 
@@ -189,14 +189,14 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 		DeployAtTransaction.ensureATAddress(deployAtTransactionData);
 		String atAddress = deployAtTransactionData.getAtAddress();
 
-		TradeBotData tradeBotData =  new TradeBotData(tradePrivateKey, DogecoinACCTv2.NAME,
+		TradeBotData tradeBotData =  new TradeBotData(tradePrivateKey, BitcoinACCTv3.NAME,
 				State.BOB_WAITING_FOR_AT_CONFIRM.name(), State.BOB_WAITING_FOR_AT_CONFIRM.value,
 				creator.getAddress(), atAddress, timestamp, tradeBotCreateRequest.qortAmount,
 				tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
 				null, null,
-				SupportedBlockchain.DOGECOIN.name(),
+				SupportedBlockchain.BITCOIN.name(),
 				tradeForeignPublicKey, tradeForeignPublicKeyHash,
-				tradeBotCreateRequest.foreignAmount, null, null, null, dogecoinReceivingAccountInfo);
+				tradeBotCreateRequest.foreignAmount, null, null, null, bitcoinReceivingAccountInfo);
 
 		TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Built AT %s. Waiting for deployment", atAddress));
 
@@ -212,15 +212,15 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 	}
 
 	/**
-	 * Creates a trade-bot entry from the 'Alice' viewpoint, i.e. matching DOGE to an existing offer.
+	 * Creates a trade-bot entry from the 'Alice' viewpoint, i.e. matching BTC to an existing offer.
 	 * <p>
 	 * Requires a chosen trade offer from Bob, passed by <tt>crossChainTradeData</tt>
-	 * and access to a Dogecoin wallet via <tt>xprv58</tt>.
+	 * and access to a Bitcoin wallet via <tt>xprv58</tt>.
 	 * <p>
 	 * The <tt>crossChainTradeData</tt> contains the current trade offer state
 	 * as extracted from the AT's data segment.
 	 * <p>
-	 * Access to a funded wallet is via a Dogecoin BIP32 hierarchical deterministic key,
+	 * Access to a funded wallet is via a Bitcoin BIP32 hierarchical deterministic key,
 	 * passed via <tt>xprv58</tt>.
 	 * <b>This key will be stored in your node's database</b>
 	 * to allow trade-bot to create/fund the necessary P2SH transactions!
@@ -230,15 +230,15 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 	 * As an example, the xprv58 can be extract from a <i>legacy, password-less</i>
 	 * Electrum wallet by going to the console tab and entering:<br>
 	 * <tt>wallet.keystore.xprv</tt><br>
-	 * which should result in a base58 string starting with either 'xprv' (for Dogecoin main-net)
-	 * or 'tprv' for (Dogecoin test-net).
+	 * which should result in a base58 string starting with either 'xprv' (for Bitcoin main-net)
+	 * or 'tprv' for (Bitcoin test-net).
 	 * <p>
 	 * It is envisaged that the value in <tt>xprv58</tt> will actually come from a Qortal-UI-managed wallet.
 	 * <p>
 	 * If sufficient funds are available, <b>this method will actually fund the P2SH-A</b>
-	 * with the Dogecoin amount expected by 'Bob'.
+	 * with the Bitcoin amount expected by 'Bob'.
 	 * <p>
-	 * If the Dogecoin transaction is successfully broadcast to the network then
+	 * If the Bitcoin transaction is successfully broadcast to the network then
 	 * we also send a MESSAGE to Bob's trade-bot to let them know.
 	 * <p>
 	 * The trade-bot entry is saved to the repository and the cross-chain trading process commences.
@@ -246,7 +246,7 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 	 * @param repository
 	 * @param crossChainTradeData chosen trade OFFER that Alice wants to match
 	 * @param xprv58 funded wallet xprv in base58
-	 * @return true if P2SH-A funding transaction successfully broadcast to Dogecoin network, false otherwise
+	 * @return true if P2SH-A funding transaction successfully broadcast to Bitcoin network, false otherwise
 	 * @throws DataException
 	 */
 	public ResponseResult startResponse(Repository repository, ATData atData, ACCT acct, CrossChainTradeData crossChainTradeData, String xprv58, String receivingAddress) throws DataException {
@@ -266,12 +266,12 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 		long now = NTP.getTime();
 		int lockTimeA = crossChainTradeData.tradeTimeout * 60 + (int) (now / 1000L);
 
-		TradeBotData tradeBotData =  new TradeBotData(tradePrivateKey, DogecoinACCTv2.NAME,
+		TradeBotData tradeBotData =  new TradeBotData(tradePrivateKey, BitcoinACCTv3.NAME,
 				State.ALICE_WAITING_FOR_AT_LOCK.name(), State.ALICE_WAITING_FOR_AT_LOCK.value,
 				receivingAddress, crossChainTradeData.qortalAtAddress, now, crossChainTradeData.qortAmount,
 				tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
 				secretA, hashOfSecretA,
-				SupportedBlockchain.DOGECOIN.name(),
+				SupportedBlockchain.BITCOIN.name(),
 				tradeForeignPublicKey, tradeForeignPublicKeyHash,
 				crossChainTradeData.expectedForeignAmount, xprv58, null, lockTimeA, receivingPublicKeyHash);
 
@@ -282,9 +282,9 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 		// Check we have enough funds via xprv58 to fund P2SH to cover expectedForeignAmount
 		long p2shFee;
 		try {
-			p2shFee = Dogecoin.getInstance().getP2shFee(now);
+			p2shFee = Bitcoin.getInstance().getP2shFee(now);
 		} catch (ForeignBlockchainException e) {
-			LOGGER.debug("Couldn't estimate Dogecoin fees?");
+			LOGGER.debug("Couldn't estimate Bitcoin fees?");
 			return ResponseResult.NETWORK_ISSUE;
 		}
 
@@ -294,17 +294,17 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 
 		// P2SH-A to be funded
 		byte[] redeemScriptBytes = BitcoinyHTLC.buildScript(tradeForeignPublicKeyHash, lockTimeA, crossChainTradeData.creatorForeignPKH, hashOfSecretA);
-		String p2shAddress = Dogecoin.getInstance().deriveP2shAddress(redeemScriptBytes);
+		String p2shAddress = Bitcoin.getInstance().deriveP2shAddress(redeemScriptBytes);
 
 		// Build transaction for funding P2SH-A
-		Transaction p2shFundingTransaction = Dogecoin.getInstance().buildSpend(tradeBotData.getForeignKey(), p2shAddress, amountA);
+		Transaction p2shFundingTransaction = Bitcoin.getInstance().buildSpend(tradeBotData.getForeignKey(), p2shAddress, amountA);
 		if (p2shFundingTransaction == null) {
 			LOGGER.debug("Unable to build P2SH-A funding transaction - lack of funds?");
 			return ResponseResult.BALANCE_ISSUE;
 		}
 
 		try {
-			Dogecoin.getInstance().broadcastTransaction(p2shFundingTransaction);
+			Bitcoin.getInstance().broadcastTransaction(p2shFundingTransaction);
 		} catch (ForeignBlockchainException e) {
 			// We couldn't fund P2SH-A at this time
 			LOGGER.debug("Couldn't broadcast P2SH-A funding transaction?");
@@ -312,7 +312,7 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 		}
 
 		// Attempt to send MESSAGE to Bob's Qortal trade address
-		byte[] messageData = DogecoinACCTv2.buildOfferMessage(tradeBotData.getTradeForeignPublicKeyHash(), tradeBotData.getHashOfSecret(), tradeBotData.getLockTimeA());
+		byte[] messageData = BitcoinACCTv3.buildOfferMessage(tradeBotData.getTradeForeignPublicKeyHash(), tradeBotData.getHashOfSecret(), tradeBotData.getLockTimeA());
 		String messageRecipient = crossChainTradeData.qortalCreatorTradeAddress;
 
 		boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
@@ -354,6 +354,7 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 			case BOB_DONE:
 			case ALICE_REFUNDED:
 			case BOB_REFUNDED:
+			case ALICE_REFUNDING_A:
 				return true;
 
 			default:
@@ -381,7 +382,7 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 			}
 
 			if (tradeBotState.requiresTradeData) {
-				tradeData = DogecoinACCTv2.getInstance().populateTradeData(repository, atData);
+				tradeData = BitcoinACCTv3.getInstance().populateTradeData(repository, atData);
 				if (tradeData == null) {
 					LOGGER.warn(() -> String.format("Unable to fetch ACCT trade data for AT %s from repository", tradeBotData.getAtAddress()));
 					return;
@@ -462,7 +463,7 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 	 * <p>
 	 * Details from Alice are used to derive P2SH-A address and this is checked for funding balance.
 	 * <p>
-	 * Assuming P2SH-A has at least expected Dogecoin balance,
+	 * Assuming P2SH-A has at least expected Bitcoin balance,
 	 * Bob's trade-bot constructs a zero-fee, PoW MESSAGE to send to Bob's AT with more trade details.
 	 * <p>
 	 * On processing this MESSAGE, Bob's AT should switch into 'TRADE' mode and only trade with Alice.
@@ -480,7 +481,7 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 			return;
 		}
 
-		Dogecoin dogecoin = Dogecoin.getInstance();
+		Bitcoin bitcoin = Bitcoin.getInstance();
 
 		String address = tradeBotData.getTradeNativeAddress();
 		List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, address, null, null, null);
@@ -489,27 +490,27 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 			if (messageTransactionData.isText())
 				continue;
 
-			// We're expecting: HASH160(secret-A), Alice's Dogecoin pubkeyhash and lockTime-A
+			// We're expecting: HASH160(secret-A), Alice's Bitcoin pubkeyhash and lockTime-A
 			byte[] messageData = messageTransactionData.getData();
-			DogecoinACCTv2.OfferMessageData offerMessageData = DogecoinACCTv2.extractOfferMessageData(messageData);
+			BitcoinACCTv3.OfferMessageData offerMessageData = BitcoinACCTv3.extractOfferMessageData(messageData);
 			if (offerMessageData == null)
 				continue;
 
-			byte[] aliceForeignPublicKeyHash = offerMessageData.partnerDogecoinPKH;
+			byte[] aliceForeignPublicKeyHash = offerMessageData.partnerBitcoinPKH;
 			byte[] hashOfSecretA = offerMessageData.hashOfSecretA;
 			int lockTimeA = (int) offerMessageData.lockTimeA;
 			long messageTimestamp = messageTransactionData.getTimestamp();
-			int refundTimeout = DogecoinACCTv2.calcRefundTimeout(messageTimestamp, lockTimeA);
+			int refundTimeout = BitcoinACCTv3.calcRefundTimeout(messageTimestamp, lockTimeA);
 
 			// Determine P2SH-A address and confirm funded
 			byte[] redeemScriptA = BitcoinyHTLC.buildScript(aliceForeignPublicKeyHash, lockTimeA, tradeBotData.getTradeForeignPublicKeyHash(), hashOfSecretA);
-			String p2shAddressA = dogecoin.deriveP2shAddress(redeemScriptA);
+			String p2shAddressA = bitcoin.deriveP2shAddress(redeemScriptA);
 
 			long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
-			long p2shFee = Dogecoin.getInstance().getP2shFee(feeTimestamp);
+			long p2shFee = Bitcoin.getInstance().getP2shFee(feeTimestamp);
 			final long minimumAmountA = tradeBotData.getForeignAmount() + p2shFee;
 
-			BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(dogecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
+			BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(bitcoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
 
 			switch (htlcStatusA) {
 				case UNFUNDED:
@@ -539,7 +540,7 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 			String aliceNativeAddress = Crypto.toAddress(messageTransactionData.getCreatorPublicKey());
 
 			// Build outgoing message, padding each part to 32 bytes to make it easier for AT to consume
-			byte[] outgoingMessageData = DogecoinACCTv2.buildTradeMessage(aliceNativeAddress, aliceForeignPublicKeyHash, hashOfSecretA, lockTimeA, refundTimeout);
+			byte[] outgoingMessageData = BitcoinACCTv3.buildTradeMessage(aliceNativeAddress, aliceForeignPublicKeyHash, hashOfSecretA, lockTimeA, refundTimeout);
 			String messageRecipient = tradeBotData.getAtAddress();
 
 			boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, outgoingMessageData);
@@ -578,7 +579,7 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 	 * <p>
 	 * If all is well, trade-bot then redeems AT using Alice's secret-A, releasing Bob's QORT to Alice.
 	 * <p>
-	 * In revealing a valid secret-A, Bob can then redeem the DOGE funds from P2SH-A.
+	 * In revealing a valid secret-A, Bob can then redeem the BTC funds from P2SH-A.
 	 * <p>
 	 * @throws ForeignBlockchainException
 	 */
@@ -587,19 +588,19 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 		if (aliceUnexpectedState(repository, tradeBotData, atData, crossChainTradeData))
 			return;
 
-		Dogecoin dogecoin = Dogecoin.getInstance();
+		Bitcoin bitcoin = Bitcoin.getInstance();
 		int lockTimeA = tradeBotData.getLockTimeA();
 
 		// Refund P2SH-A if we've passed lockTime-A
 		if (NTP.getTime() >= lockTimeA * 1000L) {
 			byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
-			String p2shAddressA = dogecoin.deriveP2shAddress(redeemScriptA);
+			String p2shAddressA = bitcoin.deriveP2shAddress(redeemScriptA);
 
 			long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
-			long p2shFee = Dogecoin.getInstance().getP2shFee(feeTimestamp);
+			long p2shFee = Bitcoin.getInstance().getP2shFee(feeTimestamp);
 			long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
 
-			BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(dogecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
+			BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(bitcoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
 
 			switch (htlcStatusA) {
 				case UNFUNDED:
@@ -645,7 +646,7 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 		}
 
 		long recipientMessageTimestamp = messageTransactionsData.get(0).getTimestamp();
-		int refundTimeout = DogecoinACCTv2.calcRefundTimeout(recipientMessageTimestamp, lockTimeA);
+		int refundTimeout = BitcoinACCTv3.calcRefundTimeout(recipientMessageTimestamp, lockTimeA);
 
 		// Our calculated refundTimeout should match AT's refundTimeout
 		if (refundTimeout != crossChainTradeData.refundTimeout) {
@@ -659,7 +660,7 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 		// Send 'redeem' MESSAGE to AT using both secret
 		byte[] secretA = tradeBotData.getSecret();
 		String qortalReceivingAddress = Base58.encode(tradeBotData.getReceivingAccountInfo()); // Actually contains whole address, not just PKH
-		byte[] messageData = DogecoinACCTv2.buildRedeemMessage(secretA, qortalReceivingAddress);
+		byte[] messageData = BitcoinACCTv3.buildRedeemMessage(secretA, qortalReceivingAddress);
 		String messageRecipient = tradeBotData.getAtAddress();
 
 		boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
@@ -686,15 +687,15 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 	}
 
 	/**
-	 * Trade-bot is waiting for Alice to redeem Bob's AT, thus revealing secret-A which is required to spend the DOGE funds from P2SH-A.
+	 * Trade-bot is waiting for Alice to redeem Bob's AT, thus revealing secret-A which is required to spend the BTC funds from P2SH-A.
 	 * <p>
 	 * It's possible that Bob's AT has reached its trading timeout and automatically refunded QORT back to Bob. In which case,
 	 * trade-bot is done with this specific trade and finalizes in refunded state.
 	 * <p>
-	 * Assuming trade-bot can extract a valid secret-A from Alice's MESSAGE then trade-bot uses that to redeem the DOGE funds from P2SH-A
-	 * to Bob's 'foreign'/Dogecoin trade legacy-format address, as derived from trade private key.
+	 * Assuming trade-bot can extract a valid secret-A from Alice's MESSAGE then trade-bot uses that to redeem the BTC funds from P2SH-A
+	 * to Bob's 'foreign'/Bitcoin trade legacy-format address, as derived from trade private key.
 	 * <p>
-	 * (This could potentially be 'improved' to send DOGE to any address of Bob's choosing by changing the transaction output).
+	 * (This could potentially be 'improved' to send BTC to any address of Bob's choosing by changing the transaction output).
 	 * <p>
 	 * If trade-bot successfully broadcasts the transaction, then this specific trade is done.
 	 * @throws ForeignBlockchainException
@@ -708,14 +709,14 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 
 		// If AT is REFUNDED or CANCELLED then something has gone wrong
 		if (crossChainTradeData.mode == AcctMode.REFUNDED || crossChainTradeData.mode == AcctMode.CANCELLED) {
-			// Alice hasn't redeemed the QORT, so there is no point in trying to redeem the DOGE
+			// Alice hasn't redeemed the QORT, so there is no point in trying to redeem the BTC
 			TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
 					() -> String.format("AT %s has auto-refunded - trade aborted", tradeBotData.getAtAddress()));
 
 			return;
 		}
 
-		byte[] secretA = DogecoinACCTv2.getInstance().findSecretA(repository, crossChainTradeData);
+		byte[] secretA = BitcoinACCTv3.getInstance().findSecretA(repository, crossChainTradeData);
 		if (secretA == null) {
 			LOGGER.debug(() -> String.format("Unable to find secret-A from redeem message to AT %s?", tradeBotData.getAtAddress()));
 			return;
@@ -723,18 +724,18 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 
 		// Use secret-A to redeem P2SH-A
 
-		Dogecoin dogecoin = Dogecoin.getInstance();
+		Bitcoin bitcoin = Bitcoin.getInstance();
 
 		byte[] receivingAccountInfo = tradeBotData.getReceivingAccountInfo();
 		int lockTimeA = crossChainTradeData.lockTimeA;
 		byte[] redeemScriptA = BitcoinyHTLC.buildScript(crossChainTradeData.partnerForeignPKH, lockTimeA, crossChainTradeData.creatorForeignPKH, crossChainTradeData.hashOfSecretA);
-		String p2shAddressA = dogecoin.deriveP2shAddress(redeemScriptA);
+		String p2shAddressA = bitcoin.deriveP2shAddress(redeemScriptA);
 
 		// Fee for redeem/refund is subtracted from P2SH-A balance.
 		long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
-		long p2shFee = Dogecoin.getInstance().getP2shFee(feeTimestamp);
+		long p2shFee = Bitcoin.getInstance().getP2shFee(feeTimestamp);
 		long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
-		BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(dogecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
+		BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(bitcoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
 
 		switch (htlcStatusA) {
 			case UNFUNDED:
@@ -755,17 +756,17 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 			case FUNDED: {
 				Coin redeemAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
 				ECKey redeemKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
-				List<TransactionOutput> fundingOutputs = dogecoin.getUnspentOutputs(p2shAddressA);
+				List<TransactionOutput> fundingOutputs = bitcoin.getUnspentOutputs(p2shAddressA);
 
-				Transaction p2shRedeemTransaction = BitcoinyHTLC.buildRedeemTransaction(dogecoin.getNetworkParameters(), redeemAmount, redeemKey,
+				Transaction p2shRedeemTransaction = BitcoinyHTLC.buildRedeemTransaction(bitcoin.getNetworkParameters(), redeemAmount, redeemKey,
 						fundingOutputs, redeemScriptA, secretA, receivingAccountInfo);
 
-				dogecoin.broadcastTransaction(p2shRedeemTransaction);
+				bitcoin.broadcastTransaction(p2shRedeemTransaction);
 				break;
 			}
 		}
 
-		String receivingAddress = dogecoin.pkhToAddress(receivingAccountInfo);
+		String receivingAddress = bitcoin.pkhToAddress(receivingAccountInfo);
 
 		TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
 				() -> String.format("P2SH-A %s redeemed. Funds should arrive at %s", tradeBotData.getAtAddress(), receivingAddress));
@@ -783,21 +784,21 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 		if (NTP.getTime() <= lockTimeA * 1000L)
 			return;
 
-		Dogecoin dogecoin = Dogecoin.getInstance();
+		Bitcoin bitcoin = Bitcoin.getInstance();
 
 		// We can't refund P2SH-A until median block time has passed lockTime-A (see BIP113)
-		int medianBlockTime = dogecoin.getMedianBlockTime();
+		int medianBlockTime = bitcoin.getMedianBlockTime();
 		if (medianBlockTime <= lockTimeA)
 			return;
 
 		byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
-		String p2shAddressA = dogecoin.deriveP2shAddress(redeemScriptA);
+		String p2shAddressA = bitcoin.deriveP2shAddress(redeemScriptA);
 
 		// Fee for redeem/refund is subtracted from P2SH-A balance.
 		long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
-		long p2shFee = Dogecoin.getInstance().getP2shFee(feeTimestamp);
+		long p2shFee = Bitcoin.getInstance().getP2shFee(feeTimestamp);
 		long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
-		BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(dogecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
+		BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(bitcoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
 
 		switch (htlcStatusA) {
 			case UNFUNDED:
@@ -819,16 +820,16 @@ public class DogecoinACCTv2TradeBot implements AcctTradeBot {
 			case FUNDED:{
 				Coin refundAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
 				ECKey refundKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
-				List<TransactionOutput> fundingOutputs = dogecoin.getUnspentOutputs(p2shAddressA);
+				List<TransactionOutput> fundingOutputs = bitcoin.getUnspentOutputs(p2shAddressA);
 
 				// Determine receive address for refund
-				String receiveAddress = dogecoin.getUnusedReceiveAddress(tradeBotData.getForeignKey());
-				Address receiving = Address.fromString(dogecoin.getNetworkParameters(), receiveAddress);
+				String receiveAddress = bitcoin.getUnusedReceiveAddress(tradeBotData.getForeignKey());
+				Address receiving = Address.fromString(bitcoin.getNetworkParameters(), receiveAddress);
 
-				Transaction p2shRefundTransaction = BitcoinyHTLC.buildRefundTransaction(dogecoin.getNetworkParameters(), refundAmount, refundKey,
+				Transaction p2shRefundTransaction = BitcoinyHTLC.buildRefundTransaction(bitcoin.getNetworkParameters(), refundAmount, refundKey,
 						fundingOutputs, redeemScriptA, lockTimeA, receiving.getHash());
 
-				dogecoin.broadcastTransaction(p2shRefundTransaction);
+				bitcoin.broadcastTransaction(p2shRefundTransaction);
 				break;
 			}
 		}

src/main/java/org/qortal/controller/tradebot/DigibyteACCTv3TradeBot.java

@@ -45,9 +45,9 @@ import static java.util.stream.Collectors.toMap;
  * 	<li>Trade-bot entries</li>
  * </ul>
  */
-public class LitecoinACCTv2TradeBot implements AcctTradeBot {
+public class DigibyteACCTv3TradeBot implements AcctTradeBot {
 
-	private static final Logger LOGGER = LogManager.getLogger(LitecoinACCTv2TradeBot.class);
+	private static final Logger LOGGER = LogManager.getLogger(DigibyteACCTv3TradeBot.class);
 
 	public enum State implements TradeBot.StateNameAndValueSupplier {
 		BOB_WAITING_FOR_AT_CONFIRM(10, false, false),
@@ -91,18 +91,18 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 	/** Maximum time Bob waits for his AT creation transaction to be confirmed into a block. (milliseconds) */
 	private static final long MAX_AT_CONFIRMATION_PERIOD = 24 * 60 * 60 * 1000L; // ms
 
-	private static LitecoinACCTv2TradeBot instance;
+	private static DigibyteACCTv3TradeBot instance;
 
 	private final List<String> endStates = Arrays.asList(State.BOB_DONE, State.BOB_REFUNDED, State.ALICE_DONE, State.ALICE_REFUNDING_A, State.ALICE_REFUNDED).stream()
 			.map(State::name)
 			.collect(Collectors.toUnmodifiableList());
 
-	private LitecoinACCTv2TradeBot() {
+	private DigibyteACCTv3TradeBot() {
 	}
 
-	public static synchronized LitecoinACCTv2TradeBot getInstance() {
+	public static synchronized DigibyteACCTv3TradeBot getInstance() {
 		if (instance == null)
-			instance = new LitecoinACCTv2TradeBot();
+			instance = new DigibyteACCTv3TradeBot();
 
 		return instance;
 	}
@@ -113,7 +113,7 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 	}
 
 	/**
-	 * Creates a new trade-bot entry from the "Bob" viewpoint, i.e. OFFERing QORT in exchange for LTC.
+	 * Creates a new trade-bot entry from the "Bob" viewpoint, i.e. OFFERing QORT in exchange for DGB.
 	 * <p>
 	 * Generates:
 	 * <ul>
@@ -122,14 +122,14 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 	 * Derives:
 	 * <ul>
 	 * 	<li>'native' (as in Qortal) public key, public key hash, address (starting with Q)</li>
-	 * 	<li>'foreign' (as in Litecoin) public key, public key hash</li>
+	 * 	<li>'foreign' (as in Digibyte) public key, public key hash</li>
 	 * </ul>
 	 * A Qortal AT is then constructed including the following as constants in the 'data segment':
 	 * <ul>
 	 * 	<li>'native'/Qortal 'trade' address - used as a MESSAGE contact</li>
-	 * 	<li>'foreign'/Litecoin public key hash - used by Alice's P2SH scripts to allow redeem</li>
+	 * 	<li>'foreign'/Digibyte public key hash - used by Alice's P2SH scripts to allow redeem</li>
 	 * 	<li>QORT amount on offer by Bob</li>
-	 * 	<li>LTC amount expected in return by Bob (from Alice)</li>
+	 * 	<li>DGB amount expected in return by Bob (from Alice)</li>
 	 * 	<li>trading timeout, in case things go wrong and everyone needs to refund</li>
 	 * </ul>
 	 * Returns a DEPLOY_AT transaction that needs to be signed and broadcast to the Qortal network.
@@ -151,17 +151,17 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 		byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
 		byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
 
-		// Convert Litecoin receiving address into public key hash (we only support P2PKH at this time)
-		Address litecoinReceivingAddress;
+		// Convert Digibyte receiving address into public key hash (we only support P2PKH at this time)
+		Address digibyteReceivingAddress;
 		try {
-			litecoinReceivingAddress = Address.fromString(Litecoin.getInstance().getNetworkParameters(), tradeBotCreateRequest.receivingAddress);
+			digibyteReceivingAddress = Address.fromString(Digibyte.getInstance().getNetworkParameters(), tradeBotCreateRequest.receivingAddress);
 		} catch (AddressFormatException e) {
-			throw new DataException("Unsupported Litecoin receiving address: " + tradeBotCreateRequest.receivingAddress);
+			throw new DataException("Unsupported Digibyte receiving address: " + tradeBotCreateRequest.receivingAddress);
 		}
-		if (litecoinReceivingAddress.getOutputScriptType() != ScriptType.P2PKH)
-			throw new DataException("Unsupported Litecoin receiving address: " + tradeBotCreateRequest.receivingAddress);
+		if (digibyteReceivingAddress.getOutputScriptType() != ScriptType.P2PKH)
+			throw new DataException("Unsupported Digibyte receiving address: " + tradeBotCreateRequest.receivingAddress);
 
-		byte[] litecoinReceivingAccountInfo = litecoinReceivingAddress.getHash();
+		byte[] digibyteReceivingAccountInfo = digibyteReceivingAddress.getHash();
 
 		PublicKeyAccount creator = new PublicKeyAccount(repository, tradeBotCreateRequest.creatorPublicKey);
 
@@ -172,11 +172,11 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 		byte[] signature = null;
 		BaseTransactionData baseTransactionData = new BaseTransactionData(timestamp, Group.NO_GROUP, reference, creator.getPublicKey(), fee, signature);
 
-		String name = "QORT/LTC ACCT";
-		String description = "QORT/LTC cross-chain trade";
+		String name = "QORT/DGB ACCT";
+		String description = "QORT/DGB cross-chain trade";
 		String aTType = "ACCT";
-		String tags = "ACCT QORT LTC";
-		byte[] creationBytes = LitecoinACCTv2.buildQortalAT(tradeNativeAddress, tradeForeignPublicKeyHash, tradeBotCreateRequest.qortAmount,
+		String tags = "ACCT QORT DGB";
+		byte[] creationBytes = DigibyteACCTv3.buildQortalAT(tradeNativeAddress, tradeForeignPublicKeyHash, tradeBotCreateRequest.qortAmount,
 				tradeBotCreateRequest.foreignAmount, tradeBotCreateRequest.tradeTimeout);
 		long amount = tradeBotCreateRequest.fundingQortAmount;
 
@@ -189,14 +189,14 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 		DeployAtTransaction.ensureATAddress(deployAtTransactionData);
 		String atAddress = deployAtTransactionData.getAtAddress();
 
-		TradeBotData tradeBotData =  new TradeBotData(tradePrivateKey, LitecoinACCTv2.NAME,
+		TradeBotData tradeBotData =  new TradeBotData(tradePrivateKey, DigibyteACCTv3.NAME,
 				State.BOB_WAITING_FOR_AT_CONFIRM.name(), State.BOB_WAITING_FOR_AT_CONFIRM.value,
 				creator.getAddress(), atAddress, timestamp, tradeBotCreateRequest.qortAmount,
 				tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
 				null, null,
-				SupportedBlockchain.LITECOIN.name(),
+				SupportedBlockchain.DIGIBYTE.name(),
 				tradeForeignPublicKey, tradeForeignPublicKeyHash,
-				tradeBotCreateRequest.foreignAmount, null, null, null, litecoinReceivingAccountInfo);
+				tradeBotCreateRequest.foreignAmount, null, null, null, digibyteReceivingAccountInfo);
 
 		TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Built AT %s. Waiting for deployment", atAddress));
 
@@ -212,15 +212,15 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 	}
 
 	/**
-	 * Creates a trade-bot entry from the 'Alice' viewpoint, i.e. matching LTC to an existing offer.
+	 * Creates a trade-bot entry from the 'Alice' viewpoint, i.e. matching DGB to an existing offer.
 	 * <p>
 	 * Requires a chosen trade offer from Bob, passed by <tt>crossChainTradeData</tt>
-	 * and access to a Litecoin wallet via <tt>xprv58</tt>.
+	 * and access to a Digibyte wallet via <tt>xprv58</tt>.
 	 * <p>
 	 * The <tt>crossChainTradeData</tt> contains the current trade offer state
 	 * as extracted from the AT's data segment.
 	 * <p>
-	 * Access to a funded wallet is via a Litecoin BIP32 hierarchical deterministic key,
+	 * Access to a funded wallet is via a Digibyte BIP32 hierarchical deterministic key,
 	 * passed via <tt>xprv58</tt>.
 	 * <b>This key will be stored in your node's database</b>
 	 * to allow trade-bot to create/fund the necessary P2SH transactions!
@@ -230,15 +230,15 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 	 * As an example, the xprv58 can be extract from a <i>legacy, password-less</i>
 	 * Electrum wallet by going to the console tab and entering:<br>
 	 * <tt>wallet.keystore.xprv</tt><br>
-	 * which should result in a base58 string starting with either 'xprv' (for Litecoin main-net)
-	 * or 'tprv' for (Litecoin test-net).
+	 * which should result in a base58 string starting with either 'xprv' (for Digibyte main-net)
+	 * or 'tprv' for (Digibyte test-net).
 	 * <p>
 	 * It is envisaged that the value in <tt>xprv58</tt> will actually come from a Qortal-UI-managed wallet.
 	 * <p>
 	 * If sufficient funds are available, <b>this method will actually fund the P2SH-A</b>
-	 * with the Litecoin amount expected by 'Bob'.
+	 * with the Digibyte amount expected by 'Bob'.
 	 * <p>
-	 * If the Litecoin transaction is successfully broadcast to the network then
+	 * If the Digibyte transaction is successfully broadcast to the network then
 	 * we also send a MESSAGE to Bob's trade-bot to let them know.
 	 * <p>
 	 * The trade-bot entry is saved to the repository and the cross-chain trading process commences.
@@ -246,7 +246,7 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 	 * @param repository
 	 * @param crossChainTradeData chosen trade OFFER that Alice wants to match
 	 * @param xprv58 funded wallet xprv in base58
-	 * @return true if P2SH-A funding transaction successfully broadcast to Litecoin network, false otherwise
+	 * @return true if P2SH-A funding transaction successfully broadcast to Digibyte network, false otherwise
 	 * @throws DataException
 	 */
 	public ResponseResult startResponse(Repository repository, ATData atData, ACCT acct, CrossChainTradeData crossChainTradeData, String xprv58, String receivingAddress) throws DataException {
@@ -266,12 +266,12 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 		long now = NTP.getTime();
 		int lockTimeA = crossChainTradeData.tradeTimeout * 60 + (int) (now / 1000L);
 
-		TradeBotData tradeBotData =  new TradeBotData(tradePrivateKey, LitecoinACCTv2.NAME,
+		TradeBotData tradeBotData =  new TradeBotData(tradePrivateKey, DigibyteACCTv3.NAME,
 				State.ALICE_WAITING_FOR_AT_LOCK.name(), State.ALICE_WAITING_FOR_AT_LOCK.value,
 				receivingAddress, crossChainTradeData.qortalAtAddress, now, crossChainTradeData.qortAmount,
 				tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
 				secretA, hashOfSecretA,
-				SupportedBlockchain.LITECOIN.name(),
+				SupportedBlockchain.DIGIBYTE.name(),
 				tradeForeignPublicKey, tradeForeignPublicKeyHash,
 				crossChainTradeData.expectedForeignAmount, xprv58, null, lockTimeA, receivingPublicKeyHash);
 
@@ -282,9 +282,9 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 		// Check we have enough funds via xprv58 to fund P2SH to cover expectedForeignAmount
 		long p2shFee;
 		try {
-			p2shFee = Litecoin.getInstance().getP2shFee(now);
+			p2shFee = Digibyte.getInstance().getP2shFee(now);
 		} catch (ForeignBlockchainException e) {
-			LOGGER.debug("Couldn't estimate Litecoin fees?");
+			LOGGER.debug("Couldn't estimate Digibyte fees?");
 			return ResponseResult.NETWORK_ISSUE;
 		}
 
@@ -294,17 +294,17 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 
 		// P2SH-A to be funded
 		byte[] redeemScriptBytes = BitcoinyHTLC.buildScript(tradeForeignPublicKeyHash, lockTimeA, crossChainTradeData.creatorForeignPKH, hashOfSecretA);
-		String p2shAddress = Litecoin.getInstance().deriveP2shAddress(redeemScriptBytes);
+		String p2shAddress = Digibyte.getInstance().deriveP2shAddress(redeemScriptBytes);
 
 		// Build transaction for funding P2SH-A
-		Transaction p2shFundingTransaction = Litecoin.getInstance().buildSpend(tradeBotData.getForeignKey(), p2shAddress, amountA);
+		Transaction p2shFundingTransaction = Digibyte.getInstance().buildSpend(tradeBotData.getForeignKey(), p2shAddress, amountA);
 		if (p2shFundingTransaction == null) {
 			LOGGER.debug("Unable to build P2SH-A funding transaction - lack of funds?");
 			return ResponseResult.BALANCE_ISSUE;
 		}
 
 		try {
-			Litecoin.getInstance().broadcastTransaction(p2shFundingTransaction);
+			Digibyte.getInstance().broadcastTransaction(p2shFundingTransaction);
 		} catch (ForeignBlockchainException e) {
 			// We couldn't fund P2SH-A at this time
 			LOGGER.debug("Couldn't broadcast P2SH-A funding transaction?");
@@ -312,7 +312,7 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 		}
 
 		// Attempt to send MESSAGE to Bob's Qortal trade address
-		byte[] messageData = LitecoinACCTv2.buildOfferMessage(tradeBotData.getTradeForeignPublicKeyHash(), tradeBotData.getHashOfSecret(), tradeBotData.getLockTimeA());
+		byte[] messageData = DigibyteACCTv3.buildOfferMessage(tradeBotData.getTradeForeignPublicKeyHash(), tradeBotData.getHashOfSecret(), tradeBotData.getLockTimeA());
 		String messageRecipient = crossChainTradeData.qortalCreatorTradeAddress;
 
 		boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
@@ -382,7 +382,7 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 			}
 
 			if (tradeBotState.requiresTradeData) {
-				tradeData = LitecoinACCTv2.getInstance().populateTradeData(repository, atData);
+				tradeData = DigibyteACCTv3.getInstance().populateTradeData(repository, atData);
 				if (tradeData == null) {
 					LOGGER.warn(() -> String.format("Unable to fetch ACCT trade data for AT %s from repository", tradeBotData.getAtAddress()));
 					return;
@@ -463,7 +463,7 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 	 * <p>
 	 * Details from Alice are used to derive P2SH-A address and this is checked for funding balance.
 	 * <p>
-	 * Assuming P2SH-A has at least expected Litecoin balance,
+	 * Assuming P2SH-A has at least expected Digibyte balance,
 	 * Bob's trade-bot constructs a zero-fee, PoW MESSAGE to send to Bob's AT with more trade details.
 	 * <p>
 	 * On processing this MESSAGE, Bob's AT should switch into 'TRADE' mode and only trade with Alice.
@@ -481,7 +481,7 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 			return;
 		}
 
-		Litecoin litecoin = Litecoin.getInstance();
+		Digibyte digibyte = Digibyte.getInstance();
 
 		String address = tradeBotData.getTradeNativeAddress();
 		List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, address, null, null, null);
@@ -490,27 +490,27 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 			if (messageTransactionData.isText())
 				continue;
 
-			// We're expecting: HASH160(secret-A), Alice's Litecoin pubkeyhash and lockTime-A
+			// We're expecting: HASH160(secret-A), Alice's Digibyte pubkeyhash and lockTime-A
 			byte[] messageData = messageTransactionData.getData();
-			LitecoinACCTv2.OfferMessageData offerMessageData = LitecoinACCTv2.extractOfferMessageData(messageData);
+			DigibyteACCTv3.OfferMessageData offerMessageData = DigibyteACCTv3.extractOfferMessageData(messageData);
 			if (offerMessageData == null)
 				continue;
 
-			byte[] aliceForeignPublicKeyHash = offerMessageData.partnerLitecoinPKH;
+			byte[] aliceForeignPublicKeyHash = offerMessageData.partnerDigibytePKH;
 			byte[] hashOfSecretA = offerMessageData.hashOfSecretA;
 			int lockTimeA = (int) offerMessageData.lockTimeA;
 			long messageTimestamp = messageTransactionData.getTimestamp();
-			int refundTimeout = LitecoinACCTv2.calcRefundTimeout(messageTimestamp, lockTimeA);
+			int refundTimeout = DigibyteACCTv3.calcRefundTimeout(messageTimestamp, lockTimeA);
 
 			// Determine P2SH-A address and confirm funded
 			byte[] redeemScriptA = BitcoinyHTLC.buildScript(aliceForeignPublicKeyHash, lockTimeA, tradeBotData.getTradeForeignPublicKeyHash(), hashOfSecretA);
-			String p2shAddressA = litecoin.deriveP2shAddress(redeemScriptA);
+			String p2shAddressA = digibyte.deriveP2shAddress(redeemScriptA);
 
 			long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
-			long p2shFee = Litecoin.getInstance().getP2shFee(feeTimestamp);
+			long p2shFee = Digibyte.getInstance().getP2shFee(feeTimestamp);
 			final long minimumAmountA = tradeBotData.getForeignAmount() + p2shFee;
 
-			BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(litecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
+			BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(digibyte.getBlockchainProvider(), p2shAddressA, minimumAmountA);
 
 			switch (htlcStatusA) {
 				case UNFUNDED:
@@ -540,7 +540,7 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 			String aliceNativeAddress = Crypto.toAddress(messageTransactionData.getCreatorPublicKey());
 
 			// Build outgoing message, padding each part to 32 bytes to make it easier for AT to consume
-			byte[] outgoingMessageData = LitecoinACCTv2.buildTradeMessage(aliceNativeAddress, aliceForeignPublicKeyHash, hashOfSecretA, lockTimeA, refundTimeout);
+			byte[] outgoingMessageData = DigibyteACCTv3.buildTradeMessage(aliceNativeAddress, aliceForeignPublicKeyHash, hashOfSecretA, lockTimeA, refundTimeout);
 			String messageRecipient = tradeBotData.getAtAddress();
 
 			boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, outgoingMessageData);
@@ -579,7 +579,7 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 	 * <p>
 	 * If all is well, trade-bot then redeems AT using Alice's secret-A, releasing Bob's QORT to Alice.
 	 * <p>
-	 * In revealing a valid secret-A, Bob can then redeem the LTC funds from P2SH-A.
+	 * In revealing a valid secret-A, Bob can then redeem the DGB funds from P2SH-A.
 	 * <p>
 	 * @throws ForeignBlockchainException
 	 */
@@ -588,19 +588,19 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 		if (aliceUnexpectedState(repository, tradeBotData, atData, crossChainTradeData))
 			return;
 
-		Litecoin litecoin = Litecoin.getInstance();
+		Digibyte digibyte = Digibyte.getInstance();
 		int lockTimeA = tradeBotData.getLockTimeA();
 
 		// Refund P2SH-A if we've passed lockTime-A
 		if (NTP.getTime() >= lockTimeA * 1000L) {
 			byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
-			String p2shAddressA = litecoin.deriveP2shAddress(redeemScriptA);
+			String p2shAddressA = digibyte.deriveP2shAddress(redeemScriptA);
 
 			long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
-			long p2shFee = Litecoin.getInstance().getP2shFee(feeTimestamp);
+			long p2shFee = Digibyte.getInstance().getP2shFee(feeTimestamp);
 			long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
 
-			BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(litecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
+			BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(digibyte.getBlockchainProvider(), p2shAddressA, minimumAmountA);
 
 			switch (htlcStatusA) {
 				case UNFUNDED:
@@ -646,7 +646,7 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 		}
 
 		long recipientMessageTimestamp = messageTransactionsData.get(0).getTimestamp();
-		int refundTimeout = LitecoinACCTv2.calcRefundTimeout(recipientMessageTimestamp, lockTimeA);
+		int refundTimeout = DigibyteACCTv3.calcRefundTimeout(recipientMessageTimestamp, lockTimeA);
 
 		// Our calculated refundTimeout should match AT's refundTimeout
 		if (refundTimeout != crossChainTradeData.refundTimeout) {
@@ -660,7 +660,7 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 		// Send 'redeem' MESSAGE to AT using both secret
 		byte[] secretA = tradeBotData.getSecret();
 		String qortalReceivingAddress = Base58.encode(tradeBotData.getReceivingAccountInfo()); // Actually contains whole address, not just PKH
-		byte[] messageData = LitecoinACCTv2.buildRedeemMessage(secretA, qortalReceivingAddress);
+		byte[] messageData = DigibyteACCTv3.buildRedeemMessage(secretA, qortalReceivingAddress);
 		String messageRecipient = tradeBotData.getAtAddress();
 
 		boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
@@ -687,15 +687,15 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 	}
 
 	/**
-	 * Trade-bot is waiting for Alice to redeem Bob's AT, thus revealing secret-A which is required to spend the LTC funds from P2SH-A.
+	 * Trade-bot is waiting for Alice to redeem Bob's AT, thus revealing secret-A which is required to spend the DGB funds from P2SH-A.
 	 * <p>
 	 * It's possible that Bob's AT has reached its trading timeout and automatically refunded QORT back to Bob. In which case,
 	 * trade-bot is done with this specific trade and finalizes in refunded state.
 	 * <p>
-	 * Assuming trade-bot can extract a valid secret-A from Alice's MESSAGE then trade-bot uses that to redeem the LTC funds from P2SH-A
-	 * to Bob's 'foreign'/Litecoin trade legacy-format address, as derived from trade private key.
+	 * Assuming trade-bot can extract a valid secret-A from Alice's MESSAGE then trade-bot uses that to redeem the DGB funds from P2SH-A
+	 * to Bob's 'foreign'/Digibyte trade legacy-format address, as derived from trade private key.
 	 * <p>
-	 * (This could potentially be 'improved' to send LTC to any address of Bob's choosing by changing the transaction output).
+	 * (This could potentially be 'improved' to send DGB to any address of Bob's choosing by changing the transaction output).
 	 * <p>
 	 * If trade-bot successfully broadcasts the transaction, then this specific trade is done.
 	 * @throws ForeignBlockchainException
@@ -709,14 +709,14 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 
 		// If AT is REFUNDED or CANCELLED then something has gone wrong
 		if (crossChainTradeData.mode == AcctMode.REFUNDED || crossChainTradeData.mode == AcctMode.CANCELLED) {
-			// Alice hasn't redeemed the QORT, so there is no point in trying to redeem the LTC
+			// Alice hasn't redeemed the QORT, so there is no point in trying to redeem the DGB
 			TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
 					() -> String.format("AT %s has auto-refunded - trade aborted", tradeBotData.getAtAddress()));
 
 			return;
 		}
 
-		byte[] secretA = LitecoinACCTv2.getInstance().findSecretA(repository, crossChainTradeData);
+		byte[] secretA = DigibyteACCTv3.getInstance().findSecretA(repository, crossChainTradeData);
 		if (secretA == null) {
 			LOGGER.debug(() -> String.format("Unable to find secret-A from redeem message to AT %s?", tradeBotData.getAtAddress()));
 			return;
@@ -724,18 +724,18 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 
 		// Use secret-A to redeem P2SH-A
 
-		Litecoin litecoin = Litecoin.getInstance();
+		Digibyte digibyte = Digibyte.getInstance();
 
 		byte[] receivingAccountInfo = tradeBotData.getReceivingAccountInfo();
 		int lockTimeA = crossChainTradeData.lockTimeA;
 		byte[] redeemScriptA = BitcoinyHTLC.buildScript(crossChainTradeData.partnerForeignPKH, lockTimeA, crossChainTradeData.creatorForeignPKH, crossChainTradeData.hashOfSecretA);
-		String p2shAddressA = litecoin.deriveP2shAddress(redeemScriptA);
+		String p2shAddressA = digibyte.deriveP2shAddress(redeemScriptA);
 
 		// Fee for redeem/refund is subtracted from P2SH-A balance.
 		long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
-		long p2shFee = Litecoin.getInstance().getP2shFee(feeTimestamp);
+		long p2shFee = Digibyte.getInstance().getP2shFee(feeTimestamp);
 		long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
-		BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(litecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
+		BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(digibyte.getBlockchainProvider(), p2shAddressA, minimumAmountA);
 
 		switch (htlcStatusA) {
 			case UNFUNDED:
@@ -756,17 +756,17 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 			case FUNDED: {
 				Coin redeemAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
 				ECKey redeemKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
-				List<TransactionOutput> fundingOutputs = litecoin.getUnspentOutputs(p2shAddressA);
+				List<TransactionOutput> fundingOutputs = digibyte.getUnspentOutputs(p2shAddressA);
 
-				Transaction p2shRedeemTransaction = BitcoinyHTLC.buildRedeemTransaction(litecoin.getNetworkParameters(), redeemAmount, redeemKey,
+				Transaction p2shRedeemTransaction = BitcoinyHTLC.buildRedeemTransaction(digibyte.getNetworkParameters(), redeemAmount, redeemKey,
 						fundingOutputs, redeemScriptA, secretA, receivingAccountInfo);
 
-				litecoin.broadcastTransaction(p2shRedeemTransaction);
+				digibyte.broadcastTransaction(p2shRedeemTransaction);
 				break;
 			}
 		}
 
-		String receivingAddress = litecoin.pkhToAddress(receivingAccountInfo);
+		String receivingAddress = digibyte.pkhToAddress(receivingAccountInfo);
 
 		TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
 				() -> String.format("P2SH-A %s redeemed. Funds should arrive at %s", tradeBotData.getAtAddress(), receivingAddress));
@@ -784,21 +784,21 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 		if (NTP.getTime() <= lockTimeA * 1000L)
 			return;
 
-		Litecoin litecoin = Litecoin.getInstance();
+		Digibyte digibyte = Digibyte.getInstance();
 
 		// We can't refund P2SH-A until median block time has passed lockTime-A (see BIP113)
-		int medianBlockTime = litecoin.getMedianBlockTime();
+		int medianBlockTime = digibyte.getMedianBlockTime();
 		if (medianBlockTime <= lockTimeA)
 			return;
 
 		byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
-		String p2shAddressA = litecoin.deriveP2shAddress(redeemScriptA);
+		String p2shAddressA = digibyte.deriveP2shAddress(redeemScriptA);
 
 		// Fee for redeem/refund is subtracted from P2SH-A balance.
 		long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
-		long p2shFee = Litecoin.getInstance().getP2shFee(feeTimestamp);
+		long p2shFee = Digibyte.getInstance().getP2shFee(feeTimestamp);
 		long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
-		BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(litecoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
+		BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(digibyte.getBlockchainProvider(), p2shAddressA, minimumAmountA);
 
 		switch (htlcStatusA) {
 			case UNFUNDED:
@@ -820,16 +820,16 @@ public class LitecoinACCTv2TradeBot implements AcctTradeBot {
 			case FUNDED:{
 				Coin refundAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
 				ECKey refundKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
-				List<TransactionOutput> fundingOutputs = litecoin.getUnspentOutputs(p2shAddressA);
+				List<TransactionOutput> fundingOutputs = digibyte.getUnspentOutputs(p2shAddressA);
 
 				// Determine receive address for refund
-				String receiveAddress = litecoin.getUnusedReceiveAddress(tradeBotData.getForeignKey());
-				Address receiving = Address.fromString(litecoin.getNetworkParameters(), receiveAddress);
+				String receiveAddress = digibyte.getUnusedReceiveAddress(tradeBotData.getForeignKey());
+				Address receiving = Address.fromString(digibyte.getNetworkParameters(), receiveAddress);
 
-				Transaction p2shRefundTransaction = BitcoinyHTLC.buildRefundTransaction(litecoin.getNetworkParameters(), refundAmount, refundKey,
+				Transaction p2shRefundTransaction = BitcoinyHTLC.buildRefundTransaction(digibyte.getNetworkParameters(), refundAmount, refundKey,
 						fundingOutputs, redeemScriptA, lockTimeA, receiving.getHash());
 
-				litecoin.broadcastTransaction(p2shRefundTransaction);
+				digibyte.broadcastTransaction(p2shRefundTransaction);
 				break;
 			}
 		}

src/main/java/org/qortal/controller/tradebot/PirateChainACCTv3TradeBot.java

@@ -0,0 +1,913 @@
+package org.qortal.controller.tradebot;
+
+import com.google.common.hash.HashCode;
+import com.rust.litewalletjni.LiteWalletJni;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.bitcoinj.core.*;
+import org.qortal.account.PrivateKeyAccount;
+import org.qortal.account.PublicKeyAccount;
+import org.qortal.api.model.crosschain.TradeBotCreateRequest;
+import org.qortal.asset.Asset;
+import org.qortal.crosschain.*;
+import org.qortal.crypto.Crypto;
+import org.qortal.data.at.ATData;
+import org.qortal.data.crosschain.CrossChainTradeData;
+import org.qortal.data.crosschain.TradeBotData;
+import org.qortal.data.transaction.BaseTransactionData;
+import org.qortal.data.transaction.DeployAtTransactionData;
+import org.qortal.data.transaction.MessageTransactionData;
+import org.qortal.group.Group;
+import org.qortal.repository.DataException;
+import org.qortal.repository.Repository;
+import org.qortal.transaction.DeployAtTransaction;
+import org.qortal.transaction.MessageTransaction;
+import org.qortal.transaction.Transaction.ValidationResult;
+import org.qortal.transform.TransformationException;
+import org.qortal.transform.transaction.DeployAtTransactionTransformer;
+import org.qortal.utils.Base58;
+import org.qortal.utils.NTP;
+
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import static java.util.Arrays.stream;
+import static java.util.stream.Collectors.toMap;
+
+/**
+ * Performing cross-chain trading steps on behalf of user.
+ * <p>
+ * We deal with three different independent state-spaces here:
+ * <ul>
+ * 	<li>Qortal blockchain</li>
+ * 	<li>Foreign blockchain</li>
+ * 	<li>Trade-bot entries</li>
+ * </ul>
+ */
+public class PirateChainACCTv3TradeBot implements AcctTradeBot {
+
+	private static final Logger LOGGER = LogManager.getLogger(PirateChainACCTv3TradeBot.class);
+
+	public enum State implements TradeBot.StateNameAndValueSupplier {
+		BOB_WAITING_FOR_AT_CONFIRM(10, false, false),
+		BOB_WAITING_FOR_MESSAGE(15, true, true),
+		BOB_WAITING_FOR_AT_REDEEM(25, true, true),
+		BOB_DONE(30, false, false),
+		BOB_REFUNDED(35, false, false),
+
+		ALICE_WAITING_FOR_AT_LOCK(85, true, true),
+		ALICE_DONE(95, false, false),
+		ALICE_REFUNDING_A(105, true, true),
+		ALICE_REFUNDED(110, false, false);
+
+		private static final Map<Integer, State> map = stream(State.values()).collect(toMap(state -> state.value, state -> state));
+
+		public final int value;
+		public final boolean requiresAtData;
+		public final boolean requiresTradeData;
+
+		State(int value, boolean requiresAtData, boolean requiresTradeData) {
+			this.value = value;
+			this.requiresAtData = requiresAtData;
+			this.requiresTradeData = requiresTradeData;
+		}
+
+		public static State valueOf(int value) {
+			return map.get(value);
+		}
+
+		@Override
+		public String getState() {
+			return this.name();
+		}
+
+		@Override
+		public int getStateValue() {
+			return this.value;
+		}
+	}
+
+	/** Maximum time Bob waits for his AT creation transaction to be confirmed into a block. (milliseconds) */
+	private static final long MAX_AT_CONFIRMATION_PERIOD = 24 * 60 * 60 * 1000L; // ms
+
+	private static PirateChainACCTv3TradeBot instance;
+
+	private final List<String> endStates = Arrays.asList(State.BOB_DONE, State.BOB_REFUNDED, State.ALICE_DONE, State.ALICE_REFUNDING_A, State.ALICE_REFUNDED).stream()
+			.map(State::name)
+			.collect(Collectors.toUnmodifiableList());
+
+	private PirateChainACCTv3TradeBot() {
+	}
+
+	public static synchronized PirateChainACCTv3TradeBot getInstance() {
+		if (instance == null)
+			instance = new PirateChainACCTv3TradeBot();
+
+		return instance;
+	}
+
+	@Override
+	public List<String> getEndStates() {
+		return this.endStates;
+	}
+
+	/**
+	 * Creates a new trade-bot entry from the "Bob" viewpoint, i.e. OFFERing QORT in exchange for ARRR.
+	 * <p>
+	 * Generates:
+	 * <ul>
+	 * 	<li>new 'trade' private key</li>
+	 * </ul>
+	 * Derives:
+	 * <ul>
+	 * 	<li>'native' (as in Qortal) public key, public key hash, address (starting with Q)</li>
+	 * 	<li>'foreign' (as in PirateChain) public key, public key hash</li>
+	 * </ul>
+	 * A Qortal AT is then constructed including the following as constants in the 'data segment':
+	 * <ul>
+	 * 	<li>'native'/Qortal 'trade' address - used as a MESSAGE contact</li>
+	 * 	<li>'foreign'/PirateChain public key hash - used by Alice's P2SH scripts to allow redeem</li>
+	 * 	<li>QORT amount on offer by Bob</li>
+	 * 	<li>ARRR amount expected in return by Bob (from Alice)</li>
+	 * 	<li>trading timeout, in case things go wrong and everyone needs to refund</li>
+	 * </ul>
+	 * Returns a DEPLOY_AT transaction that needs to be signed and broadcast to the Qortal network.
+	 * <p>
+	 * Trade-bot will wait for Bob's AT to be deployed before taking next step.
+	 * <p>
+	 * @param repository
+	 * @param tradeBotCreateRequest
+	 * @return raw, unsigned DEPLOY_AT transaction
+	 * @throws DataException
+	 */
+	public byte[] createTrade(Repository repository, TradeBotCreateRequest tradeBotCreateRequest) throws DataException {
+		byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
+
+		byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
+		byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
+		String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
+
+		byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
+		byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
+
+		// ARRR wallet must be loaded before a trade can be created
+		// This is to stop trades from nodes on unsupported architectures (e.g. 32bit)
+		if (!LiteWalletJni.isLoaded()) {
+			throw new DataException("Pirate wallet not found. Check wallets screen for details.");
+		}
+
+		if (!PirateChain.getInstance().isValidAddress(tradeBotCreateRequest.receivingAddress)) {
+			throw new DataException("Unsupported Pirate Chain receiving address: " + tradeBotCreateRequest.receivingAddress);
+		}
+
+		Bech32.Bech32Data decodedReceivingAddress = Bech32.decode(tradeBotCreateRequest.receivingAddress);
+		byte[] pirateChainReceivingAccountInfo = decodedReceivingAddress.data;
+
+		PublicKeyAccount creator = new PublicKeyAccount(repository, tradeBotCreateRequest.creatorPublicKey);
+
+		// Deploy AT
+		long timestamp = NTP.getTime();
+		byte[] reference = creator.getLastReference();
+		long fee = 0L;
+		byte[] signature = null;
+		BaseTransactionData baseTransactionData = new BaseTransactionData(timestamp, Group.NO_GROUP, reference, creator.getPublicKey(), fee, signature);
+
+		String name = "QORT/ARRR ACCT";
+		String description = "QORT/ARRR cross-chain trade";
+		String aTType = "ACCT";
+		String tags = "ACCT QORT ARRR";
+		byte[] creationBytes = PirateChainACCTv3.buildQortalAT(tradeNativeAddress, tradeForeignPublicKey, tradeBotCreateRequest.qortAmount,
+				tradeBotCreateRequest.foreignAmount, tradeBotCreateRequest.tradeTimeout);
+		long amount = tradeBotCreateRequest.fundingQortAmount;
+
+		DeployAtTransactionData deployAtTransactionData = new DeployAtTransactionData(baseTransactionData, name, description, aTType, tags, creationBytes, amount, Asset.QORT);
+
+		DeployAtTransaction deployAtTransaction = new DeployAtTransaction(repository, deployAtTransactionData);
+		fee = deployAtTransaction.calcRecommendedFee();
+		deployAtTransactionData.setFee(fee);
+
+		DeployAtTransaction.ensureATAddress(deployAtTransactionData);
+		String atAddress = deployAtTransactionData.getAtAddress();
+
+		TradeBotData tradeBotData =  new TradeBotData(tradePrivateKey, PirateChainACCTv3.NAME,
+				State.BOB_WAITING_FOR_AT_CONFIRM.name(), State.BOB_WAITING_FOR_AT_CONFIRM.value,
+				creator.getAddress(), atAddress, timestamp, tradeBotCreateRequest.qortAmount,
+				tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
+				null, null,
+				SupportedBlockchain.PIRATECHAIN.name(),
+				tradeForeignPublicKey, tradeForeignPublicKeyHash,
+				tradeBotCreateRequest.foreignAmount, null, null, null, pirateChainReceivingAccountInfo);
+
+		TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Built AT %s. Waiting for deployment", atAddress));
+
+		// Attempt to backup the trade bot data
+		TradeBot.backupTradeBotData(repository, null);
+
+		// Return to user for signing and broadcast as we don't have their Qortal private key
+		try {
+			return DeployAtTransactionTransformer.toBytes(deployAtTransactionData);
+		} catch (TransformationException e) {
+			throw new DataException("Failed to transform DEPLOY_AT transaction?", e);
+		}
+	}
+
+	/**
+	 * Creates a trade-bot entry from the 'Alice' viewpoint, i.e. matching ARRR to an existing offer.
+	 * <p>
+	 * Requires a chosen trade offer from Bob, passed by <tt>crossChainTradeData</tt>
+	 * and access to a PirateChain wallet via <tt>xprv58</tt>.
+	 * <p>
+	 * The <tt>crossChainTradeData</tt> contains the current trade offer state
+	 * as extracted from the AT's data segment.
+	 * <p>
+	 * Access to a funded wallet is via a PirateChain BIP32 hierarchical deterministic key,
+	 * passed via <tt>xprv58</tt>.
+	 * <b>This key will be stored in your node's database</b>
+	 * to allow trade-bot to create/fund the necessary P2SH transactions!
+	 * However, due to the nature of BIP32 keys, it is possible to give the trade-bot
+	 * only a subset of wallet access (see BIP32 for more details).
+	 * <p>
+	 * As an example, the xprv58 can be extract from a <i>legacy, password-less</i>
+	 * Electrum wallet by going to the console tab and entering:<br>
+	 * <tt>wallet.keystore.xprv</tt><br>
+	 * which should result in a base58 string starting with either 'xprv' (for PirateChain main-net)
+	 * or 'tprv' for (PirateChain test-net).
+	 * <p>
+	 * It is envisaged that the value in <tt>xprv58</tt> will actually come from a Qortal-UI-managed wallet.
+	 * <p>
+	 * If sufficient funds are available, <b>this method will actually fund the P2SH-A</b>
+	 * with the PirateChain amount expected by 'Bob'.
+	 * <p>
+	 * If the PirateChain transaction is successfully broadcast to the network then
+	 * we also send a MESSAGE to Bob's trade-bot to let them know.
+	 * <p>
+	 * The trade-bot entry is saved to the repository and the cross-chain trading process commences.
+	 * <p>
+	 * @param repository
+	 * @param crossChainTradeData chosen trade OFFER that Alice wants to match
+	 * @param seed58 funded wallet xprv in base58
+	 * @return true if P2SH-A funding transaction successfully broadcast to PirateChain network, false otherwise
+	 * @throws DataException
+	 */
+	public ResponseResult startResponse(Repository repository, ATData atData, ACCT acct, CrossChainTradeData crossChainTradeData, String seed58, String receivingAddress) throws DataException {
+		byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
+		byte[] secretA = TradeBot.generateSecret();
+		byte[] hashOfSecretA = Crypto.hash160(secretA);
+
+		byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
+		byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
+		String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
+
+		byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
+		byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
+		byte[] receivingPublicKeyHash = Base58.decode(receivingAddress); // Actually the whole address, not just PKH
+
+		String tradePrivateKey58 = Base58.encode(tradePrivateKey);
+        String tradeForeignPublicKey58 = Base58.encode(tradeForeignPublicKey);
+		String secret58 = Base58.encode(secretA);
+
+		// We need to generate lockTime-A: add tradeTimeout to now
+		long now = NTP.getTime();
+		int lockTimeA = crossChainTradeData.tradeTimeout * 60 + (int) (now / 1000L);
+
+		TradeBotData tradeBotData =  new TradeBotData(tradePrivateKey, PirateChainACCTv3.NAME,
+				State.ALICE_WAITING_FOR_AT_LOCK.name(), State.ALICE_WAITING_FOR_AT_LOCK.value,
+				receivingAddress, crossChainTradeData.qortalAtAddress, now, crossChainTradeData.qortAmount,
+				tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
+				secretA, hashOfSecretA,
+				SupportedBlockchain.PIRATECHAIN.name(),
+				tradeForeignPublicKey, tradeForeignPublicKeyHash,
+				crossChainTradeData.expectedForeignAmount, seed58, null, lockTimeA, receivingPublicKeyHash);
+
+		// Attempt to backup the trade bot data
+		// Include tradeBotData as an additional parameter, since it's not in the repository yet
+		TradeBot.backupTradeBotData(repository, Arrays.asList(tradeBotData));
+
+		// Check we have enough funds via xprv58 to fund P2SH to cover expectedForeignAmount
+		long p2shFee;
+		try {
+			p2shFee = PirateChain.getInstance().getP2shFee(now);
+		} catch (ForeignBlockchainException e) {
+			LOGGER.debug("Couldn't estimate PirateChain fees?");
+			return ResponseResult.NETWORK_ISSUE;
+		}
+
+		// Fee for redeem/refund is subtracted from P2SH-A balance.
+		// Do not include fee for funding transaction as this is covered by buildSpend()
+		long amountA = crossChainTradeData.expectedForeignAmount + p2shFee /*redeeming/refunding P2SH-A*/;
+
+		// P2SH-A to be funded
+		byte[] redeemScriptBytes = PirateChainHTLC.buildScript(tradeForeignPublicKey, lockTimeA, crossChainTradeData.creatorForeignPKH, hashOfSecretA);
+		String p2shAddressT3 = PirateChain.getInstance().deriveP2shAddress(redeemScriptBytes); // Use t3 prefix when funding
+		byte[] redeemScriptWithPrefixBytes = PirateChainHTLC.buildScriptWithPrefix(tradeForeignPublicKey, lockTimeA, crossChainTradeData.creatorForeignPKH, hashOfSecretA);
+		String redeemScriptWithPrefix58 = Base58.encode(redeemScriptWithPrefixBytes);
+
+		// Send to P2SH address
+		try {
+			String txid = PirateChain.getInstance().fundP2SH(seed58, p2shAddressT3, amountA, redeemScriptWithPrefix58);
+			LOGGER.info("fundingTxidHex: {}", txid);
+
+		} catch (ForeignBlockchainException e) {
+			LOGGER.debug("Unable to build and send P2SH-A funding transaction - lack of funds?");
+			return ResponseResult.BALANCE_ISSUE;
+		}
+
+		// Attempt to send MESSAGE to Bob's Qortal trade address
+		byte[] messageData = PirateChainACCTv3.buildOfferMessage(tradeBotData.getTradeForeignPublicKey(), tradeBotData.getHashOfSecret(), tradeBotData.getLockTimeA());
+		String messageRecipient = crossChainTradeData.qortalCreatorTradeAddress;
+
+		boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
+		if (!isMessageAlreadySent) {
+			PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
+			MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
+
+			messageTransaction.computeNonce();
+			messageTransaction.sign(sender);
+
+			// reset repository state to prevent deadlock
+			repository.discardChanges();
+			ValidationResult result = messageTransaction.importAsUnconfirmed();
+
+			if (result != ValidationResult.OK) {
+				LOGGER.warn(() -> String.format("Unable to send MESSAGE to Bob's trade-bot %s: %s", messageRecipient, result.name()));
+				return ResponseResult.NETWORK_ISSUE;
+			}
+		}
+
+		TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Funding P2SH-A %s. Messaged Bob. Waiting for AT-lock", p2shAddressT3));
+
+		return ResponseResult.OK;
+	}
+
+	public static String hex(byte[] bytes) {
+		StringBuilder result = new StringBuilder();
+		for (byte aByte : bytes) {
+			result.append(String.format("%02x", aByte));
+			// upper case
+			// result.append(String.format("%02X", aByte));
+		}
+		return result.toString();
+	}
+
+	@Override
+	public boolean canDelete(Repository repository, TradeBotData tradeBotData) throws DataException {
+		State tradeBotState = State.valueOf(tradeBotData.getStateValue());
+		if (tradeBotState == null)
+			return true;
+
+		// If the AT doesn't exist then we might as well let the user tidy up
+		if (!repository.getATRepository().exists(tradeBotData.getAtAddress()))
+			return true;
+
+		switch (tradeBotState) {
+			case BOB_WAITING_FOR_AT_CONFIRM:
+			case ALICE_DONE:
+			case BOB_DONE:
+			case ALICE_REFUNDED:
+			case BOB_REFUNDED:
+			case ALICE_REFUNDING_A:
+				return true;
+
+			default:
+				return false;
+		}
+	}
+
+	@Override
+	public void progress(Repository repository, TradeBotData tradeBotData) throws DataException, ForeignBlockchainException {
+		State tradeBotState = State.valueOf(tradeBotData.getStateValue());
+		if (tradeBotState == null) {
+			LOGGER.info(() -> String.format("Trade-bot entry for AT %s has invalid state?", tradeBotData.getAtAddress()));
+			return;
+		}
+
+		ATData atData = null;
+		CrossChainTradeData tradeData = null;
+
+		if (tradeBotState.requiresAtData) {
+			// Attempt to fetch AT data
+			atData = repository.getATRepository().fromATAddress(tradeBotData.getAtAddress());
+			if (atData == null) {
+				LOGGER.debug(() -> String.format("Unable to fetch trade AT %s from repository", tradeBotData.getAtAddress()));
+				return;
+			}
+
+			if (tradeBotState.requiresTradeData) {
+				tradeData = PirateChainACCTv3.getInstance().populateTradeData(repository, atData);
+				if (tradeData == null) {
+					LOGGER.warn(() -> String.format("Unable to fetch ACCT trade data for AT %s from repository", tradeBotData.getAtAddress()));
+					return;
+				}
+			}
+		}
+
+		switch (tradeBotState) {
+			case BOB_WAITING_FOR_AT_CONFIRM:
+				handleBobWaitingForAtConfirm(repository, tradeBotData);
+				break;
+
+			case BOB_WAITING_FOR_MESSAGE:
+				TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
+				handleBobWaitingForMessage(repository, tradeBotData, atData, tradeData);
+				break;
+
+			case ALICE_WAITING_FOR_AT_LOCK:
+				TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
+				handleAliceWaitingForAtLock(repository, tradeBotData, atData, tradeData);
+				break;
+
+			case BOB_WAITING_FOR_AT_REDEEM:
+				TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
+				handleBobWaitingForAtRedeem(repository, tradeBotData, atData, tradeData);
+				break;
+
+			case ALICE_DONE:
+			case BOB_DONE:
+				break;
+
+			case ALICE_REFUNDING_A:
+				TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
+				handleAliceRefundingP2shA(repository, tradeBotData, atData, tradeData);
+				break;
+
+			case ALICE_REFUNDED:
+			case BOB_REFUNDED:
+				break;
+		}
+	}
+
+	/**
+	 * Trade-bot is waiting for Bob's AT to deploy.
+	 * <p>
+	 * If AT is deployed, then trade-bot's next step is to wait for MESSAGE from Alice.
+	 */
+	private void handleBobWaitingForAtConfirm(Repository repository, TradeBotData tradeBotData) throws DataException {
+		if (!repository.getATRepository().exists(tradeBotData.getAtAddress())) {
+			if (NTP.getTime() - tradeBotData.getTimestamp() <= MAX_AT_CONFIRMATION_PERIOD)
+				return;
+
+			// We've waited ages for AT to be confirmed into a block but something has gone awry.
+			// After this long we assume transaction loss so give up with trade-bot entry too.
+			tradeBotData.setState(State.BOB_REFUNDED.name());
+			tradeBotData.setStateValue(State.BOB_REFUNDED.value);
+			tradeBotData.setTimestamp(NTP.getTime());
+			// We delete trade-bot entry here instead of saving, hence not using updateTradeBotState()
+			repository.getCrossChainRepository().delete(tradeBotData.getTradePrivateKey());
+			repository.saveChanges();
+
+			LOGGER.info(() -> String.format("AT %s never confirmed. Giving up on trade", tradeBotData.getAtAddress()));
+			TradeBot.notifyStateChange(tradeBotData);
+			return;
+		}
+
+		TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_MESSAGE,
+				() -> String.format("AT %s confirmed ready. Waiting for trade message", tradeBotData.getAtAddress()));
+	}
+
+	/**
+	 * Trade-bot is waiting for MESSAGE from Alice's trade-bot, containing Alice's trade info.
+	 * <p>
+	 * It's possible Bob has cancelling his trade offer, receiving an automatic QORT refund,
+	 * in which case trade-bot is done with this specific trade and finalizes on refunded state.
+	 * <p>
+	 * Assuming trade is still on offer, trade-bot checks the contents of MESSAGE from Alice's trade-bot.
+	 * <p>
+	 * Details from Alice are used to derive P2SH-A address and this is checked for funding balance.
+	 * <p>
+	 * Assuming P2SH-A has at least expected PirateChain balance,
+	 * Bob's trade-bot constructs a zero-fee, PoW MESSAGE to send to Bob's AT with more trade details.
+	 * <p>
+	 * On processing this MESSAGE, Bob's AT should switch into 'TRADE' mode and only trade with Alice.
+	 * <p>
+	 * Trade-bot's next step is to wait for Alice to redeem the AT, which will allow Bob to
+	 * extract secret-A needed to redeem Alice's P2SH.
+	 * @throws ForeignBlockchainException
+	 */
+	private void handleBobWaitingForMessage(Repository repository, TradeBotData tradeBotData,
+			ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
+		// If AT has finished then Bob likely cancelled his trade offer
+		if (atData.getIsFinished()) {
+			TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
+					() -> String.format("AT %s cancelled - trading aborted", tradeBotData.getAtAddress()));
+			return;
+		}
+
+		PirateChain pirateChain = PirateChain.getInstance();
+
+		String address = tradeBotData.getTradeNativeAddress();
+		List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, address, null, null, null);
+
+		for (MessageTransactionData messageTransactionData : messageTransactionsData) {
+			if (messageTransactionData.isText())
+				continue;
+
+			// We're expecting: HASH160(secret-A), Alice's PirateChain pubkeyhash and lockTime-A
+			byte[] messageData = messageTransactionData.getData();
+			PirateChainACCTv3.OfferMessageData offerMessageData = PirateChainACCTv3.extractOfferMessageData(messageData);
+			if (offerMessageData == null)
+				continue;
+
+			byte[] aliceForeignPublicKey = offerMessageData.partnerPirateChainPublicKey;
+			byte[] hashOfSecretA = offerMessageData.hashOfSecretA;
+			int lockTimeA = (int) offerMessageData.lockTimeA;
+			long messageTimestamp = messageTransactionData.getTimestamp();
+			int refundTimeout = PirateChainACCTv3.calcRefundTimeout(messageTimestamp, lockTimeA);
+
+			// Determine P2SH-A address and confirm funded
+			byte[] redeemScriptA = PirateChainHTLC.buildScript(aliceForeignPublicKey, lockTimeA, tradeBotData.getTradeForeignPublicKey(), hashOfSecretA);
+			String p2shAddress = pirateChain.deriveP2shAddressBPrefix(redeemScriptA); // Use 'b' prefix when checking status
+
+			long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
+			long p2shFee = PirateChain.getInstance().getP2shFee(feeTimestamp);
+			final long minimumAmountA = tradeBotData.getForeignAmount() + p2shFee;
+
+			BitcoinyHTLC.Status htlcStatusA = PirateChainHTLC.determineHtlcStatus(pirateChain.getBlockchainProvider(), p2shAddress, minimumAmountA);
+
+			switch (htlcStatusA) {
+				case UNFUNDED:
+				case FUNDING_IN_PROGRESS:
+					// There might be another MESSAGE from someone else with an actually funded P2SH-A...
+					continue;
+
+				case REDEEM_IN_PROGRESS:
+				case REDEEMED:
+					// We've already redeemed this?
+					TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
+							() -> String.format("P2SH-A %s already spent? Assuming trade complete", p2shAddress));
+					return;
+
+				case REFUND_IN_PROGRESS:
+				case REFUNDED:
+					// This P2SH-A is burnt, but there might be another MESSAGE from someone else with an actually funded P2SH-A...
+					continue;
+
+				case FUNDED:
+					// Fall-through out of switch...
+					break;
+			}
+
+			// Good to go - send MESSAGE to AT
+
+			String aliceNativeAddress = Crypto.toAddress(messageTransactionData.getCreatorPublicKey());
+
+			// Build outgoing message, padding each part to 32 bytes to make it easier for AT to consume
+			byte[] outgoingMessageData = PirateChainACCTv3.buildTradeMessage(aliceNativeAddress, aliceForeignPublicKey, hashOfSecretA, lockTimeA, refundTimeout);
+			String messageRecipient = tradeBotData.getAtAddress();
+
+			boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, outgoingMessageData);
+			if (!isMessageAlreadySent) {
+				PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
+				MessageTransaction outgoingMessageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, outgoingMessageData, false, false);
+
+				outgoingMessageTransaction.computeNonce();
+				outgoingMessageTransaction.sign(sender);
+
+				// reset repository state to prevent deadlock
+				repository.discardChanges();
+				ValidationResult result = outgoingMessageTransaction.importAsUnconfirmed();
+
+				if (result != ValidationResult.OK) {
+					LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
+					return;
+				}
+			}
+
+			TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_AT_REDEEM,
+					() -> String.format("Locked AT %s to %s. Waiting for AT redeem", tradeBotData.getAtAddress(), aliceNativeAddress));
+
+			return;
+		}
+	}
+
+	/**
+	 * Trade-bot is waiting for Bob's AT to switch to TRADE mode and lock trade to Alice only.
+	 * <p>
+	 * It's possible that Bob has cancelled his trade offer in the mean time, or that somehow
+	 * this process has taken so long that we've reached P2SH-A's locktime, or that someone else
+	 * has managed to trade with Bob. In any of these cases, trade-bot switches to begin the refunding process.
+	 * <p>
+	 * Assuming Bob's AT is locked to Alice, trade-bot checks AT's state data to make sure it is correct.
+	 * <p>
+	 * If all is well, trade-bot then redeems AT using Alice's secret-A, releasing Bob's QORT to Alice.
+	 * <p>
+	 * In revealing a valid secret-A, Bob can then redeem the ARRR funds from P2SH-A.
+	 * <p>
+	 * @throws ForeignBlockchainException
+	 */
+	private void handleAliceWaitingForAtLock(Repository repository, TradeBotData tradeBotData,
+			ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
+		if (aliceUnexpectedState(repository, tradeBotData, atData, crossChainTradeData))
+			return;
+
+		PirateChain pirateChain = PirateChain.getInstance();
+		int lockTimeA = tradeBotData.getLockTimeA();
+
+		// Refund P2SH-A if we've passed lockTime-A
+		if (NTP.getTime() >= lockTimeA * 1000L) {
+			byte[] redeemScriptA = PirateChainHTLC.buildScript(tradeBotData.getTradeForeignPublicKey(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
+			String p2shAddress = pirateChain.deriveP2shAddressBPrefix(redeemScriptA); // Use 'b' prefix when checking status
+
+			long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
+			long p2shFee = PirateChain.getInstance().getP2shFee(feeTimestamp);
+			long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
+
+			BitcoinyHTLC.Status htlcStatusA = PirateChainHTLC.determineHtlcStatus(pirateChain.getBlockchainProvider(), p2shAddress, minimumAmountA);
+
+			switch (htlcStatusA) {
+				case UNFUNDED:
+				case FUNDING_IN_PROGRESS:
+				case FUNDED:
+					break;
+
+				case REDEEM_IN_PROGRESS:
+				case REDEEMED:
+					// Already redeemed?
+					TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
+							() -> String.format("P2SH-A %s already spent? Assuming trade completed", p2shAddress));
+					return;
+
+				case REFUND_IN_PROGRESS:
+				case REFUNDED:
+					TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
+							() -> String.format("P2SH-A %s already refunded. Trade aborted", p2shAddress));
+					return;
+
+			}
+
+			TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
+					() -> atData.getIsFinished()
+					? String.format("AT %s cancelled. Refunding P2SH-A %s - aborting trade", tradeBotData.getAtAddress(), p2shAddress)
+					: String.format("LockTime-A reached, refunding P2SH-A %s - aborting trade", p2shAddress));
+
+			return;
+		}
+
+		// We're waiting for AT to be in TRADE mode
+		if (crossChainTradeData.mode != AcctMode.TRADING)
+			return;
+
+		// AT is in TRADE mode and locked to us as checked by aliceUnexpectedState() above
+
+		// Find our MESSAGE to AT from previous state
+		List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(tradeBotData.getTradeNativePublicKey(),
+				crossChainTradeData.qortalCreatorTradeAddress, null, null, null);
+		if (messageTransactionsData == null || messageTransactionsData.isEmpty()) {
+			LOGGER.warn(() -> String.format("Unable to find our message to trade creator %s?", crossChainTradeData.qortalCreatorTradeAddress));
+			return;
+		}
+
+		long recipientMessageTimestamp = messageTransactionsData.get(0).getTimestamp();
+		int refundTimeout = PirateChainACCTv3.calcRefundTimeout(recipientMessageTimestamp, lockTimeA);
+
+		// Our calculated refundTimeout should match AT's refundTimeout
+		if (refundTimeout != crossChainTradeData.refundTimeout) {
+			LOGGER.debug(() -> String.format("Trade AT refundTimeout '%d' doesn't match our refundTimeout '%d'", crossChainTradeData.refundTimeout, refundTimeout));
+			// We'll eventually refund
+			return;
+		}
+
+		// We're good to redeem AT
+
+		// Send 'redeem' MESSAGE to AT using both secret
+		byte[] secretA = tradeBotData.getSecret();
+		String qortalReceivingAddress = Base58.encode(tradeBotData.getReceivingAccountInfo()); // Actually contains whole address, not just PKH
+		byte[] messageData = PirateChainACCTv3.buildRedeemMessage(secretA, qortalReceivingAddress);
+		String messageRecipient = tradeBotData.getAtAddress();
+
+		boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
+		if (!isMessageAlreadySent) {
+			PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
+			MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
+
+			messageTransaction.computeNonce();
+			messageTransaction.sign(sender);
+
+			// Reset repository state to prevent deadlock
+			repository.discardChanges();
+			ValidationResult result = messageTransaction.importAsUnconfirmed();
+
+			if (result != ValidationResult.OK) {
+				LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
+				return;
+			}
+		}
+
+		TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
+				() -> String.format("Redeeming AT %s. Funds should arrive at %s",
+						tradeBotData.getAtAddress(), qortalReceivingAddress));
+	}
+
+	/**
+	 * Trade-bot is waiting for Alice to redeem Bob's AT, thus revealing secret-A which is required to spend the ARRR funds from P2SH-A.
+	 * <p>
+	 * It's possible that Bob's AT has reached its trading timeout and automatically refunded QORT back to Bob. In which case,
+	 * trade-bot is done with this specific trade and finalizes in refunded state.
+	 * <p>
+	 * Assuming trade-bot can extract a valid secret-A from Alice's MESSAGE then trade-bot uses that to redeem the ARRR funds from P2SH-A
+	 * to Bob's 'foreign'/PirateChain trade legacy-format address, as derived from trade private key.
+	 * <p>
+	 * (This could potentially be 'improved' to send ARRR to any address of Bob's choosing by changing the transaction output).
+	 * <p>
+	 * If trade-bot successfully broadcasts the transaction, then this specific trade is done.
+	 * @throws ForeignBlockchainException
+	 */
+	private void handleBobWaitingForAtRedeem(Repository repository, TradeBotData tradeBotData,
+			ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
+		// AT should be 'finished' once Alice has redeemed QORT funds
+		if (!atData.getIsFinished())
+			// Not finished yet
+			return;
+
+		// If AT is REFUNDED or CANCELLED then something has gone wrong
+		if (crossChainTradeData.mode == AcctMode.REFUNDED || crossChainTradeData.mode == AcctMode.CANCELLED) {
+			// Alice hasn't redeemed the QORT, so there is no point in trying to redeem the ARRR
+			TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
+					() -> String.format("AT %s has auto-refunded - trade aborted", tradeBotData.getAtAddress()));
+
+			return;
+		}
+
+		byte[] secretA = PirateChainACCTv3.getInstance().findSecretA(repository, crossChainTradeData);
+		if (secretA == null) {
+			LOGGER.debug(() -> String.format("Unable to find secret-A from redeem message to AT %s?", tradeBotData.getAtAddress()));
+			return;
+		}
+
+		// Use secret-A to redeem P2SH-A
+
+		PirateChain pirateChain = PirateChain.getInstance();
+
+		byte[] receivingAccountInfo = tradeBotData.getReceivingAccountInfo();
+		int lockTimeA = crossChainTradeData.lockTimeA;
+		byte[] redeemScriptA = PirateChainHTLC.buildScript(crossChainTradeData.partnerForeignPKH, lockTimeA, crossChainTradeData.creatorForeignPKH, crossChainTradeData.hashOfSecretA);
+		String p2shAddress = pirateChain.deriveP2shAddressBPrefix(redeemScriptA); // Use 'b' prefix when checking status
+		String p2shAddressT3 = pirateChain.deriveP2shAddress(redeemScriptA); // Use 't3' prefix when refunding
+
+		// Fee for redeem/refund is subtracted from P2SH-A balance.
+		long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
+		long p2shFee = PirateChain.getInstance().getP2shFee(feeTimestamp);
+		long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
+		String receivingAddress = Bech32.encode("zs", receivingAccountInfo);
+
+		BitcoinyHTLC.Status htlcStatusA = PirateChainHTLC.determineHtlcStatus(pirateChain.getBlockchainProvider(), p2shAddress, minimumAmountA);
+
+		switch (htlcStatusA) {
+			case UNFUNDED:
+			case FUNDING_IN_PROGRESS:
+				// P2SH-A suddenly not funded? Our best bet at this point is to hope for AT auto-refund
+				return;
+
+			case REDEEM_IN_PROGRESS:
+			case REDEEMED:
+				// Double-check that we have redeemed P2SH-A...
+				break;
+
+			case REFUND_IN_PROGRESS:
+			case REFUNDED:
+				// Wait for AT to auto-refund
+				return;
+
+			case FUNDED: {
+				// Get funding txid
+				String fundingTxidHex = PirateChainHTLC.getUnspentFundingTxid(pirateChain.getBlockchainProvider(), p2shAddress, minimumAmountA);
+				if (fundingTxidHex == null) {
+					throw new ForeignBlockchainException("Missing funding txid when redeeming P2SH");
+				}
+				String fundingTxid58 = Base58.encode(HashCode.fromString(fundingTxidHex).asBytes());
+
+				// Redeem P2SH
+				Coin redeemAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
+				byte[] privateKey = tradeBotData.getTradePrivateKey();
+				String secret58 = Base58.encode(secretA);
+				String privateKey58 = Base58.encode(privateKey);
+				String redeemScript58 = Base58.encode(redeemScriptA);
+
+				String txid = PirateChain.getInstance().redeemP2sh(p2shAddressT3, receivingAddress, redeemAmount.value,
+						redeemScript58, fundingTxid58, secret58, privateKey58);
+				LOGGER.info("Redeem txid: {}", txid);
+				break;
+			}
+		}
+
+		TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
+				() -> String.format("P2SH-A %s redeemed. Funds should arrive at %s", tradeBotData.getAtAddress(), receivingAddress));
+	}
+
+	/**
+	 * Trade-bot is attempting to refund P2SH-A.
+	 * @throws ForeignBlockchainException
+	 */
+	private void handleAliceRefundingP2shA(Repository repository, TradeBotData tradeBotData,
+			ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
+		int lockTimeA = tradeBotData.getLockTimeA();
+
+		// We can't refund P2SH-A until lockTime-A has passed
+		if (NTP.getTime() <= lockTimeA * 1000L)
+			return;
+
+		PirateChain pirateChain = PirateChain.getInstance();
+
+		// We can't refund P2SH-A until median block time has passed lockTime-A (see BIP113)
+		int medianBlockTime = pirateChain.getMedianBlockTime();
+		if (medianBlockTime <= lockTimeA)
+			return;
+
+		byte[] redeemScriptA = PirateChainHTLC.buildScript(tradeBotData.getTradeForeignPublicKey(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
+		String p2shAddress = pirateChain.deriveP2shAddressBPrefix(redeemScriptA); // Use 'b' prefix when checking status
+		String p2shAddressT3 = pirateChain.deriveP2shAddress(redeemScriptA); // Use 't3' prefix when refunding
+
+		// Fee for redeem/refund is subtracted from P2SH-A balance.
+		long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
+		long p2shFee = PirateChain.getInstance().getP2shFee(feeTimestamp);
+		long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
+		BitcoinyHTLC.Status htlcStatusA = PirateChainHTLC.determineHtlcStatus(pirateChain.getBlockchainProvider(), p2shAddress, minimumAmountA);
+
+		switch (htlcStatusA) {
+			case UNFUNDED:
+			case FUNDING_IN_PROGRESS:
+				// Still waiting for P2SH-A to be funded...
+				return;
+
+			case REDEEM_IN_PROGRESS:
+			case REDEEMED:
+				// Too late!
+				TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
+						() -> String.format("P2SH-A %s already spent!", p2shAddress));
+				return;
+
+			case REFUND_IN_PROGRESS:
+			case REFUNDED:
+				break;
+
+			case FUNDED:{
+				// Get funding txid
+				String fundingTxidHex = PirateChainHTLC.getUnspentFundingTxid(pirateChain.getBlockchainProvider(), p2shAddress, minimumAmountA);
+				if (fundingTxidHex == null) {
+					throw new ForeignBlockchainException("Missing funding txid when refunding P2SH");
+				}
+				String fundingTxid58 = Base58.encode(HashCode.fromString(fundingTxidHex).asBytes());
+
+				Coin refundAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
+				byte[] privateKey = tradeBotData.getTradePrivateKey();
+				String privateKey58 = Base58.encode(privateKey);
+				String redeemScript58 = Base58.encode(redeemScriptA);
+				String receivingAddress = pirateChain.getWalletAddress(tradeBotData.getForeignKey());
+
+				String txid = PirateChain.getInstance().refundP2sh(p2shAddressT3,
+						receivingAddress, refundAmount.value, redeemScript58, fundingTxid58, lockTimeA, privateKey58);
+				LOGGER.info("Refund txid: {}", txid);
+				break;
+			}
+		}
+
+		TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
+				() -> String.format("LockTime-A reached. Refunded P2SH-A %s. Trade aborted", p2shAddress));
+	}
+
+	/**
+	 * Returns true if Alice finds AT unexpectedly cancelled, refunded, redeemed or locked to someone else.
+	 * <p>
+	 * Will automatically update trade-bot state to <tt>ALICE_REFUNDING_A</tt> or <tt>ALICE_DONE</tt> as necessary.
+	 * 
+	 * @throws DataException
+	 * @throws ForeignBlockchainException
+	 */
+	private boolean aliceUnexpectedState(Repository repository, TradeBotData tradeBotData,
+			ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
+		// This is OK
+		if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.OFFERING)
+			return false;
+
+		boolean isAtLockedToUs = tradeBotData.getTradeNativeAddress().equals(crossChainTradeData.qortalPartnerAddress);
+
+		if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.TRADING)
+			if (isAtLockedToUs) {
+				// AT is trading with us - OK
+				return false;
+			} else {
+				TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
+						() -> String.format("AT %s trading with someone else: %s. Refunding & aborting trade", tradeBotData.getAtAddress(), crossChainTradeData.qortalPartnerAddress));
+
+				return true;
+			}
+
+		if (atData.getIsFinished() && crossChainTradeData.mode == AcctMode.REDEEMED && isAtLockedToUs) {
+			// We've redeemed already?
+			TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
+					() -> String.format("AT %s already redeemed by us. Trade completed", tradeBotData.getAtAddress()));
+		} else {
+			// Any other state is not good, so start defensive refund
+			TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
+					() -> String.format("AT %s cancelled/refunded/redeemed by someone else/invalid state. Refunding & aborting trade", tradeBotData.getAtAddress()));
+		}
+
+		return true;
+	}
+
+	private long calcFeeTimestamp(int lockTimeA, int tradeTimeout) {
+		return (lockTimeA - tradeTimeout * 60) * 1000L;
+	}
+
+}

src/main/java/org/qortal/controller/tradebot/RavencoinACCTv3TradeBot.java

@@ -0,0 +1,885 @@
+package org.qortal.controller.tradebot;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.bitcoinj.core.*;
+import org.bitcoinj.script.Script.ScriptType;
+import org.qortal.account.PrivateKeyAccount;
+import org.qortal.account.PublicKeyAccount;
+import org.qortal.api.model.crosschain.TradeBotCreateRequest;
+import org.qortal.asset.Asset;
+import org.qortal.crosschain.*;
+import org.qortal.crypto.Crypto;
+import org.qortal.data.at.ATData;
+import org.qortal.data.crosschain.CrossChainTradeData;
+import org.qortal.data.crosschain.TradeBotData;
+import org.qortal.data.transaction.BaseTransactionData;
+import org.qortal.data.transaction.DeployAtTransactionData;
+import org.qortal.data.transaction.MessageTransactionData;
+import org.qortal.group.Group;
+import org.qortal.repository.DataException;
+import org.qortal.repository.Repository;
+import org.qortal.transaction.DeployAtTransaction;
+import org.qortal.transaction.MessageTransaction;
+import org.qortal.transaction.Transaction.ValidationResult;
+import org.qortal.transform.TransformationException;
+import org.qortal.transform.transaction.DeployAtTransactionTransformer;
+import org.qortal.utils.Base58;
+import org.qortal.utils.NTP;
+
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import static java.util.Arrays.stream;
+import static java.util.stream.Collectors.toMap;
+
+/**
+ * Performing cross-chain trading steps on behalf of user.
+ * <p>
+ * We deal with three different independent state-spaces here:
+ * <ul>
+ * 	<li>Qortal blockchain</li>
+ * 	<li>Foreign blockchain</li>
+ * 	<li>Trade-bot entries</li>
+ * </ul>
+ */
+public class RavencoinACCTv3TradeBot implements AcctTradeBot {
+
+	private static final Logger LOGGER = LogManager.getLogger(RavencoinACCTv3TradeBot.class);
+
+	public enum State implements TradeBot.StateNameAndValueSupplier {
+		BOB_WAITING_FOR_AT_CONFIRM(10, false, false),
+		BOB_WAITING_FOR_MESSAGE(15, true, true),
+		BOB_WAITING_FOR_AT_REDEEM(25, true, true),
+		BOB_DONE(30, false, false),
+		BOB_REFUNDED(35, false, false),
+
+		ALICE_WAITING_FOR_AT_LOCK(85, true, true),
+		ALICE_DONE(95, false, false),
+		ALICE_REFUNDING_A(105, true, true),
+		ALICE_REFUNDED(110, false, false);
+
+		private static final Map<Integer, State> map = stream(State.values()).collect(toMap(state -> state.value, state -> state));
+
+		public final int value;
+		public final boolean requiresAtData;
+		public final boolean requiresTradeData;
+
+		State(int value, boolean requiresAtData, boolean requiresTradeData) {
+			this.value = value;
+			this.requiresAtData = requiresAtData;
+			this.requiresTradeData = requiresTradeData;
+		}
+
+		public static State valueOf(int value) {
+			return map.get(value);
+		}
+
+		@Override
+		public String getState() {
+			return this.name();
+		}
+
+		@Override
+		public int getStateValue() {
+			return this.value;
+		}
+	}
+
+	/** Maximum time Bob waits for his AT creation transaction to be confirmed into a block. (milliseconds) */
+	private static final long MAX_AT_CONFIRMATION_PERIOD = 24 * 60 * 60 * 1000L; // ms
+
+	private static RavencoinACCTv3TradeBot instance;
+
+	private final List<String> endStates = Arrays.asList(State.BOB_DONE, State.BOB_REFUNDED, State.ALICE_DONE, State.ALICE_REFUNDING_A, State.ALICE_REFUNDED).stream()
+			.map(State::name)
+			.collect(Collectors.toUnmodifiableList());
+
+	private RavencoinACCTv3TradeBot() {
+	}
+
+	public static synchronized RavencoinACCTv3TradeBot getInstance() {
+		if (instance == null)
+			instance = new RavencoinACCTv3TradeBot();
+
+		return instance;
+	}
+
+	@Override
+	public List<String> getEndStates() {
+		return this.endStates;
+	}
+
+	/**
+	 * Creates a new trade-bot entry from the "Bob" viewpoint, i.e. OFFERing QORT in exchange for RVN.
+	 * <p>
+	 * Generates:
+	 * <ul>
+	 * 	<li>new 'trade' private key</li>
+	 * </ul>
+	 * Derives:
+	 * <ul>
+	 * 	<li>'native' (as in Qortal) public key, public key hash, address (starting with Q)</li>
+	 * 	<li>'foreign' (as in Ravencoin) public key, public key hash</li>
+	 * </ul>
+	 * A Qortal AT is then constructed including the following as constants in the 'data segment':
+	 * <ul>
+	 * 	<li>'native'/Qortal 'trade' address - used as a MESSAGE contact</li>
+	 * 	<li>'foreign'/Ravencoin public key hash - used by Alice's P2SH scripts to allow redeem</li>
+	 * 	<li>QORT amount on offer by Bob</li>
+	 * 	<li>RVN amount expected in return by Bob (from Alice)</li>
+	 * 	<li>trading timeout, in case things go wrong and everyone needs to refund</li>
+	 * </ul>
+	 * Returns a DEPLOY_AT transaction that needs to be signed and broadcast to the Qortal network.
+	 * <p>
+	 * Trade-bot will wait for Bob's AT to be deployed before taking next step.
+	 * <p>
+	 * @param repository
+	 * @param tradeBotCreateRequest
+	 * @return raw, unsigned DEPLOY_AT transaction
+	 * @throws DataException
+	 */
+	public byte[] createTrade(Repository repository, TradeBotCreateRequest tradeBotCreateRequest) throws DataException {
+		byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
+
+		byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
+		byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
+		String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
+
+		byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
+		byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
+
+		// Convert Ravencoin receiving address into public key hash (we only support P2PKH at this time)
+		Address ravencoinReceivingAddress;
+		try {
+			ravencoinReceivingAddress = Address.fromString(Ravencoin.getInstance().getNetworkParameters(), tradeBotCreateRequest.receivingAddress);
+		} catch (AddressFormatException e) {
+			throw new DataException("Unsupported Ravencoin receiving address: " + tradeBotCreateRequest.receivingAddress);
+		}
+		if (ravencoinReceivingAddress.getOutputScriptType() != ScriptType.P2PKH)
+			throw new DataException("Unsupported Ravencoin receiving address: " + tradeBotCreateRequest.receivingAddress);
+
+		byte[] ravencoinReceivingAccountInfo = ravencoinReceivingAddress.getHash();
+
+		PublicKeyAccount creator = new PublicKeyAccount(repository, tradeBotCreateRequest.creatorPublicKey);
+
+		// Deploy AT
+		long timestamp = NTP.getTime();
+		byte[] reference = creator.getLastReference();
+		long fee = 0L;
+		byte[] signature = null;
+		BaseTransactionData baseTransactionData = new BaseTransactionData(timestamp, Group.NO_GROUP, reference, creator.getPublicKey(), fee, signature);
+
+		String name = "QORT/RVN ACCT";
+		String description = "QORT/RVN cross-chain trade";
+		String aTType = "ACCT";
+		String tags = "ACCT QORT RVN";
+		byte[] creationBytes = RavencoinACCTv3.buildQortalAT(tradeNativeAddress, tradeForeignPublicKeyHash, tradeBotCreateRequest.qortAmount,
+				tradeBotCreateRequest.foreignAmount, tradeBotCreateRequest.tradeTimeout);
+		long amount = tradeBotCreateRequest.fundingQortAmount;
+
+		DeployAtTransactionData deployAtTransactionData = new DeployAtTransactionData(baseTransactionData, name, description, aTType, tags, creationBytes, amount, Asset.QORT);
+
+		DeployAtTransaction deployAtTransaction = new DeployAtTransaction(repository, deployAtTransactionData);
+		fee = deployAtTransaction.calcRecommendedFee();
+		deployAtTransactionData.setFee(fee);
+
+		DeployAtTransaction.ensureATAddress(deployAtTransactionData);
+		String atAddress = deployAtTransactionData.getAtAddress();
+
+		TradeBotData tradeBotData =  new TradeBotData(tradePrivateKey, RavencoinACCTv3.NAME,
+				State.BOB_WAITING_FOR_AT_CONFIRM.name(), State.BOB_WAITING_FOR_AT_CONFIRM.value,
+				creator.getAddress(), atAddress, timestamp, tradeBotCreateRequest.qortAmount,
+				tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
+				null, null,
+				SupportedBlockchain.RAVENCOIN.name(),
+				tradeForeignPublicKey, tradeForeignPublicKeyHash,
+				tradeBotCreateRequest.foreignAmount, null, null, null, ravencoinReceivingAccountInfo);
+
+		TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Built AT %s. Waiting for deployment", atAddress));
+
+		// Attempt to backup the trade bot data
+		TradeBot.backupTradeBotData(repository, null);
+
+		// Return to user for signing and broadcast as we don't have their Qortal private key
+		try {
+			return DeployAtTransactionTransformer.toBytes(deployAtTransactionData);
+		} catch (TransformationException e) {
+			throw new DataException("Failed to transform DEPLOY_AT transaction?", e);
+		}
+	}
+
+	/**
+	 * Creates a trade-bot entry from the 'Alice' viewpoint, i.e. matching RVN to an existing offer.
+	 * <p>
+	 * Requires a chosen trade offer from Bob, passed by <tt>crossChainTradeData</tt>
+	 * and access to a Ravencoin wallet via <tt>xprv58</tt>.
+	 * <p>
+	 * The <tt>crossChainTradeData</tt> contains the current trade offer state
+	 * as extracted from the AT's data segment.
+	 * <p>
+	 * Access to a funded wallet is via a Ravencoin BIP32 hierarchical deterministic key,
+	 * passed via <tt>xprv58</tt>.
+	 * <b>This key will be stored in your node's database</b>
+	 * to allow trade-bot to create/fund the necessary P2SH transactions!
+	 * However, due to the nature of BIP32 keys, it is possible to give the trade-bot
+	 * only a subset of wallet access (see BIP32 for more details).
+	 * <p>
+	 * As an example, the xprv58 can be extract from a <i>legacy, password-less</i>
+	 * Electrum wallet by going to the console tab and entering:<br>
+	 * <tt>wallet.keystore.xprv</tt><br>
+	 * which should result in a base58 string starting with either 'xprv' (for Ravencoin main-net)
+	 * or 'tprv' for (Ravencoin test-net).
+	 * <p>
+	 * It is envisaged that the value in <tt>xprv58</tt> will actually come from a Qortal-UI-managed wallet.
+	 * <p>
+	 * If sufficient funds are available, <b>this method will actually fund the P2SH-A</b>
+	 * with the Ravencoin amount expected by 'Bob'.
+	 * <p>
+	 * If the Ravencoin transaction is successfully broadcast to the network then
+	 * we also send a MESSAGE to Bob's trade-bot to let them know.
+	 * <p>
+	 * The trade-bot entry is saved to the repository and the cross-chain trading process commences.
+	 * <p>
+	 * @param repository
+	 * @param crossChainTradeData chosen trade OFFER that Alice wants to match
+	 * @param xprv58 funded wallet xprv in base58
+	 * @return true if P2SH-A funding transaction successfully broadcast to Ravencoin network, false otherwise
+	 * @throws DataException
+	 */
+	public ResponseResult startResponse(Repository repository, ATData atData, ACCT acct, CrossChainTradeData crossChainTradeData, String xprv58, String receivingAddress) throws DataException {
+		byte[] tradePrivateKey = TradeBot.generateTradePrivateKey();
+		byte[] secretA = TradeBot.generateSecret();
+		byte[] hashOfSecretA = Crypto.hash160(secretA);
+
+		byte[] tradeNativePublicKey = TradeBot.deriveTradeNativePublicKey(tradePrivateKey);
+		byte[] tradeNativePublicKeyHash = Crypto.hash160(tradeNativePublicKey);
+		String tradeNativeAddress = Crypto.toAddress(tradeNativePublicKey);
+
+		byte[] tradeForeignPublicKey = TradeBot.deriveTradeForeignPublicKey(tradePrivateKey);
+		byte[] tradeForeignPublicKeyHash = Crypto.hash160(tradeForeignPublicKey);
+		byte[] receivingPublicKeyHash = Base58.decode(receivingAddress); // Actually the whole address, not just PKH
+
+		// We need to generate lockTime-A: add tradeTimeout to now
+		long now = NTP.getTime();
+		int lockTimeA = crossChainTradeData.tradeTimeout * 60 + (int) (now / 1000L);
+
+		TradeBotData tradeBotData =  new TradeBotData(tradePrivateKey, RavencoinACCTv3.NAME,
+				State.ALICE_WAITING_FOR_AT_LOCK.name(), State.ALICE_WAITING_FOR_AT_LOCK.value,
+				receivingAddress, crossChainTradeData.qortalAtAddress, now, crossChainTradeData.qortAmount,
+				tradeNativePublicKey, tradeNativePublicKeyHash, tradeNativeAddress,
+				secretA, hashOfSecretA,
+				SupportedBlockchain.RAVENCOIN.name(),
+				tradeForeignPublicKey, tradeForeignPublicKeyHash,
+				crossChainTradeData.expectedForeignAmount, xprv58, null, lockTimeA, receivingPublicKeyHash);
+
+		// Attempt to backup the trade bot data
+		// Include tradeBotData as an additional parameter, since it's not in the repository yet
+		TradeBot.backupTradeBotData(repository, Arrays.asList(tradeBotData));
+
+		// Check we have enough funds via xprv58 to fund P2SH to cover expectedForeignAmount
+		long p2shFee;
+		try {
+			p2shFee = Ravencoin.getInstance().getP2shFee(now);
+		} catch (ForeignBlockchainException e) {
+			LOGGER.debug("Couldn't estimate Ravencoin fees?");
+			return ResponseResult.NETWORK_ISSUE;
+		}
+
+		// Fee for redeem/refund is subtracted from P2SH-A balance.
+		// Do not include fee for funding transaction as this is covered by buildSpend()
+		long amountA = crossChainTradeData.expectedForeignAmount + p2shFee /*redeeming/refunding P2SH-A*/;
+
+		// P2SH-A to be funded
+		byte[] redeemScriptBytes = BitcoinyHTLC.buildScript(tradeForeignPublicKeyHash, lockTimeA, crossChainTradeData.creatorForeignPKH, hashOfSecretA);
+		String p2shAddress = Ravencoin.getInstance().deriveP2shAddress(redeemScriptBytes);
+
+		// Build transaction for funding P2SH-A
+		Transaction p2shFundingTransaction = Ravencoin.getInstance().buildSpend(tradeBotData.getForeignKey(), p2shAddress, amountA);
+		if (p2shFundingTransaction == null) {
+			LOGGER.debug("Unable to build P2SH-A funding transaction - lack of funds?");
+			return ResponseResult.BALANCE_ISSUE;
+		}
+
+		try {
+			Ravencoin.getInstance().broadcastTransaction(p2shFundingTransaction);
+		} catch (ForeignBlockchainException e) {
+			// We couldn't fund P2SH-A at this time
+			LOGGER.debug("Couldn't broadcast P2SH-A funding transaction?");
+			return ResponseResult.NETWORK_ISSUE;
+		}
+
+		// Attempt to send MESSAGE to Bob's Qortal trade address
+		byte[] messageData = RavencoinACCTv3.buildOfferMessage(tradeBotData.getTradeForeignPublicKeyHash(), tradeBotData.getHashOfSecret(), tradeBotData.getLockTimeA());
+		String messageRecipient = crossChainTradeData.qortalCreatorTradeAddress;
+
+		boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
+		if (!isMessageAlreadySent) {
+			PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
+			MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
+
+			messageTransaction.computeNonce();
+			messageTransaction.sign(sender);
+
+			// reset repository state to prevent deadlock
+			repository.discardChanges();
+			ValidationResult result = messageTransaction.importAsUnconfirmed();
+
+			if (result != ValidationResult.OK) {
+				LOGGER.warn(() -> String.format("Unable to send MESSAGE to Bob's trade-bot %s: %s", messageRecipient, result.name()));
+				return ResponseResult.NETWORK_ISSUE;
+			}
+		}
+
+		TradeBot.updateTradeBotState(repository, tradeBotData, () -> String.format("Funding P2SH-A %s. Messaged Bob. Waiting for AT-lock", p2shAddress));
+
+		return ResponseResult.OK;
+	}
+
+	@Override
+	public boolean canDelete(Repository repository, TradeBotData tradeBotData) throws DataException {
+		State tradeBotState = State.valueOf(tradeBotData.getStateValue());
+		if (tradeBotState == null)
+			return true;
+
+		// If the AT doesn't exist then we might as well let the user tidy up
+		if (!repository.getATRepository().exists(tradeBotData.getAtAddress()))
+			return true;
+
+		switch (tradeBotState) {
+			case BOB_WAITING_FOR_AT_CONFIRM:
+			case ALICE_DONE:
+			case BOB_DONE:
+			case ALICE_REFUNDED:
+			case BOB_REFUNDED:
+			case ALICE_REFUNDING_A:
+				return true;
+
+			default:
+				return false;
+		}
+	}
+
+	@Override
+	public void progress(Repository repository, TradeBotData tradeBotData) throws DataException, ForeignBlockchainException {
+		State tradeBotState = State.valueOf(tradeBotData.getStateValue());
+		if (tradeBotState == null) {
+			LOGGER.info(() -> String.format("Trade-bot entry for AT %s has invalid state?", tradeBotData.getAtAddress()));
+			return;
+		}
+
+		ATData atData = null;
+		CrossChainTradeData tradeData = null;
+
+		if (tradeBotState.requiresAtData) {
+			// Attempt to fetch AT data
+			atData = repository.getATRepository().fromATAddress(tradeBotData.getAtAddress());
+			if (atData == null) {
+				LOGGER.debug(() -> String.format("Unable to fetch trade AT %s from repository", tradeBotData.getAtAddress()));
+				return;
+			}
+
+			if (tradeBotState.requiresTradeData) {
+				tradeData = RavencoinACCTv3.getInstance().populateTradeData(repository, atData);
+				if (tradeData == null) {
+					LOGGER.warn(() -> String.format("Unable to fetch ACCT trade data for AT %s from repository", tradeBotData.getAtAddress()));
+					return;
+				}
+			}
+		}
+
+		switch (tradeBotState) {
+			case BOB_WAITING_FOR_AT_CONFIRM:
+				handleBobWaitingForAtConfirm(repository, tradeBotData);
+				break;
+
+			case BOB_WAITING_FOR_MESSAGE:
+				TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
+				handleBobWaitingForMessage(repository, tradeBotData, atData, tradeData);
+				break;
+
+			case ALICE_WAITING_FOR_AT_LOCK:
+				TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
+				handleAliceWaitingForAtLock(repository, tradeBotData, atData, tradeData);
+				break;
+
+			case BOB_WAITING_FOR_AT_REDEEM:
+				TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
+				handleBobWaitingForAtRedeem(repository, tradeBotData, atData, tradeData);
+				break;
+
+			case ALICE_DONE:
+			case BOB_DONE:
+				break;
+
+			case ALICE_REFUNDING_A:
+				TradeBot.getInstance().updatePresence(repository, tradeBotData, tradeData);
+				handleAliceRefundingP2shA(repository, tradeBotData, atData, tradeData);
+				break;
+
+			case ALICE_REFUNDED:
+			case BOB_REFUNDED:
+				break;
+		}
+	}
+
+	/**
+	 * Trade-bot is waiting for Bob's AT to deploy.
+	 * <p>
+	 * If AT is deployed, then trade-bot's next step is to wait for MESSAGE from Alice.
+	 */
+	private void handleBobWaitingForAtConfirm(Repository repository, TradeBotData tradeBotData) throws DataException {
+		if (!repository.getATRepository().exists(tradeBotData.getAtAddress())) {
+			if (NTP.getTime() - tradeBotData.getTimestamp() <= MAX_AT_CONFIRMATION_PERIOD)
+				return;
+
+			// We've waited ages for AT to be confirmed into a block but something has gone awry.
+			// After this long we assume transaction loss so give up with trade-bot entry too.
+			tradeBotData.setState(State.BOB_REFUNDED.name());
+			tradeBotData.setStateValue(State.BOB_REFUNDED.value);
+			tradeBotData.setTimestamp(NTP.getTime());
+			// We delete trade-bot entry here instead of saving, hence not using updateTradeBotState()
+			repository.getCrossChainRepository().delete(tradeBotData.getTradePrivateKey());
+			repository.saveChanges();
+
+			LOGGER.info(() -> String.format("AT %s never confirmed. Giving up on trade", tradeBotData.getAtAddress()));
+			TradeBot.notifyStateChange(tradeBotData);
+			return;
+		}
+
+		TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_MESSAGE,
+				() -> String.format("AT %s confirmed ready. Waiting for trade message", tradeBotData.getAtAddress()));
+	}
+
+	/**
+	 * Trade-bot is waiting for MESSAGE from Alice's trade-bot, containing Alice's trade info.
+	 * <p>
+	 * It's possible Bob has cancelling his trade offer, receiving an automatic QORT refund,
+	 * in which case trade-bot is done with this specific trade and finalizes on refunded state.
+	 * <p>
+	 * Assuming trade is still on offer, trade-bot checks the contents of MESSAGE from Alice's trade-bot.
+	 * <p>
+	 * Details from Alice are used to derive P2SH-A address and this is checked for funding balance.
+	 * <p>
+	 * Assuming P2SH-A has at least expected Ravencoin balance,
+	 * Bob's trade-bot constructs a zero-fee, PoW MESSAGE to send to Bob's AT with more trade details.
+	 * <p>
+	 * On processing this MESSAGE, Bob's AT should switch into 'TRADE' mode and only trade with Alice.
+	 * <p>
+	 * Trade-bot's next step is to wait for Alice to redeem the AT, which will allow Bob to
+	 * extract secret-A needed to redeem Alice's P2SH.
+	 * @throws ForeignBlockchainException
+	 */
+	private void handleBobWaitingForMessage(Repository repository, TradeBotData tradeBotData,
+			ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
+		// If AT has finished then Bob likely cancelled his trade offer
+		if (atData.getIsFinished()) {
+			TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
+					() -> String.format("AT %s cancelled - trading aborted", tradeBotData.getAtAddress()));
+			return;
+		}
+
+		Ravencoin ravencoin = Ravencoin.getInstance();
+
+		String address = tradeBotData.getTradeNativeAddress();
+		List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, address, null, null, null);
+
+		for (MessageTransactionData messageTransactionData : messageTransactionsData) {
+			if (messageTransactionData.isText())
+				continue;
+
+			// We're expecting: HASH160(secret-A), Alice's Ravencoin pubkeyhash and lockTime-A
+			byte[] messageData = messageTransactionData.getData();
+			RavencoinACCTv3.OfferMessageData offerMessageData = RavencoinACCTv3.extractOfferMessageData(messageData);
+			if (offerMessageData == null)
+				continue;
+
+			byte[] aliceForeignPublicKeyHash = offerMessageData.partnerRavencoinPKH;
+			byte[] hashOfSecretA = offerMessageData.hashOfSecretA;
+			int lockTimeA = (int) offerMessageData.lockTimeA;
+			long messageTimestamp = messageTransactionData.getTimestamp();
+			int refundTimeout = RavencoinACCTv3.calcRefundTimeout(messageTimestamp, lockTimeA);
+
+			// Determine P2SH-A address and confirm funded
+			byte[] redeemScriptA = BitcoinyHTLC.buildScript(aliceForeignPublicKeyHash, lockTimeA, tradeBotData.getTradeForeignPublicKeyHash(), hashOfSecretA);
+			String p2shAddressA = ravencoin.deriveP2shAddress(redeemScriptA);
+
+			long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
+			long p2shFee = Ravencoin.getInstance().getP2shFee(feeTimestamp);
+			final long minimumAmountA = tradeBotData.getForeignAmount() + p2shFee;
+
+			BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(ravencoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
+
+			switch (htlcStatusA) {
+				case UNFUNDED:
+				case FUNDING_IN_PROGRESS:
+					// There might be another MESSAGE from someone else with an actually funded P2SH-A...
+					continue;
+
+				case REDEEM_IN_PROGRESS:
+				case REDEEMED:
+					// We've already redeemed this?
+					TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
+							() -> String.format("P2SH-A %s already spent? Assuming trade complete", p2shAddressA));
+					return;
+
+				case REFUND_IN_PROGRESS:
+				case REFUNDED:
+					// This P2SH-A is burnt, but there might be another MESSAGE from someone else with an actually funded P2SH-A...
+					continue;
+
+				case FUNDED:
+					// Fall-through out of switch...
+					break;
+			}
+
+			// Good to go - send MESSAGE to AT
+
+			String aliceNativeAddress = Crypto.toAddress(messageTransactionData.getCreatorPublicKey());
+
+			// Build outgoing message, padding each part to 32 bytes to make it easier for AT to consume
+			byte[] outgoingMessageData = RavencoinACCTv3.buildTradeMessage(aliceNativeAddress, aliceForeignPublicKeyHash, hashOfSecretA, lockTimeA, refundTimeout);
+			String messageRecipient = tradeBotData.getAtAddress();
+
+			boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, outgoingMessageData);
+			if (!isMessageAlreadySent) {
+				PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
+				MessageTransaction outgoingMessageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, outgoingMessageData, false, false);
+
+				outgoingMessageTransaction.computeNonce();
+				outgoingMessageTransaction.sign(sender);
+
+				// reset repository state to prevent deadlock
+				repository.discardChanges();
+				ValidationResult result = outgoingMessageTransaction.importAsUnconfirmed();
+
+				if (result != ValidationResult.OK) {
+					LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
+					return;
+				}
+			}
+
+			TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_WAITING_FOR_AT_REDEEM,
+					() -> String.format("Locked AT %s to %s. Waiting for AT redeem", tradeBotData.getAtAddress(), aliceNativeAddress));
+
+			return;
+		}
+	}
+
+	/**
+	 * Trade-bot is waiting for Bob's AT to switch to TRADE mode and lock trade to Alice only.
+	 * <p>
+	 * It's possible that Bob has cancelled his trade offer in the mean time, or that somehow
+	 * this process has taken so long that we've reached P2SH-A's locktime, or that someone else
+	 * has managed to trade with Bob. In any of these cases, trade-bot switches to begin the refunding process.
+	 * <p>
+	 * Assuming Bob's AT is locked to Alice, trade-bot checks AT's state data to make sure it is correct.
+	 * <p>
+	 * If all is well, trade-bot then redeems AT using Alice's secret-A, releasing Bob's QORT to Alice.
+	 * <p>
+	 * In revealing a valid secret-A, Bob can then redeem the RVN funds from P2SH-A.
+	 * <p>
+	 * @throws ForeignBlockchainException
+	 */
+	private void handleAliceWaitingForAtLock(Repository repository, TradeBotData tradeBotData,
+			ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
+		if (aliceUnexpectedState(repository, tradeBotData, atData, crossChainTradeData))
+			return;
+
+		Ravencoin ravencoin = Ravencoin.getInstance();
+		int lockTimeA = tradeBotData.getLockTimeA();
+
+		// Refund P2SH-A if we've passed lockTime-A
+		if (NTP.getTime() >= lockTimeA * 1000L) {
+			byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
+			String p2shAddressA = ravencoin.deriveP2shAddress(redeemScriptA);
+
+			long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
+			long p2shFee = Ravencoin.getInstance().getP2shFee(feeTimestamp);
+			long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
+
+			BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(ravencoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
+
+			switch (htlcStatusA) {
+				case UNFUNDED:
+				case FUNDING_IN_PROGRESS:
+				case FUNDED:
+					break;
+
+				case REDEEM_IN_PROGRESS:
+				case REDEEMED:
+					// Already redeemed?
+					TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
+							() -> String.format("P2SH-A %s already spent? Assuming trade completed", p2shAddressA));
+					return;
+
+				case REFUND_IN_PROGRESS:
+				case REFUNDED:
+					TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
+							() -> String.format("P2SH-A %s already refunded. Trade aborted", p2shAddressA));
+					return;
+
+			}
+
+			TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
+					() -> atData.getIsFinished()
+					? String.format("AT %s cancelled. Refunding P2SH-A %s - aborting trade", tradeBotData.getAtAddress(), p2shAddressA)
+					: String.format("LockTime-A reached, refunding P2SH-A %s - aborting trade", p2shAddressA));
+
+			return;
+		}
+
+		// We're waiting for AT to be in TRADE mode
+		if (crossChainTradeData.mode != AcctMode.TRADING)
+			return;
+
+		// AT is in TRADE mode and locked to us as checked by aliceUnexpectedState() above
+
+		// Find our MESSAGE to AT from previous state
+		List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(tradeBotData.getTradeNativePublicKey(),
+				crossChainTradeData.qortalCreatorTradeAddress, null, null, null);
+		if (messageTransactionsData == null || messageTransactionsData.isEmpty()) {
+			LOGGER.warn(() -> String.format("Unable to find our message to trade creator %s?", crossChainTradeData.qortalCreatorTradeAddress));
+			return;
+		}
+
+		long recipientMessageTimestamp = messageTransactionsData.get(0).getTimestamp();
+		int refundTimeout = RavencoinACCTv3.calcRefundTimeout(recipientMessageTimestamp, lockTimeA);
+
+		// Our calculated refundTimeout should match AT's refundTimeout
+		if (refundTimeout != crossChainTradeData.refundTimeout) {
+			LOGGER.debug(() -> String.format("Trade AT refundTimeout '%d' doesn't match our refundTimeout '%d'", crossChainTradeData.refundTimeout, refundTimeout));
+			// We'll eventually refund
+			return;
+		}
+
+		// We're good to redeem AT
+
+		// Send 'redeem' MESSAGE to AT using both secret
+		byte[] secretA = tradeBotData.getSecret();
+		String qortalReceivingAddress = Base58.encode(tradeBotData.getReceivingAccountInfo()); // Actually contains whole address, not just PKH
+		byte[] messageData = RavencoinACCTv3.buildRedeemMessage(secretA, qortalReceivingAddress);
+		String messageRecipient = tradeBotData.getAtAddress();
+
+		boolean isMessageAlreadySent = repository.getMessageRepository().exists(tradeBotData.getTradeNativePublicKey(), messageRecipient, messageData);
+		if (!isMessageAlreadySent) {
+			PrivateKeyAccount sender = new PrivateKeyAccount(repository, tradeBotData.getTradePrivateKey());
+			MessageTransaction messageTransaction = MessageTransaction.build(repository, sender, Group.NO_GROUP, messageRecipient, messageData, false, false);
+
+			messageTransaction.computeNonce();
+			messageTransaction.sign(sender);
+
+			// Reset repository state to prevent deadlock
+			repository.discardChanges();
+			ValidationResult result = messageTransaction.importAsUnconfirmed();
+
+			if (result != ValidationResult.OK) {
+				LOGGER.warn(() -> String.format("Unable to send MESSAGE to AT %s: %s", messageRecipient, result.name()));
+				return;
+			}
+		}
+
+		TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
+				() -> String.format("Redeeming AT %s. Funds should arrive at %s",
+						tradeBotData.getAtAddress(), qortalReceivingAddress));
+	}
+
+	/**
+	 * Trade-bot is waiting for Alice to redeem Bob's AT, thus revealing secret-A which is required to spend the RVN funds from P2SH-A.
+	 * <p>
+	 * It's possible that Bob's AT has reached its trading timeout and automatically refunded QORT back to Bob. In which case,
+	 * trade-bot is done with this specific trade and finalizes in refunded state.
+	 * <p>
+	 * Assuming trade-bot can extract a valid secret-A from Alice's MESSAGE then trade-bot uses that to redeem the RVN funds from P2SH-A
+	 * to Bob's 'foreign'/Ravencoin trade legacy-format address, as derived from trade private key.
+	 * <p>
+	 * (This could potentially be 'improved' to send RVN to any address of Bob's choosing by changing the transaction output).
+	 * <p>
+	 * If trade-bot successfully broadcasts the transaction, then this specific trade is done.
+	 * @throws ForeignBlockchainException
+	 */
+	private void handleBobWaitingForAtRedeem(Repository repository, TradeBotData tradeBotData,
+			ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
+		// AT should be 'finished' once Alice has redeemed QORT funds
+		if (!atData.getIsFinished())
+			// Not finished yet
+			return;
+
+		// If AT is REFUNDED or CANCELLED then something has gone wrong
+		if (crossChainTradeData.mode == AcctMode.REFUNDED || crossChainTradeData.mode == AcctMode.CANCELLED) {
+			// Alice hasn't redeemed the QORT, so there is no point in trying to redeem the RVN
+			TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_REFUNDED,
+					() -> String.format("AT %s has auto-refunded - trade aborted", tradeBotData.getAtAddress()));
+
+			return;
+		}
+
+		byte[] secretA = RavencoinACCTv3.getInstance().findSecretA(repository, crossChainTradeData);
+		if (secretA == null) {
+			LOGGER.debug(() -> String.format("Unable to find secret-A from redeem message to AT %s?", tradeBotData.getAtAddress()));
+			return;
+		}
+
+		// Use secret-A to redeem P2SH-A
+
+		Ravencoin ravencoin = Ravencoin.getInstance();
+
+		byte[] receivingAccountInfo = tradeBotData.getReceivingAccountInfo();
+		int lockTimeA = crossChainTradeData.lockTimeA;
+		byte[] redeemScriptA = BitcoinyHTLC.buildScript(crossChainTradeData.partnerForeignPKH, lockTimeA, crossChainTradeData.creatorForeignPKH, crossChainTradeData.hashOfSecretA);
+		String p2shAddressA = ravencoin.deriveP2shAddress(redeemScriptA);
+
+		// Fee for redeem/refund is subtracted from P2SH-A balance.
+		long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
+		long p2shFee = Ravencoin.getInstance().getP2shFee(feeTimestamp);
+		long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
+		BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(ravencoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
+
+		switch (htlcStatusA) {
+			case UNFUNDED:
+			case FUNDING_IN_PROGRESS:
+				// P2SH-A suddenly not funded? Our best bet at this point is to hope for AT auto-refund
+				return;
+
+			case REDEEM_IN_PROGRESS:
+			case REDEEMED:
+				// Double-check that we have redeemed P2SH-A...
+				break;
+
+			case REFUND_IN_PROGRESS:
+			case REFUNDED:
+				// Wait for AT to auto-refund
+				return;
+
+			case FUNDED: {
+				Coin redeemAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
+				ECKey redeemKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
+				List<TransactionOutput> fundingOutputs = ravencoin.getUnspentOutputs(p2shAddressA);
+
+				Transaction p2shRedeemTransaction = BitcoinyHTLC.buildRedeemTransaction(ravencoin.getNetworkParameters(), redeemAmount, redeemKey,
+						fundingOutputs, redeemScriptA, secretA, receivingAccountInfo);
+
+				ravencoin.broadcastTransaction(p2shRedeemTransaction);
+				break;
+			}
+		}
+
+		String receivingAddress = ravencoin.pkhToAddress(receivingAccountInfo);
+
+		TradeBot.updateTradeBotState(repository, tradeBotData, State.BOB_DONE,
+				() -> String.format("P2SH-A %s redeemed. Funds should arrive at %s", tradeBotData.getAtAddress(), receivingAddress));
+	}
+
+	/**
+	 * Trade-bot is attempting to refund P2SH-A.
+	 * @throws ForeignBlockchainException
+	 */
+	private void handleAliceRefundingP2shA(Repository repository, TradeBotData tradeBotData,
+			ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
+		int lockTimeA = tradeBotData.getLockTimeA();
+
+		// We can't refund P2SH-A until lockTime-A has passed
+		if (NTP.getTime() <= lockTimeA * 1000L)
+			return;
+
+		Ravencoin ravencoin = Ravencoin.getInstance();
+
+		// We can't refund P2SH-A until median block time has passed lockTime-A (see BIP113)
+		int medianBlockTime = ravencoin.getMedianBlockTime();
+		if (medianBlockTime <= lockTimeA)
+			return;
+
+		byte[] redeemScriptA = BitcoinyHTLC.buildScript(tradeBotData.getTradeForeignPublicKeyHash(), lockTimeA, crossChainTradeData.creatorForeignPKH, tradeBotData.getHashOfSecret());
+		String p2shAddressA = ravencoin.deriveP2shAddress(redeemScriptA);
+
+		// Fee for redeem/refund is subtracted from P2SH-A balance.
+		long feeTimestamp = calcFeeTimestamp(lockTimeA, crossChainTradeData.tradeTimeout);
+		long p2shFee = Ravencoin.getInstance().getP2shFee(feeTimestamp);
+		long minimumAmountA = crossChainTradeData.expectedForeignAmount + p2shFee;
+		BitcoinyHTLC.Status htlcStatusA = BitcoinyHTLC.determineHtlcStatus(ravencoin.getBlockchainProvider(), p2shAddressA, minimumAmountA);
+
+		switch (htlcStatusA) {
+			case UNFUNDED:
+			case FUNDING_IN_PROGRESS:
+				// Still waiting for P2SH-A to be funded...
+				return;
+
+			case REDEEM_IN_PROGRESS:
+			case REDEEMED:
+				// Too late!
+				TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
+						() -> String.format("P2SH-A %s already spent!", p2shAddressA));
+				return;
+
+			case REFUND_IN_PROGRESS:
+			case REFUNDED:
+				break;
+
+			case FUNDED:{
+				Coin refundAmount = Coin.valueOf(crossChainTradeData.expectedForeignAmount);
+				ECKey refundKey = ECKey.fromPrivate(tradeBotData.getTradePrivateKey());
+				List<TransactionOutput> fundingOutputs = ravencoin.getUnspentOutputs(p2shAddressA);
+
+				// Determine receive address for refund
+				String receiveAddress = ravencoin.getUnusedReceiveAddress(tradeBotData.getForeignKey());
+				Address receiving = Address.fromString(ravencoin.getNetworkParameters(), receiveAddress);
+
+				Transaction p2shRefundTransaction = BitcoinyHTLC.buildRefundTransaction(ravencoin.getNetworkParameters(), refundAmount, refundKey,
+						fundingOutputs, redeemScriptA, lockTimeA, receiving.getHash());
+
+				ravencoin.broadcastTransaction(p2shRefundTransaction);
+				break;
+			}
+		}
+
+		TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDED,
+				() -> String.format("LockTime-A reached. Refunded P2SH-A %s. Trade aborted", p2shAddressA));
+	}
+
+	/**
+	 * Returns true if Alice finds AT unexpectedly cancelled, refunded, redeemed or locked to someone else.
+	 * <p>
+	 * Will automatically update trade-bot state to <tt>ALICE_REFUNDING_A</tt> or <tt>ALICE_DONE</tt> as necessary.
+	 * 
+	 * @throws DataException
+	 * @throws ForeignBlockchainException
+	 */
+	private boolean aliceUnexpectedState(Repository repository, TradeBotData tradeBotData,
+			ATData atData, CrossChainTradeData crossChainTradeData) throws DataException, ForeignBlockchainException {
+		// This is OK
+		if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.OFFERING)
+			return false;
+
+		boolean isAtLockedToUs = tradeBotData.getTradeNativeAddress().equals(crossChainTradeData.qortalPartnerAddress);
+
+		if (!atData.getIsFinished() && crossChainTradeData.mode == AcctMode.TRADING)
+			if (isAtLockedToUs) {
+				// AT is trading with us - OK
+				return false;
+			} else {
+				TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
+						() -> String.format("AT %s trading with someone else: %s. Refunding & aborting trade", tradeBotData.getAtAddress(), crossChainTradeData.qortalPartnerAddress));
+
+				return true;
+			}
+
+		if (atData.getIsFinished() && crossChainTradeData.mode == AcctMode.REDEEMED && isAtLockedToUs) {
+			// We've redeemed already?
+			TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_DONE,
+					() -> String.format("AT %s already redeemed by us. Trade completed", tradeBotData.getAtAddress()));
+		} else {
+			// Any other state is not good, so start defensive refund
+			TradeBot.updateTradeBotState(repository, tradeBotData, State.ALICE_REFUNDING_A,
+					() -> String.format("AT %s cancelled/refunded/redeemed by someone else/invalid state. Refunding & aborting trade", tradeBotData.getAtAddress()));
+		}
+
+		return true;
+	}
+
+	private long calcFeeTimestamp(int lockTimeA, int tradeTimeout) {
+		return (lockTimeA - tradeTimeout * 60) * 1000L;
+	}
+
+}

src/main/java/org/qortal/controller/tradebot/TradeBot.java

@@ -2,41 +2,38 @@ package org.qortal.controller.tradebot;
 
 import java.awt.TrayIcon.MessageType;
 import java.security.SecureRandom;
-import java.util.Collections;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-import java.util.Random;
-import java.util.concurrent.locks.ReentrantLock;
+import java.util.*;
+import java.util.function.Supplier;
 
 import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
-import org.apache.logging.log4j.util.Supplier;
 import org.bitcoinj.core.ECKey;
 import org.qortal.account.PrivateKeyAccount;
 import org.qortal.api.model.crosschain.TradeBotCreateRequest;
 import org.qortal.controller.Controller;
+import org.qortal.controller.Synchronizer;
 import org.qortal.controller.tradebot.AcctTradeBot.ResponseResult;
 import org.qortal.crosschain.*;
+import org.qortal.crypto.Crypto;
 import org.qortal.data.at.ATData;
 import org.qortal.data.crosschain.CrossChainTradeData;
 import org.qortal.data.crosschain.TradeBotData;
-import org.qortal.data.transaction.BaseTransactionData;
-import org.qortal.data.transaction.PresenceTransactionData;
+import org.qortal.data.network.TradePresenceData;
 import org.qortal.event.Event;
 import org.qortal.event.EventBus;
 import org.qortal.event.Listener;
-import org.qortal.group.Group;
 import org.qortal.gui.SysTray;
+import org.qortal.network.Network;
+import org.qortal.network.Peer;
+import org.qortal.network.message.GetTradePresencesMessage;
+import org.qortal.network.message.Message;
+import org.qortal.network.message.TradePresencesMessage;
 import org.qortal.repository.DataException;
 import org.qortal.repository.Repository;
 import org.qortal.repository.RepositoryManager;
 import org.qortal.repository.hsqldb.HSQLDBImportExport;
 import org.qortal.settings.Settings;
-import org.qortal.transaction.PresenceTransaction;
-import org.qortal.transaction.PresenceTransaction.PresenceType;
-import org.qortal.transaction.Transaction.ValidationResult;
-import org.qortal.transform.transaction.TransactionTransformer;
+import org.qortal.utils.ByteArray;
 import org.qortal.utils.NTP;
 
 import com.google.common.primitives.Longs;
@@ -56,6 +53,15 @@ public class TradeBot implements Listener {
 	private static final Logger LOGGER = LogManager.getLogger(TradeBot.class);
 	private static final Random RANDOM = new SecureRandom();
 
+	/** Maximum lifetime of trade presence timestamp. 30 mins in ms. */
+	private static final long PRESENCE_LIFETIME = 30 * 60 * 1000L;
+	/** How soon before expiry of our own trade presence timestamp that we want to trigger renewal. 5 mins in ms. */
+	private static final long EARLY_RENEWAL_PERIOD = 5 * 60 * 1000L;
+	/** Trade presence timestamps are rounded up to this nearest interval. Bigger values improve grouping of entries in [GET_]TRADE_PRESENCES network messages. 15 mins in ms. */
+	private static final long EXPIRY_ROUNDING = 15 * 60 * 1000L;
+	/** How often we want to broadcast our list of all known trade presences to peers. 5 mins in ms. */
+	private static final long PRESENCE_BROADCAST_INTERVAL = 5 * 60 * 1000L;
+
 	public interface StateNameAndValueSupplier {
 		public String getState();
 		public int getStateValue();
@@ -73,20 +79,39 @@ public class TradeBot implements Listener {
 		}
 	}
 
+	public static class TradePresenceEvent implements Event {
+		private final TradePresenceData tradePresenceData;
+
+		public TradePresenceEvent(TradePresenceData tradePresenceData) {
+			this.tradePresenceData = tradePresenceData;
+		}
+
+		public TradePresenceData getTradePresenceData() {
+			return this.tradePresenceData;
+		}
+	}
+
 	private static final Map<Class<? extends ACCT>, Supplier<AcctTradeBot>> acctTradeBotSuppliers = new HashMap<>();
 	static {
 		acctTradeBotSuppliers.put(BitcoinACCTv1.class, BitcoinACCTv1TradeBot::getInstance);
+		acctTradeBotSuppliers.put(BitcoinACCTv3.class, BitcoinACCTv3TradeBot::getInstance);
 		acctTradeBotSuppliers.put(LitecoinACCTv1.class, LitecoinACCTv1TradeBot::getInstance);
-		acctTradeBotSuppliers.put(LitecoinACCTv2.class, LitecoinACCTv2TradeBot::getInstance);
 		acctTradeBotSuppliers.put(LitecoinACCTv3.class, LitecoinACCTv3TradeBot::getInstance);
 		acctTradeBotSuppliers.put(DogecoinACCTv1.class, DogecoinACCTv1TradeBot::getInstance);
-		acctTradeBotSuppliers.put(DogecoinACCTv2.class, DogecoinACCTv2TradeBot::getInstance);
 		acctTradeBotSuppliers.put(DogecoinACCTv3.class, DogecoinACCTv3TradeBot::getInstance);
+		acctTradeBotSuppliers.put(DigibyteACCTv3.class, DigibyteACCTv3TradeBot::getInstance);
+		acctTradeBotSuppliers.put(RavencoinACCTv3.class, RavencoinACCTv3TradeBot::getInstance);
+		acctTradeBotSuppliers.put(PirateChainACCTv3.class, PirateChainACCTv3TradeBot::getInstance);
 	}
 
 	private static TradeBot instance;
 
-	private final Map<String, Long> presenceTimestampsByAtAddress = Collections.synchronizedMap(new HashMap<>());
+	private final Map<ByteArray, Long> ourTradePresenceTimestampsByPubkey = Collections.synchronizedMap(new HashMap<>());
+	private final List<TradePresenceData> pendingTradePresences = Collections.synchronizedList(new ArrayList<>());
+
+	private final Map<ByteArray, TradePresenceData> allTradePresencesByPubkey = Collections.synchronizedMap(new HashMap<>());
+	private Map<ByteArray, TradePresenceData> safeAllTradePresencesByPubkey = Collections.emptyMap();
+	private long nextTradePresenceBroadcastTimestamp = 0L;
 
 	private TradeBot() {
 		EventBus.INSTANCE.addListener(event -> TradeBot.getInstance().listen(event));
@@ -213,10 +238,17 @@ public class TradeBot implements Listener {
 
 	@Override
 	public void listen(Event event) {
-		if (!(event instanceof Controller.NewBlockEvent))
+		if (!(event instanceof Synchronizer.NewChainTipEvent))
+			return;
+
+		// Don't process trade bots or broadcast presence timestamps if our chain is more than 60 minutes old
+		final Long minLatestBlockTimestamp = NTP.getTime() - (60 * 60 * 1000L);
+		if (!Controller.getInstance().isUpToDate(minLatestBlockTimestamp))
 			return;
 
 		synchronized (this) {
+			expireOldPresenceTimestamps();
+
 			List<TradeBotData> allTradeBotData;
 
 			try (final Repository repository = RepositoryManager.getRepository()) {
@@ -247,6 +279,8 @@ public class TradeBot implements Listener {
 				} catch (ForeignBlockchainException e) {
 					LOGGER.warn(() -> String.format("Foreign blockchain issue processing trade-bot entry for AT %s: %s", tradeBotData.getAtAddress(), e.getMessage()));
 				}
+
+			broadcastPresenceTimestamps();
 		}
 	}
 
@@ -257,14 +291,14 @@ public class TradeBot implements Listener {
 	}
 
 	public static byte[] deriveTradeNativePublicKey(byte[] privateKey) {
-		return PrivateKeyAccount.toPublicKey(privateKey);
+		return Crypto.toPublicKey(privateKey);
 	}
 
 	public static byte[] deriveTradeForeignPublicKey(byte[] privateKey) {
 		return ECKey.fromPrivate(privateKey).getPubKey();
 	}
 
-	/*package*/ static byte[] generateSecret() {
+	/*package*/ public static byte[] generateSecret() {
 		byte[] secret = new byte[32];
 		RANDOM.nextBytes(secret);
 		return secret;
@@ -324,6 +358,33 @@ public class TradeBot implements Listener {
 	}
 
 	// PRESENCE-related
+
+	public Collection<TradePresenceData> getAllTradePresences() {
+		return this.safeAllTradePresencesByPubkey.values();
+	}
+
+	/** Trade presence timestamps expire in the 'future' so any that reach 'now' have expired and are removed. */
+	private void expireOldPresenceTimestamps() {
+		long now = NTP.getTime();
+
+		int allRemovedCount = 0;
+		synchronized (this.allTradePresencesByPubkey) {
+			int preRemoveCount = this.allTradePresencesByPubkey.size();
+			this.allTradePresencesByPubkey.values().removeIf(tradePresenceData -> tradePresenceData.getTimestamp() <= now);
+			allRemovedCount = this.allTradePresencesByPubkey.size() - preRemoveCount;
+		}
+
+		int ourRemovedCount = 0;
+		synchronized (this.ourTradePresenceTimestampsByPubkey) {
+			int preRemoveCount = this.ourTradePresenceTimestampsByPubkey.size();
+			this.ourTradePresenceTimestampsByPubkey.values().removeIf(timestamp -> timestamp < now);
+			ourRemovedCount = this.ourTradePresenceTimestampsByPubkey.size() - preRemoveCount;
+		}
+
+		if (allRemovedCount > 0)
+			LOGGER.debug("Removed {} expired trade presences, of which {} ours", allRemovedCount, ourRemovedCount);
+	}
+
 	/*package*/ void updatePresence(Repository repository, TradeBotData tradeBotData, CrossChainTradeData tradeData)
 			throws DataException {
 		String atAddress = tradeBotData.getAtAddress();
@@ -332,44 +393,292 @@ public class TradeBot implements Listener {
 		String signerAddress = tradeNativeAccount.getAddress();
 
 		/*
-		 * There's no point in Alice trying to build a PRESENCE transaction
-		 * for an AT that isn't locked to her, as other peers won't be able
-		 * to validate the PRESENCE transaction as signing public key won't
-		 * be visible.
-		 */
-		if (!signerAddress.equals(tradeData.qortalCreatorTradeAddress) && !signerAddress.equals(tradeData.qortalPartnerAddress))
-			// Signer is neither Bob, nor Alice, or trade not yet locked to Alice
+		* There's no point in Alice trying to broadcast presence for an AT that isn't locked to her,
+		* as other peers won't be able to verify as signing public key isn't yet in the AT's data segment.
+		*/
+		if (!signerAddress.equals(tradeData.qortalCreatorTradeAddress) && !signerAddress.equals(tradeData.qortalPartnerAddress)) {
+			// Signer is neither Bob, nor trade locked to Alice
+			LOGGER.trace("Can't provide trade presence for our AT {} as it's not yet locked to Alice", atAddress);
 			return;
+		}
 
 		long now = NTP.getTime();
-		long threshold = now - PresenceType.TRADE_BOT.getLifetime();
+		long newExpiry = generateExpiry(now);
+		ByteArray pubkeyByteArray = ByteArray.wrap(tradeNativeAccount.getPublicKey());
 
-		long timestamp = presenceTimestampsByAtAddress.compute(atAddress, (k, v) -> (v == null || v < threshold) ? now : v);
+		// If map entry's timestamp is missing, or within early renewal period, use the new expiry - otherwise use existing timestamp.
+		synchronized (this.ourTradePresenceTimestampsByPubkey) {
+			Long currentTimestamp = this.ourTradePresenceTimestampsByPubkey.get(pubkeyByteArray);
 
-		// If timestamp hasn't been updated then nothing to do
-		if (timestamp != now)
+			if (currentTimestamp != null && currentTimestamp - now > EARLY_RENEWAL_PERIOD) {
+				// timestamp still good
+				LOGGER.trace("Current trade presence timestamp {} still good for our trade {}", currentTimestamp, atAddress);
+				return;
+			}
+
+			this.ourTradePresenceTimestampsByPubkey.put(pubkeyByteArray, newExpiry);
+		}
+
+		// Create signature
+		byte[] signature = tradeNativeAccount.sign(Longs.toByteArray(newExpiry));
+
+		// Add new trade presence to queue to be broadcast around network
+		TradePresenceData tradePresenceData = new TradePresenceData(newExpiry, tradeNativeAccount.getPublicKey(), signature, atAddress);
+		this.pendingTradePresences.add(tradePresenceData);
+
+		this.allTradePresencesByPubkey.put(pubkeyByteArray, tradePresenceData);
+		rebuildSafeAllTradePresences();
+
+		LOGGER.trace("New trade presence timestamp {} for our trade {}", newExpiry, atAddress);
+
+		EventBus.INSTANCE.notify(new TradePresenceEvent(tradePresenceData));
+	}
+
+	private void rebuildSafeAllTradePresences() {
+		synchronized (this.allTradePresencesByPubkey) {
+			// Collect into a *new* unmodifiable map.
+			this.safeAllTradePresencesByPubkey = Map.copyOf(this.allTradePresencesByPubkey);
+		}
+	}
+
+	private void broadcastPresenceTimestamps() {
+		// If we have new trade presences that are pending broadcast, send those as a priority
+		if (!this.pendingTradePresences.isEmpty()) {
+			// Create a copy for Network to safely use in another thread
+			List<TradePresenceData> safeTradePresences;
+			synchronized (this.pendingTradePresences) {
+				safeTradePresences = List.copyOf(this.pendingTradePresences);
+				this.pendingTradePresences.clear();
+			}
+
+			LOGGER.debug("Broadcasting {} new trade presences", safeTradePresences.size());
+
+			TradePresencesMessage tradePresencesMessage = new TradePresencesMessage(safeTradePresences);
+			Network.getInstance().broadcast(peer -> tradePresencesMessage);
+
+			return;
+		}
+
+		// As we have no new trade presences, check whether it's time to do a general broadcast
+		Long now = NTP.getTime();
+		if (now == null || now < nextTradePresenceBroadcastTimestamp)
 			return;
 
-		int txGroupId = Group.NO_GROUP;
-		byte[] reference = new byte[TransactionTransformer.SIGNATURE_LENGTH];
-		byte[] creatorPublicKey = tradeNativeAccount.getPublicKey();
-		long fee = 0L;
+		nextTradePresenceBroadcastTimestamp = now + PRESENCE_BROADCAST_INTERVAL;
 
-		BaseTransactionData baseTransactionData = new BaseTransactionData(timestamp, txGroupId, reference, creatorPublicKey, fee, null);
+		List<TradePresenceData> safeTradePresences = List.copyOf(this.safeAllTradePresencesByPubkey.values());
 
-		int nonce = 0;
-		byte[] timestampSignature = tradeNativeAccount.sign(Longs.toByteArray(timestamp));
+		if (safeTradePresences.isEmpty())
+			return;
 
-		PresenceTransactionData transactionData = new PresenceTransactionData(baseTransactionData, nonce, PresenceType.TRADE_BOT, timestampSignature);
+		LOGGER.debug("Broadcasting all {} known trade presences. Next broadcast timestamp: {}",
+				safeTradePresences.size(), nextTradePresenceBroadcastTimestamp
+		);
 
-		PresenceTransaction presenceTransaction = new PresenceTransaction(repository, transactionData);
-		presenceTransaction.computeNonce();
+		GetTradePresencesMessage getTradePresencesMessage = new GetTradePresencesMessage(safeTradePresences);
+		Network.getInstance().broadcast(peer -> getTradePresencesMessage);
+	}
 
-		presenceTransaction.sign(tradeNativeAccount);
+	// Network message processing
 
-		ValidationResult result = presenceTransaction.importAsUnconfirmed();
-		if (result != ValidationResult.OK)
-			LOGGER.debug(() -> String.format("Unable to build trade-bot PRESENCE transaction for %s: %s", tradeBotData.getAtAddress(), result.name()));
+	public void onGetTradePresencesMessage(Peer peer, Message message) {
+		GetTradePresencesMessage getTradePresencesMessage = (GetTradePresencesMessage) message;
+
+		List<TradePresenceData> peersTradePresences = getTradePresencesMessage.getTradePresences();
+
+		// Create mutable copy from safe snapshot
+		Map<ByteArray, TradePresenceData> entriesUnknownToPeer = new HashMap<>(this.safeAllTradePresencesByPubkey);
+		int knownCount = entriesUnknownToPeer.size();
+
+		for (TradePresenceData peersTradePresence : peersTradePresences) {
+			ByteArray pubkeyByteArray = ByteArray.wrap(peersTradePresence.getPublicKey());
+
+			TradePresenceData ourEntry = entriesUnknownToPeer.get(pubkeyByteArray);
+
+			if (ourEntry != null && ourEntry.getTimestamp() == peersTradePresence.getTimestamp())
+				entriesUnknownToPeer.remove(pubkeyByteArray);
+		}
+
+		if (entriesUnknownToPeer.isEmpty())
+			return;
+
+		LOGGER.debug("Sending {} trade presences to peer {} after excluding their {} from known {}",
+				entriesUnknownToPeer.size(), peer, peersTradePresences.size(), knownCount
+		);
+
+		// Send complement to peer
+		List<TradePresenceData> safeTradePresences = List.copyOf(entriesUnknownToPeer.values());
+		Message responseMessage = new TradePresencesMessage(safeTradePresences);
+		if (!peer.sendMessage(responseMessage)) {
+			peer.disconnect("failed to send TRADE_PRESENCES response");
+			return;
+		}
+	}
+
+	public void onTradePresencesMessage(Peer peer, Message message) {
+		TradePresencesMessage tradePresencesMessage = (TradePresencesMessage) message;
+
+		List<TradePresenceData> peersTradePresences = tradePresencesMessage.getTradePresences();
+
+		long now = NTP.getTime();
+		// Timestamps before this are too far into the past
+		long pastThreshold = now;
+		// Timestamps after this are too far into the future
+		long futureThreshold = now + PRESENCE_LIFETIME;
+
+		Map<ByteArray, Supplier<ACCT>> acctSuppliersByCodeHash = SupportedBlockchain.getAcctMap();
+
+		int newCount = 0;
+
+		try (final Repository repository = RepositoryManager.getRepository()) {
+			for (TradePresenceData peersTradePresence : peersTradePresences) {
+				long timestamp = peersTradePresence.getTimestamp();
+
+				// Ignore if timestamp is out of bounds
+				if (timestamp < pastThreshold || timestamp > futureThreshold) {
+					if (timestamp < pastThreshold)
+						LOGGER.trace("Ignoring trade presence {} from peer {} as timestamp {} is too old vs {}",
+								peersTradePresence.getAtAddress(), peer, timestamp, pastThreshold
+								);
+					else
+						LOGGER.trace("Ignoring trade presence {} from peer {} as timestamp {} is too new vs {}",
+								peersTradePresence.getAtAddress(), peer, timestamp, pastThreshold
+						);
+
+					continue;
+				}
+
+				ByteArray pubkeyByteArray = ByteArray.wrap(peersTradePresence.getPublicKey());
+
+				// Ignore if we've previously verified this timestamp+publickey combo or sent timestamp is older
+				TradePresenceData existingTradeData = this.safeAllTradePresencesByPubkey.get(pubkeyByteArray);
+				if (existingTradeData != null && timestamp <= existingTradeData.getTimestamp()) {
+					if (timestamp == existingTradeData.getTimestamp())
+						LOGGER.trace("Ignoring trade presence {} from peer {} as we have verified timestamp {} before",
+								peersTradePresence.getAtAddress(), peer, timestamp
+						);
+					else
+						LOGGER.trace("Ignoring trade presence {} from peer {} as timestamp {} is older than latest {}",
+								peersTradePresence.getAtAddress(), peer, timestamp, existingTradeData.getTimestamp()
+						);
+
+					continue;
+				}
+
+				// Check timestamp signature
+				byte[] timestampSignature = peersTradePresence.getSignature();
+				byte[] timestampBytes = Longs.toByteArray(timestamp);
+				byte[] publicKey = peersTradePresence.getPublicKey();
+				if (!Crypto.verify(publicKey, timestampSignature, timestampBytes)) {
+					LOGGER.trace("Ignoring trade presence {} from peer {} as signature failed to verify",
+							peersTradePresence.getAtAddress(), peer
+					);
+
+					continue;
+				}
+
+				ATData atData = repository.getATRepository().fromATAddress(peersTradePresence.getAtAddress());
+				if (atData == null || atData.getIsFrozen() || atData.getIsFinished()) {
+					if (atData == null)
+						LOGGER.trace("Ignoring trade presence {} from peer {} as AT doesn't exist",
+								peersTradePresence.getAtAddress(), peer
+						);
+					else
+						LOGGER.trace("Ignoring trade presence {} from peer {} as AT is frozen or finished",
+								peersTradePresence.getAtAddress(), peer
+						);
+
+					continue;
+				}
+
+				ByteArray atCodeHash = ByteArray.wrap(atData.getCodeHash());
+				Supplier<ACCT> acctSupplier = acctSuppliersByCodeHash.get(atCodeHash);
+				if (acctSupplier == null) {
+					LOGGER.trace("Ignoring trade presence {} from peer {} as AT isn't a known ACCT?",
+							peersTradePresence.getAtAddress(), peer
+					);
+
+					continue;
+				}
+
+				CrossChainTradeData tradeData = acctSupplier.get().populateTradeData(repository, atData);
+				if (tradeData == null) {
+					LOGGER.trace("Ignoring trade presence {} from peer {} as trade data not found?",
+							peersTradePresence.getAtAddress(), peer
+					);
+
+					continue;
+				}
+
+				// Convert signer's public key to address form
+				String signerAddress = peersTradePresence.getTradeAddress();
+
+				// Signer's public key (in address form) must match Bob's / Alice's trade public key (in address form)
+				if (!signerAddress.equals(tradeData.qortalCreatorTradeAddress) && !signerAddress.equals(tradeData.qortalPartnerAddress)) {
+					LOGGER.trace("Ignoring trade presence {} from peer {} as signer isn't Alice or Bob?",
+							peersTradePresence.getAtAddress(), peer
+					);
+
+					continue;
+				}
+
+				// This is new to us
+				this.allTradePresencesByPubkey.put(pubkeyByteArray, peersTradePresence);
+				++newCount;
+
+				LOGGER.trace("Added trade presence {} from peer {} with timestamp {}",
+						peersTradePresence.getAtAddress(), peer, timestamp
+				);
+
+				EventBus.INSTANCE.notify(new TradePresenceEvent(peersTradePresence));
+			}
+		} catch (DataException e) {
+			LOGGER.error("Couldn't process TRADE_PRESENCES message due to repository issue", e);
+		}
+
+		if (newCount > 0) {
+			LOGGER.debug("New trade presences: {}", newCount);
+			rebuildSafeAllTradePresences();
+		}
+	}
+
+	public void bridgePresence(long timestamp, byte[] publicKey, byte[] signature, String atAddress) {
+		long expiry = generateExpiry(timestamp);
+		ByteArray pubkeyByteArray = ByteArray.wrap(publicKey);
+
+		TradePresenceData fakeTradePresenceData = new TradePresenceData(expiry, publicKey, signature, atAddress);
+
+		// Only bridge if trade presence expiry timestamp is newer
+		TradePresenceData computedTradePresenceData = this.allTradePresencesByPubkey.compute(pubkeyByteArray, (k, v) ->
+				v == null || v.getTimestamp() < expiry ? fakeTradePresenceData : v
+		);
+
+		if (computedTradePresenceData == fakeTradePresenceData) {
+			LOGGER.trace("Bridged PRESENCE transaction for trade {} with timestamp {}", atAddress, expiry);
+			rebuildSafeAllTradePresences();
+
+			EventBus.INSTANCE.notify(new TradePresenceEvent(fakeTradePresenceData));
+		}
+	}
+
+	/** Decorates a CrossChainTradeData object with Alice / Bob trade-bot presence timestamp, if available. */
+	public void decorateTradeDataWithPresence(CrossChainTradeData crossChainTradeData) {
+		// Match by AT address, then check for Bob vs Alice
+		this.safeAllTradePresencesByPubkey.values().stream()
+				.filter(tradePresenceData -> tradePresenceData.getAtAddress().equals(crossChainTradeData.qortalAtAddress))
+				.forEach(tradePresenceData -> {
+					String signerAddress = tradePresenceData.getTradeAddress();
+
+					// Signer's public key (in address form) must match Bob's / Alice's trade public key (in address form)
+					if (signerAddress.equals(crossChainTradeData.qortalCreatorTradeAddress))
+						crossChainTradeData.creatorPresenceExpiry = tradePresenceData.getTimestamp();
+					else if (signerAddress.equals(crossChainTradeData.qortalPartnerAddress))
+						crossChainTradeData.partnerPresenceExpiry = tradePresenceData.getTimestamp();
+				});
+	}
+
+	private long generateExpiry(long timestamp) {
+		return ((timestamp - 1) / EXPIRY_ROUNDING) * EXPIRY_ROUNDING + PRESENCE_LIFETIME;
 	}
 
 }

src/main/java/org/qortal/crosschain/Bitcoin.java

@@ -7,6 +7,7 @@ import java.util.Map;
 
 import org.bitcoinj.core.Context;
 import org.bitcoinj.core.NetworkParameters;
+import org.bitcoinj.core.Transaction;
 import org.bitcoinj.params.MainNetParams;
 import org.bitcoinj.params.RegTestParams;
 import org.bitcoinj.params.TestNet3Params;
@@ -18,10 +19,12 @@ public class Bitcoin extends Bitcoiny {
 
 	public static final String CURRENCY_CODE = "BTC";
 
+	private static final long MINIMUM_ORDER_AMOUNT = 100000; // 0.001 BTC minimum order, due to high fees
+
 	// Temporary values until a dynamic fee system is written.
 	private static final long OLD_FEE_AMOUNT = 4_000L; // Not 5000 so that existing P2SH-B can output 1000, avoiding dust issue, leaving 4000 for fees.
 	private static final long NEW_FEE_TIMESTAMP = 1598280000000L; // milliseconds since epoch
-	private static final long NEW_FEE_AMOUNT = 10_000L;
+	private static final long NEW_FEE_AMOUNT = 6_000L;
 
 	private static final long NON_MAINNET_FEE = 1000L; // enough for TESTNET3 and should be OK for REGTEST
 
@@ -42,30 +45,40 @@ public class Bitcoin extends Bitcoiny {
 			public Collection<ElectrumX.Server> getServers() {
 				return Arrays.asList(
 						// Servers chosen on NO BASIS WHATSOEVER from various sources!
-						new Server("hodlers.beer", Server.ConnectionType.SSL, 50002),
-						new Server("btc.lastingcoin.net", Server.ConnectionType.SSL, 50002),
-						new Server("electrum.bitaroo.net", Server.ConnectionType.SSL, 50002),
-						new Server("bitcoin.grey.pw", Server.ConnectionType.SSL, 50002),
+						// Status verified at https://1209k.com/bitcoin-eye/ele.php?chain=btc
+						//CLOSED new Server("bitcoin.grey.pw", Server.ConnectionType.SSL, 50002),
+						//CLOSED new Server("btc.litepay.ch", Server.ConnectionType.SSL, 50002),
+						//CLOSED new Server("electrum.pabu.io", Server.ConnectionType.SSL, 50002),
+						//CLOSED new Server("electrumx.hodlwallet.com", Server.ConnectionType.SSL, 50002),
+						//CLOSED new Server("gd42.org", Server.ConnectionType.SSL, 50002),
+						//CLOSED new Server("korea.electrum-server.com", Server.ConnectionType.SSL, 50002),
+						//CLOSED new Server("prospero.bitsrc.net", Server.ConnectionType.SSL, 50002),
+						//1.15.0 new Server("alviss.coinjoined.com", Server.ConnectionType.SSL, 50002),
+						//1.15.0 new Server("electrum.acinq.co", Server.ConnectionType.SSL, 50002),
+						//1.14.0 new Server("electrum.coinext.com.br", Server.ConnectionType.SSL, 50002),
+						new Server("104.248.139.211", Server.ConnectionType.SSL, 50002),
+						new Server("142.93.6.38", Server.ConnectionType.SSL, 50002),
+						new Server("157.245.172.236", Server.ConnectionType.SSL, 50002),
+						new Server("167.172.226.175", Server.ConnectionType.SSL, 50002),
+						new Server("167.172.42.31", Server.ConnectionType.SSL, 50002),
+						new Server("178.62.80.20", Server.ConnectionType.SSL, 50002),
 						new Server("185.64.116.15", Server.ConnectionType.SSL, 50002),
-						new Server("alviss.coinjoined.com", Server.ConnectionType.SSL, 50002),
-						new Server("btc.litepay.ch", Server.ConnectionType.SSL, 50002),
-						new Server("xtrum.com", Server.ConnectionType.SSL, 50002),
-						new Server("electrum.acinq.co", Server.ConnectionType.SSL, 50002),
-						new Server("caleb.vegas", Server.ConnectionType.SSL, 50002),
-						new Server("electrum.coinext.com.br", Server.ConnectionType.TCP, 50001),
-						new Server("korea.electrum-server.com", Server.ConnectionType.TCP, 50001),
-						new Server("eai.coincited.net", Server.ConnectionType.TCP, 50001),
-						new Server("electrum.coinext.com.br", Server.ConnectionType.SSL, 50002),
-						new Server("node1.btccuracao.com", Server.ConnectionType.SSL, 50002),
-						new Server("korea.electrum-server.com", Server.ConnectionType.SSL, 50002),
-						new Server("btce.iiiiiii.biz", Server.ConnectionType.SSL, 50002),
+						new Server("68.183.188.105", Server.ConnectionType.SSL, 50002),
 						new Server("bitcoin.lukechilds.co", Server.ConnectionType.SSL, 50002),
-						new Server("guichet.centure.cc", Server.ConnectionType.SSL, 50002),
-						new Server("electrumx.hodlwallet.com", Server.ConnectionType.SSL, 50002),
+						new Server("blkhub.net", Server.ConnectionType.SSL, 50002),
+						new Server("btc.lastingcoin.net", Server.ConnectionType.SSL, 50002),
+						new Server("btce.iiiiiii.biz", Server.ConnectionType.SSL, 50002),
+						new Server("caleb.vegas", Server.ConnectionType.SSL, 50002),
 						new Server("eai.coincited.net", Server.ConnectionType.SSL, 50002),
-						new Server("prospero.bitsrc.net", Server.ConnectionType.SSL, 50002),
-						new Server("gd42.org", Server.ConnectionType.SSL, 50002),
-						new Server("electrum.pabu.io", Server.ConnectionType.SSL, 50002));
+						new Server("electrum.bitaroo.net", Server.ConnectionType.SSL, 50002),
+						new Server("electrumx.dev", Server.ConnectionType.SSL, 50002),
+						new Server("elx.bitske.com", Server.ConnectionType.SSL, 50002),
+						new Server("fortress.qtornado.com", Server.ConnectionType.SSL, 50002),
+						new Server("guichet.centure.cc", Server.ConnectionType.SSL, 50002),
+						new Server("kareoke.qoppa.org", Server.ConnectionType.SSL, 50002),
+						new Server("hodlers.beer", Server.ConnectionType.SSL, 50002),
+						new Server("node1.btccuracao.com", Server.ConnectionType.SSL, 50002),
+						new Server("xtrum.com", Server.ConnectionType.SSL, 50002));
 			}
 
 			@Override
@@ -161,6 +174,8 @@ public class Bitcoin extends Bitcoiny {
 			Context bitcoinjContext = new Context(bitcoinNet.getParams());
 
 			instance = new Bitcoin(bitcoinNet, electrumX, bitcoinjContext, CURRENCY_CODE);
+
+			electrumX.setBlockchain(instance);
 		}
 
 		return instance;
@@ -172,6 +187,11 @@ public class Bitcoin extends Bitcoiny {
 		instance = null;
 	}
 
+	@Override
+	public long getMinimumOrderAmount() {
+		return MINIMUM_ORDER_AMOUNT;
+	}
+
 	// Actual useful methods for use by other classes
 
 	/**
@@ -185,4 +205,17 @@ public class Bitcoin extends Bitcoiny {
 		return this.bitcoinNet.getP2shFee(timestamp);
 	}
 
+	/**
+ 	* Returns bitcoinj transaction sending <tt>amount</tt> to <tt>recipient</tt> using 20 sat/byte fee.
+ 	*
+ 	* @param xprv58 BIP32 private key
+ 	* @param recipient P2PKH address
+ 	* @param amount unscaled amount
+ 	* @return transaction, or null if insufficient funds
+ 	*/
+	@Override
+	public Transaction buildSpend(String xprv58, String recipient, long amount) {
+		return buildSpend(xprv58, recipient, amount, 20L);
+	}
+
 }

src/main/java/org/qortal/crosschain/BitcoinACCTv3.java

@@ -0,0 +1,858 @@
+package org.qortal.crosschain;
+
+import com.google.common.hash.HashCode;
+import com.google.common.primitives.Bytes;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.ciyam.at.*;
+import org.qortal.account.Account;
+import org.qortal.asset.Asset;
+import org.qortal.at.QortalFunctionCode;
+import org.qortal.crypto.Crypto;
+import org.qortal.data.at.ATData;
+import org.qortal.data.at.ATStateData;
+import org.qortal.data.crosschain.CrossChainTradeData;
+import org.qortal.data.transaction.MessageTransactionData;
+import org.qortal.repository.DataException;
+import org.qortal.repository.Repository;
+import org.qortal.utils.Base58;
+import org.qortal.utils.BitTwiddling;
+
+import java.nio.ByteBuffer;
+import java.util.Arrays;
+import java.util.List;
+
+import static org.ciyam.at.OpCode.calcOffset;
+
+/**
+ * Cross-chain trade AT
+ * 
+ * <p>
+ * <ul>
+ * <li>Bob generates Bitcoin & Qortal 'trade' keys
+ * 		<ul>
+ * 			<li>private key required to sign P2SH redeem tx</li>
+ * 			<li>private key could be used to create 'secret' (e.g. double-SHA256)</li>
+ * 			<li>encrypted private key could be stored in Qortal AT for access by Bob from any node</li>
+ * 		</ul>
+ * </li>
+ * <li>Bob deploys Qortal AT
+ * 		<ul>
+ * 		</ul>
+ * </li>
+ * <li>Alice finds Qortal AT and wants to trade
+ * 		<ul>
+ * 			<li>Alice generates Bitcoin & Qortal 'trade' keys</li>
+ * 			<li>Alice funds Bitcoin P2SH-A</li>
+ * 			<li>Alice sends 'offer' MESSAGE to Bob from her Qortal trade address, containing:
+ * 				<ul>
+ * 					<li>hash-of-secret-A</li>
+ * 					<li>her 'trade' Bitcoin PKH</li>
+ * 				</ul>
+ * 			</li>
+ * 		</ul>
+ * </li>
+ * <li>Bob receives "offer" MESSAGE
+ * 		<ul>
+ * 			<li>Checks Alice's P2SH-A</li>
+ * 			<li>Sends 'trade' MESSAGE to Qortal AT from his trade address, containing:
+ * 				<ul>
+ * 					<li>Alice's trade Qortal address</li>
+ * 					<li>Alice's trade Bitcoin PKH</li>
+ * 					<li>hash-of-secret-A</li>
+ * 				</ul>
+ * 			</li>
+ * 		</ul>
+ * </li>
+ * <li>Alice checks Qortal AT to confirm it's locked to her
+ * 		<ul>
+ * 			<li>Alice sends 'redeem' MESSAGE to Qortal AT from her trade address, containing:
+ * 				<ul>
+ * 					<li>secret-A</li>
+ * 					<li>Qortal receiving address of her chosing</li>
+ * 				</ul>
+ * 			</li>
+ * 			<li>AT's QORT funds are sent to Qortal receiving address</li>
+ * 		</ul>
+ * </li>
+ * <li>Bob checks AT, extracts secret-A
+ * 		<ul>
+ * 			<li>Bob redeems P2SH-A using his Bitcoin trade key and secret-A</li>
+ * 			<li>P2SH-A BTC funds end up at Bitcoin address determined by redeem transaction output(s)</li>
+ * 		</ul>
+ * </li>
+ * </ul>
+ */
+public class BitcoinACCTv3 implements ACCT {
+
+	private static final Logger LOGGER = LogManager.getLogger(BitcoinACCTv3.class);
+
+	public static final String NAME = BitcoinACCTv3.class.getSimpleName();
+	public static final byte[] CODE_BYTES_HASH = HashCode.fromString("676fb9350708dafa054eb0262d655039e393c1eb4918ec582f8d45524c9b4860").asBytes(); // SHA256 of AT code bytes
+
+	public static final int SECRET_LENGTH = 32;
+
+	/** <b>Value</b> offset into AT segment where 'mode' variable (long) is stored. (Multiply by MachineState.VALUE_SIZE for byte offset). */
+	private static final int MODE_VALUE_OFFSET = 61;
+	/** <b>Byte</b> offset into AT state data where 'mode' variable (long) is stored. */
+	public static final int MODE_BYTE_OFFSET = MachineState.HEADER_LENGTH + (MODE_VALUE_OFFSET * MachineState.VALUE_SIZE);
+
+	public static class OfferMessageData {
+		public byte[] partnerBitcoinPKH;
+		public byte[] hashOfSecretA;
+		public long lockTimeA;
+	}
+	public static final int OFFER_MESSAGE_LENGTH = 20 /*partnerBitcoinPKH*/ + 20 /*hashOfSecretA*/ + 8 /*lockTimeA*/;
+	public static final int TRADE_MESSAGE_LENGTH = 32 /*partner's Qortal trade address (padded from 25 to 32)*/
+			+ 24 /*partner's Bitcoin PKH (padded from 20 to 24)*/
+			+ 8 /*AT trade timeout (minutes)*/
+			+ 24 /*hash of secret-A (padded from 20 to 24)*/
+			+ 8 /*lockTimeA*/;
+	public static final int REDEEM_MESSAGE_LENGTH = 32 /*secret-A*/ + 32 /*partner's Qortal receiving address padded from 25 to 32*/;
+	public static final int CANCEL_MESSAGE_LENGTH = 32 /*AT creator's Qortal address*/;
+
+	private static BitcoinACCTv3 instance;
+
+	private BitcoinACCTv3() {
+	}
+
+	public static synchronized BitcoinACCTv3 getInstance() {
+		if (instance == null)
+			instance = new BitcoinACCTv3();
+
+		return instance;
+	}
+
+	@Override
+	public byte[] getCodeBytesHash() {
+		return CODE_BYTES_HASH;
+	}
+
+	@Override
+	public int getModeByteOffset() {
+		return MODE_BYTE_OFFSET;
+	}
+
+	@Override
+	public ForeignBlockchain getBlockchain() {
+		return Bitcoin.getInstance();
+	}
+
+	/**
+	 * Returns Qortal AT creation bytes for cross-chain trading AT.
+	 * <p>
+	 * <tt>tradeTimeout</tt> (minutes) is the time window for the trade partner to send the
+	 * 32-byte secret to the AT, before the AT automatically refunds the AT's creator.
+	 * 
+	 * @param creatorTradeAddress AT creator's trade Qortal address
+	 * @param bitcoinPublicKeyHash 20-byte HASH160 of creator's trade Bitcoin public key
+	 * @param qortAmount how much QORT to pay trade partner if they send correct 32-byte secrets to AT
+	 * @param bitcoinAmount how much BTC the AT creator is expecting to trade
+	 * @param tradeTimeout suggested timeout for entire trade
+	 */
+	public static byte[] buildQortalAT(String creatorTradeAddress, byte[] bitcoinPublicKeyHash, long qortAmount, long bitcoinAmount, int tradeTimeout) {
+		if (bitcoinPublicKeyHash.length != 20)
+			throw new IllegalArgumentException("Bitcoin public key hash should be 20 bytes");
+
+		// Labels for data segment addresses
+		int addrCounter = 0;
+
+		// Constants (with corresponding dataByteBuffer.put*() calls below)
+
+		final int addrCreatorTradeAddress1 = addrCounter++;
+		final int addrCreatorTradeAddress2 = addrCounter++;
+		final int addrCreatorTradeAddress3 = addrCounter++;
+		final int addrCreatorTradeAddress4 = addrCounter++;
+
+		final int addrBitcoinPublicKeyHash = addrCounter;
+		addrCounter += 4;
+
+		final int addrQortAmount = addrCounter++;
+		final int addrBitcoinAmount = addrCounter++;
+		final int addrTradeTimeout = addrCounter++;
+
+		final int addrMessageTxnType = addrCounter++;
+		final int addrExpectedTradeMessageLength = addrCounter++;
+		final int addrExpectedRedeemMessageLength = addrCounter++;
+
+		final int addrCreatorAddressPointer = addrCounter++;
+		final int addrQortalPartnerAddressPointer = addrCounter++;
+		final int addrMessageSenderPointer = addrCounter++;
+
+		final int addrTradeMessagePartnerBitcoinPKHOffset = addrCounter++;
+		final int addrPartnerBitcoinPKHPointer = addrCounter++;
+		final int addrTradeMessageHashOfSecretAOffset = addrCounter++;
+		final int addrHashOfSecretAPointer = addrCounter++;
+
+		final int addrRedeemMessageReceivingAddressOffset = addrCounter++;
+
+		final int addrMessageDataPointer = addrCounter++;
+		final int addrMessageDataLength = addrCounter++;
+
+		final int addrPartnerReceivingAddressPointer = addrCounter++;
+
+		final int addrEndOfConstants = addrCounter;
+
+		// Variables
+
+		final int addrCreatorAddress1 = addrCounter++;
+		final int addrCreatorAddress2 = addrCounter++;
+		final int addrCreatorAddress3 = addrCounter++;
+		final int addrCreatorAddress4 = addrCounter++;
+
+		final int addrQortalPartnerAddress1 = addrCounter++;
+		final int addrQortalPartnerAddress2 = addrCounter++;
+		final int addrQortalPartnerAddress3 = addrCounter++;
+		final int addrQortalPartnerAddress4 = addrCounter++;
+
+		final int addrLockTimeA = addrCounter++;
+		final int addrRefundTimeout = addrCounter++;
+		final int addrRefundTimestamp = addrCounter++;
+		final int addrLastTxnTimestamp = addrCounter++;
+		final int addrBlockTimestamp = addrCounter++;
+		final int addrTxnType = addrCounter++;
+		final int addrResult = addrCounter++;
+
+		final int addrMessageSender1 = addrCounter++;
+		final int addrMessageSender2 = addrCounter++;
+		final int addrMessageSender3 = addrCounter++;
+		final int addrMessageSender4 = addrCounter++;
+
+		final int addrMessageLength = addrCounter++;
+
+		final int addrMessageData = addrCounter;
+		addrCounter += 4;
+
+		final int addrHashOfSecretA = addrCounter;
+		addrCounter += 4;
+
+		final int addrPartnerBitcoinPKH = addrCounter;
+		addrCounter += 4;
+
+		final int addrPartnerReceivingAddress = addrCounter;
+		addrCounter += 4;
+
+		final int addrMode = addrCounter++;
+		assert addrMode == MODE_VALUE_OFFSET : String.format("addrMode %d does not match MODE_VALUE_OFFSET %d", addrMode, MODE_VALUE_OFFSET);
+
+		// Data segment
+		ByteBuffer dataByteBuffer = ByteBuffer.allocate(addrCounter * MachineState.VALUE_SIZE);
+
+		// AT creator's trade Qortal address, decoded from Base58
+		assert dataByteBuffer.position() == addrCreatorTradeAddress1 * MachineState.VALUE_SIZE : "addrCreatorTradeAddress1 incorrect";
+		byte[] creatorTradeAddressBytes = Base58.decode(creatorTradeAddress);
+		dataByteBuffer.put(Bytes.ensureCapacity(creatorTradeAddressBytes, 32, 0));
+
+		// Bitcoin public key hash
+		assert dataByteBuffer.position() == addrBitcoinPublicKeyHash * MachineState.VALUE_SIZE : "addrBitcoinPublicKeyHash incorrect";
+		dataByteBuffer.put(Bytes.ensureCapacity(bitcoinPublicKeyHash, 32, 0));
+
+		// Redeem Qort amount
+		assert dataByteBuffer.position() == addrQortAmount * MachineState.VALUE_SIZE : "addrQortAmount incorrect";
+		dataByteBuffer.putLong(qortAmount);
+
+		// Expected Bitcoin amount
+		assert dataByteBuffer.position() == addrBitcoinAmount * MachineState.VALUE_SIZE : "addrBitcoinAmount incorrect";
+		dataByteBuffer.putLong(bitcoinAmount);
+
+		// Suggested trade timeout (minutes)
+		assert dataByteBuffer.position() == addrTradeTimeout * MachineState.VALUE_SIZE : "addrTradeTimeout incorrect";
+		dataByteBuffer.putLong(tradeTimeout);
+
+		// We're only interested in MESSAGE transactions
+		assert dataByteBuffer.position() == addrMessageTxnType * MachineState.VALUE_SIZE : "addrMessageTxnType incorrect";
+		dataByteBuffer.putLong(API.ATTransactionType.MESSAGE.value);
+
+		// Expected length of 'trade' MESSAGE data from AT creator
+		assert dataByteBuffer.position() == addrExpectedTradeMessageLength * MachineState.VALUE_SIZE : "addrExpectedTradeMessageLength incorrect";
+		dataByteBuffer.putLong(TRADE_MESSAGE_LENGTH);
+
+		// Expected length of 'redeem' MESSAGE data from trade partner
+		assert dataByteBuffer.position() == addrExpectedRedeemMessageLength * MachineState.VALUE_SIZE : "addrExpectedRedeemMessageLength incorrect";
+		dataByteBuffer.putLong(REDEEM_MESSAGE_LENGTH);
+
+		// Index into data segment of AT creator's address, used by GET_B_IND
+		assert dataByteBuffer.position() == addrCreatorAddressPointer * MachineState.VALUE_SIZE : "addrCreatorAddressPointer incorrect";
+		dataByteBuffer.putLong(addrCreatorAddress1);
+
+		// Index into data segment of partner's Qortal address, used by SET_B_IND
+		assert dataByteBuffer.position() == addrQortalPartnerAddressPointer * MachineState.VALUE_SIZE : "addrQortalPartnerAddressPointer incorrect";
+		dataByteBuffer.putLong(addrQortalPartnerAddress1);
+
+		// Index into data segment of (temporary) transaction's sender's address, used by GET_B_IND
+		assert dataByteBuffer.position() == addrMessageSenderPointer * MachineState.VALUE_SIZE : "addrMessageSenderPointer incorrect";
+		dataByteBuffer.putLong(addrMessageSender1);
+
+		// Offset into 'trade' MESSAGE data payload for extracting partner's Bitcoin PKH
+		assert dataByteBuffer.position() == addrTradeMessagePartnerBitcoinPKHOffset * MachineState.VALUE_SIZE : "addrTradeMessagePartnerBitcoinPKHOffset incorrect";
+		dataByteBuffer.putLong(32L);
+
+		// Index into data segment of partner's Bitcoin PKH, used by GET_B_IND
+		assert dataByteBuffer.position() == addrPartnerBitcoinPKHPointer * MachineState.VALUE_SIZE : "addrPartnerBitcoinPKHPointer incorrect";
+		dataByteBuffer.putLong(addrPartnerBitcoinPKH);
+
+		// Offset into 'trade' MESSAGE data payload for extracting hash-of-secret-A
+		assert dataByteBuffer.position() == addrTradeMessageHashOfSecretAOffset * MachineState.VALUE_SIZE : "addrTradeMessageHashOfSecretAOffset incorrect";
+		dataByteBuffer.putLong(64L);
+
+		// Index into data segment to hash of secret A, used by GET_B_IND
+		assert dataByteBuffer.position() == addrHashOfSecretAPointer * MachineState.VALUE_SIZE : "addrHashOfSecretAPointer incorrect";
+		dataByteBuffer.putLong(addrHashOfSecretA);
+
+		// Offset into 'redeem' MESSAGE data payload for extracting Qortal receiving address
+		assert dataByteBuffer.position() == addrRedeemMessageReceivingAddressOffset * MachineState.VALUE_SIZE : "addrRedeemMessageReceivingAddressOffset incorrect";
+		dataByteBuffer.putLong(32L);
+
+		// Source location and length for hashing any passed secret
+		assert dataByteBuffer.position() == addrMessageDataPointer * MachineState.VALUE_SIZE : "addrMessageDataPointer incorrect";
+		dataByteBuffer.putLong(addrMessageData);
+		assert dataByteBuffer.position() == addrMessageDataLength * MachineState.VALUE_SIZE : "addrMessageDataLength incorrect";
+		dataByteBuffer.putLong(32L);
+
+		// Pointer into data segment of where to save partner's receiving Qortal address, used by GET_B_IND
+		assert dataByteBuffer.position() == addrPartnerReceivingAddressPointer * MachineState.VALUE_SIZE : "addrPartnerReceivingAddressPointer incorrect";
+		dataByteBuffer.putLong(addrPartnerReceivingAddress);
+
+		assert dataByteBuffer.position() == addrEndOfConstants * MachineState.VALUE_SIZE : "dataByteBuffer position not at end of constants";
+
+		// Code labels
+		Integer labelRefund = null;
+
+		Integer labelTradeTxnLoop = null;
+		Integer labelCheckTradeTxn = null;
+		Integer labelCheckCancelTxn = null;
+		Integer labelNotTradeNorCancelTxn = null;
+		Integer labelCheckNonRefundTradeTxn = null;
+		Integer labelTradeTxnExtract = null;
+		Integer labelRedeemTxnLoop = null;
+		Integer labelCheckRedeemTxn = null;
+		Integer labelCheckRedeemTxnSender = null;
+		Integer labelPayout = null;
+
+		ByteBuffer codeByteBuffer = ByteBuffer.allocate(768);
+
+		// Two-pass version
+		for (int pass = 0; pass < 2; ++pass) {
+			codeByteBuffer.clear();
+
+			try {
+				/* Initialization */
+
+				// Use AT creation 'timestamp' as starting point for finding transactions sent to AT
+				codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_CREATION_TIMESTAMP, addrLastTxnTimestamp));
+
+				// Load B register with AT creator's address so we can save it into addrCreatorAddress1-4
+				codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_CREATOR_INTO_B));
+				codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrCreatorAddressPointer));
+
+				// Set restart position to after this opcode
+				codeByteBuffer.put(OpCode.SET_PCS.compile());
+
+				/* Loop, waiting for message from AT creator's trade address containing trade partner details, or AT owner's address to cancel offer */
+
+				/* Transaction processing loop */
+				labelTradeTxnLoop = codeByteBuffer.position();
+
+				/* Sleep until message arrives */
+				codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.SLEEP_UNTIL_MESSAGE.value, addrLastTxnTimestamp));
+
+				// Find next transaction (if any) to this AT since the last one (referenced by addrLastTxnTimestamp)
+				codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PUT_TX_AFTER_TIMESTAMP_INTO_A, addrLastTxnTimestamp));
+				// If no transaction found, A will be zero. If A is zero, set addrResult to 1, otherwise 0.
+				codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.CHECK_A_IS_ZERO, addrResult));
+				// If addrResult is zero (i.e. A is non-zero, transaction was found) then go check transaction
+				codeByteBuffer.put(OpCode.BZR_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelCheckTradeTxn)));
+				// Stop and wait for next block
+				codeByteBuffer.put(OpCode.STP_IMD.compile());
+
+				/* Check transaction */
+				labelCheckTradeTxn = codeByteBuffer.position();
+
+				// Update our 'last found transaction's timestamp' using 'timestamp' from transaction
+				codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TIMESTAMP_FROM_TX_IN_A, addrLastTxnTimestamp));
+				// Extract transaction type (message/payment) from transaction and save type in addrTxnType
+				codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TYPE_FROM_TX_IN_A, addrTxnType));
+				// If transaction type is not MESSAGE type then go look for another transaction
+				codeByteBuffer.put(OpCode.BNE_DAT.compile(addrTxnType, addrMessageTxnType, calcOffset(codeByteBuffer, labelTradeTxnLoop)));
+
+				/* Check transaction's sender. We're expecting AT creator's trade address for 'trade' message, or AT creator's own address for 'cancel' message. */
+
+				// Extract sender address from transaction into B register
+				codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_ADDRESS_FROM_TX_IN_A_INTO_B));
+				// Save B register into data segment starting at addrMessageSender1 (as pointed to by addrMessageSenderPointer)
+				codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageSenderPointer));
+				// Compare each part of message sender's address with AT creator's trade address. If they don't match, check for cancel situation.
+				codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrCreatorTradeAddress1, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
+				codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrCreatorTradeAddress2, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
+				codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrCreatorTradeAddress3, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
+				codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrCreatorTradeAddress4, calcOffset(codeByteBuffer, labelCheckCancelTxn)));
+				// Message sender's address matches AT creator's trade address so go process 'trade' message
+				codeByteBuffer.put(OpCode.JMP_ADR.compile(labelCheckNonRefundTradeTxn == null ? 0 : labelCheckNonRefundTradeTxn));
+
+				/* Checking message sender for possible cancel message */
+				labelCheckCancelTxn = codeByteBuffer.position();
+
+				// Compare each part of message sender's address with AT creator's address. If they don't match, look for another transaction.
+				codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrCreatorAddress1, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
+				codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrCreatorAddress2, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
+				codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrCreatorAddress3, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
+				codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrCreatorAddress4, calcOffset(codeByteBuffer, labelNotTradeNorCancelTxn)));
+				// Partner address is AT creator's address, so cancel offer and finish.
+				codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.CANCELLED.value));
+				// We're finished forever (finishing auto-refunds remaining balance to AT creator)
+				codeByteBuffer.put(OpCode.FIN_IMD.compile());
+
+				/* Not trade nor cancel message */
+				labelNotTradeNorCancelTxn = codeByteBuffer.position();
+
+				// Loop to find another transaction
+				codeByteBuffer.put(OpCode.JMP_ADR.compile(labelTradeTxnLoop == null ? 0 : labelTradeTxnLoop));
+
+				/* Possible switch-to-trade-mode message */
+				labelCheckNonRefundTradeTxn = codeByteBuffer.position();
+
+				// Check 'trade' message we received has expected number of message bytes
+				codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(QortalFunctionCode.GET_MESSAGE_LENGTH_FROM_TX_IN_A.value, addrMessageLength));
+				// If message length matches, branch to info extraction code
+				codeByteBuffer.put(OpCode.BEQ_DAT.compile(addrMessageLength, addrExpectedTradeMessageLength, calcOffset(codeByteBuffer, labelTradeTxnExtract)));
+				// Message length didn't match - go back to finding another 'trade' MESSAGE transaction
+				codeByteBuffer.put(OpCode.JMP_ADR.compile(labelTradeTxnLoop == null ? 0 : labelTradeTxnLoop));
+
+				/* Extracting info from 'trade' MESSAGE transaction */
+				labelTradeTxnExtract = codeByteBuffer.position();
+
+				// Extract message from transaction into B register
+				codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_MESSAGE_FROM_TX_IN_A_INTO_B));
+				// Save B register into data segment starting at addrQortalPartnerAddress1 (as pointed to by addrQortalPartnerAddressPointer)
+				codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrQortalPartnerAddressPointer));
+
+				// Extract trade partner's Bitcoin public key hash (PKH) from message into B
+				codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrTradeMessagePartnerBitcoinPKHOffset));
+				// Store partner's Bitcoin PKH (we only really use values from B1-B3)
+				codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrPartnerBitcoinPKHPointer));
+				// Extract AT trade timeout (minutes) (from B4)
+				codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_B4, addrRefundTimeout));
+
+				// Grab next 32 bytes
+				codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrTradeMessageHashOfSecretAOffset));
+
+				// Extract hash-of-secret-A (we only really use values from B1-B3)
+				codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrHashOfSecretAPointer));
+				// Extract lockTime-A (from B4)
+				codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_B4, addrLockTimeA));
+
+				// Calculate trade timeout refund 'timestamp' by adding addrRefundTimeout minutes to this transaction's 'timestamp', then save into addrRefundTimestamp
+				codeByteBuffer.put(OpCode.EXT_FUN_RET_DAT_2.compile(FunctionCode.ADD_MINUTES_TO_TIMESTAMP, addrRefundTimestamp, addrLastTxnTimestamp, addrRefundTimeout));
+
+				/* We are in 'trade mode' */
+				codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.TRADING.value));
+
+				// Set restart position to after this opcode
+				codeByteBuffer.put(OpCode.SET_PCS.compile());
+
+				/* Loop, waiting for trade timeout or 'redeem' MESSAGE from Qortal trade partner */
+
+				// Fetch current block 'timestamp'
+				codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_BLOCK_TIMESTAMP, addrBlockTimestamp));
+				// If we're not past refund 'timestamp' then look for next transaction
+				codeByteBuffer.put(OpCode.BLT_DAT.compile(addrBlockTimestamp, addrRefundTimestamp, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
+				// We're past refund 'timestamp' so go refund everything back to AT creator
+				codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRefund == null ? 0 : labelRefund));
+
+				/* Transaction processing loop */
+				labelRedeemTxnLoop = codeByteBuffer.position();
+
+				// Find next transaction to this AT since the last one (if any)
+				codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PUT_TX_AFTER_TIMESTAMP_INTO_A, addrLastTxnTimestamp));
+				// If no transaction found, A will be zero. If A is zero, set addrComparator to 1, otherwise 0.
+				codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.CHECK_A_IS_ZERO, addrResult));
+				// If addrResult is zero (i.e. A is non-zero, transaction was found) then go check transaction
+				codeByteBuffer.put(OpCode.BZR_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelCheckRedeemTxn)));
+				// Stop and wait for next block
+				codeByteBuffer.put(OpCode.STP_IMD.compile());
+
+				/* Check transaction */
+				labelCheckRedeemTxn = codeByteBuffer.position();
+
+				// Update our 'last found transaction's timestamp' using 'timestamp' from transaction
+				codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TIMESTAMP_FROM_TX_IN_A, addrLastTxnTimestamp));
+				// Extract transaction type (message/payment) from transaction and save type in addrTxnType
+				codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(FunctionCode.GET_TYPE_FROM_TX_IN_A, addrTxnType));
+				// If transaction type is not MESSAGE type then go look for another transaction
+				codeByteBuffer.put(OpCode.BNE_DAT.compile(addrTxnType, addrMessageTxnType, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
+
+				/* Check message payload length */
+				codeByteBuffer.put(OpCode.EXT_FUN_RET.compile(QortalFunctionCode.GET_MESSAGE_LENGTH_FROM_TX_IN_A.value, addrMessageLength));
+				// If message length matches, branch to sender checking code
+				codeByteBuffer.put(OpCode.BEQ_DAT.compile(addrMessageLength, addrExpectedRedeemMessageLength, calcOffset(codeByteBuffer, labelCheckRedeemTxnSender)));
+				// Message length didn't match - go back to finding another 'redeem' MESSAGE transaction
+				codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRedeemTxnLoop == null ? 0 : labelRedeemTxnLoop));
+
+				/* Check transaction's sender */
+				labelCheckRedeemTxnSender = codeByteBuffer.position();
+
+				// Extract sender address from transaction into B register
+				codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_ADDRESS_FROM_TX_IN_A_INTO_B));
+				// Save B register into data segment starting at addrMessageSender1 (as pointed to by addrMessageSenderPointer)
+				codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageSenderPointer));
+				// Compare each part of transaction's sender's address with expected address. If they don't match, look for another transaction.
+				codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender1, addrQortalPartnerAddress1, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
+				codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender2, addrQortalPartnerAddress2, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
+				codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender3, addrQortalPartnerAddress3, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
+				codeByteBuffer.put(OpCode.BNE_DAT.compile(addrMessageSender4, addrQortalPartnerAddress4, calcOffset(codeByteBuffer, labelRedeemTxnLoop)));
+
+				/* Check 'secret-A' in transaction's message */
+
+				// Extract secret-A from first 32 bytes of message from transaction into B register
+				codeByteBuffer.put(OpCode.EXT_FUN.compile(FunctionCode.PUT_MESSAGE_FROM_TX_IN_A_INTO_B));
+				// Save B register into data segment starting at addrMessageData (as pointed to by addrMessageDataPointer)
+				codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrMessageDataPointer));
+				// Load B register with expected hash result (as pointed to by addrHashOfSecretAPointer)
+				codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.SET_B_IND, addrHashOfSecretAPointer));
+				// Perform HASH160 using source data at addrMessageData. (Location and length specified via addrMessageDataPointer and addrMessageDataLength).
+				// Save the equality result (1 if they match, 0 otherwise) into addrResult.
+				codeByteBuffer.put(OpCode.EXT_FUN_RET_DAT_2.compile(FunctionCode.CHECK_HASH160_WITH_B, addrResult, addrMessageDataPointer, addrMessageDataLength));
+				// If hashes don't match, addrResult will be zero so go find another transaction
+				codeByteBuffer.put(OpCode.BNZ_DAT.compile(addrResult, calcOffset(codeByteBuffer, labelPayout)));
+				codeByteBuffer.put(OpCode.JMP_ADR.compile(labelRedeemTxnLoop == null ? 0 : labelRedeemTxnLoop));
+
+				/* Success! Pay arranged amount to receiving address */
+				labelPayout = codeByteBuffer.position();
+
+				// Extract Qortal receiving address from next 32 bytes of message from transaction into B register
+				codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(QortalFunctionCode.PUT_PARTIAL_MESSAGE_FROM_TX_IN_A_INTO_B.value, addrRedeemMessageReceivingAddressOffset));
+				// Save B register into data segment starting at addrPartnerReceivingAddress (as pointed to by addrPartnerReceivingAddressPointer)
+				codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.GET_B_IND, addrPartnerReceivingAddressPointer));
+				// Pay AT's balance to receiving address
+				codeByteBuffer.put(OpCode.EXT_FUN_DAT.compile(FunctionCode.PAY_TO_ADDRESS_IN_B, addrQortAmount));
+				// Set redeemed mode
+				codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.REDEEMED.value));
+				// We're finished forever (finishing auto-refunds remaining balance to AT creator)
+				codeByteBuffer.put(OpCode.FIN_IMD.compile());
+
+				// Fall-through to refunding any remaining balance back to AT creator
+
+				/* Refund balance back to AT creator */
+				labelRefund = codeByteBuffer.position();
+
+				/* NOP - to ensure BITCOIN ACCT is unique */
+				codeByteBuffer.put(OpCode.NOP.compile());
+
+				// Set refunded mode
+				codeByteBuffer.put(OpCode.SET_VAL.compile(addrMode, AcctMode.REFUNDED.value));
+				// We're finished forever (finishing auto-refunds remaining balance to AT creator)
+				codeByteBuffer.put(OpCode.FIN_IMD.compile());
+			} catch (CompilationException e) {
+				throw new IllegalStateException("Unable to compile BTC-QORT ACCT?", e);
+			}
+		}
+
+		codeByteBuffer.flip();
+
+		byte[] codeBytes = new byte[codeByteBuffer.limit()];
+		codeByteBuffer.get(codeBytes);
+
+		assert Arrays.equals(Crypto.digest(codeBytes), BitcoinACCTv3.CODE_BYTES_HASH)
+			: String.format("BTCACCT.CODE_BYTES_HASH mismatch: expected %s, actual %s", HashCode.fromBytes(CODE_BYTES_HASH), HashCode.fromBytes(Crypto.digest(codeBytes)));
+
+		final short ciyamAtVersion = 2;
+		final short numCallStackPages = 0;
+		final short numUserStackPages = 0;
+		final long minActivationAmount = 0L;
+
+		return MachineState.toCreationBytes(ciyamAtVersion, codeBytes, dataByteBuffer.array(), numCallStackPages, numUserStackPages, minActivationAmount);
+	}
+
+	/**
+	 * Returns CrossChainTradeData with useful info extracted from AT.
+	 */
+	@Override
+	public CrossChainTradeData populateTradeData(Repository repository, ATData atData) throws DataException {
+		ATStateData atStateData = repository.getATRepository().getLatestATState(atData.getATAddress());
+		return populateTradeData(repository, atData.getCreatorPublicKey(), atData.getCreation(), atStateData);
+	}
+
+	/**
+	 * Returns CrossChainTradeData with useful info extracted from AT.
+	 */
+	@Override
+	public CrossChainTradeData populateTradeData(Repository repository, ATStateData atStateData) throws DataException {
+		ATData atData = repository.getATRepository().fromATAddress(atStateData.getATAddress());
+		return populateTradeData(repository, atData.getCreatorPublicKey(), atData.getCreation(), atStateData);
+	}
+
+	/**
+	 * Returns CrossChainTradeData with useful info extracted from AT.
+	 */
+	public CrossChainTradeData populateTradeData(Repository repository, byte[] creatorPublicKey, long creationTimestamp, ATStateData atStateData) throws DataException {
+		byte[] addressBytes = new byte[25]; // for general use
+		String atAddress = atStateData.getATAddress();
+
+		CrossChainTradeData tradeData = new CrossChainTradeData();
+
+		tradeData.foreignBlockchain = SupportedBlockchain.BITCOIN.name();
+		tradeData.acctName = NAME;
+
+		tradeData.qortalAtAddress = atAddress;
+		tradeData.qortalCreator = Crypto.toAddress(creatorPublicKey);
+		tradeData.creationTimestamp = creationTimestamp;
+
+		Account atAccount = new Account(repository, atAddress);
+		tradeData.qortBalance = atAccount.getConfirmedBalance(Asset.QORT);
+
+		byte[] stateData = atStateData.getStateData();
+		ByteBuffer dataByteBuffer = ByteBuffer.wrap(stateData);
+		dataByteBuffer.position(MachineState.HEADER_LENGTH);
+
+		/* Constants */
+
+		// Skip creator's trade address
+		dataByteBuffer.get(addressBytes);
+		tradeData.qortalCreatorTradeAddress = Base58.encode(addressBytes);
+		dataByteBuffer.position(dataByteBuffer.position() + 32 - addressBytes.length);
+
+		// Creator's Bitcoin/foreign public key hash
+		tradeData.creatorForeignPKH = new byte[20];
+		dataByteBuffer.get(tradeData.creatorForeignPKH);
+		dataByteBuffer.position(dataByteBuffer.position() + 32 - tradeData.creatorForeignPKH.length); // skip to 32 bytes
+
+		// We don't use secret-B
+		tradeData.hashOfSecretB = null;
+
+		// Redeem payout
+		tradeData.qortAmount = dataByteBuffer.getLong();
+
+		// Expected BTC amount
+		tradeData.expectedForeignAmount = dataByteBuffer.getLong();
+
+		// Trade timeout
+		tradeData.tradeTimeout = (int) dataByteBuffer.getLong();
+
+		// Skip MESSAGE transaction type
+		dataByteBuffer.position(dataByteBuffer.position() + 8);
+
+		// Skip expected 'trade' message length
+		dataByteBuffer.position(dataByteBuffer.position() + 8);
+
+		// Skip expected 'redeem' message length
+		dataByteBuffer.position(dataByteBuffer.position() + 8);
+
+		// Skip pointer to creator's address
+		dataByteBuffer.position(dataByteBuffer.position() + 8);
+
+		// Skip pointer to partner's Qortal trade address
+		dataByteBuffer.position(dataByteBuffer.position() + 8);
+
+		// Skip pointer to message sender
+		dataByteBuffer.position(dataByteBuffer.position() + 8);
+
+		// Skip 'trade' message data offset for partner's Bitcoin PKH
+		dataByteBuffer.position(dataByteBuffer.position() + 8);
+
+		// Skip pointer to partner's Bitcoin PKH
+		dataByteBuffer.position(dataByteBuffer.position() + 8);
+
+		// Skip 'trade' message data offset for hash-of-secret-A
+		dataByteBuffer.position(dataByteBuffer.position() + 8);
+
+		// Skip pointer to hash-of-secret-A
+		dataByteBuffer.position(dataByteBuffer.position() + 8);
+
+		// Skip 'redeem' message data offset for partner's Qortal receiving address
+		dataByteBuffer.position(dataByteBuffer.position() + 8);
+
+		// Skip pointer to message data
+		dataByteBuffer.position(dataByteBuffer.position() + 8);
+
+		// Skip message data length
+		dataByteBuffer.position(dataByteBuffer.position() + 8);
+
+		// Skip pointer to partner's receiving address
+		dataByteBuffer.position(dataByteBuffer.position() + 8);
+
+		/* End of constants / begin variables */
+
+		// Skip AT creator's address
+		dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
+
+		// Partner's trade address (if present)
+		dataByteBuffer.get(addressBytes);
+		String qortalRecipient = Base58.encode(addressBytes);
+		dataByteBuffer.position(dataByteBuffer.position() + 32 - addressBytes.length);
+
+		// Potential lockTimeA (if in trade mode)
+		int lockTimeA = (int) dataByteBuffer.getLong();
+
+		// AT refund timeout (probably only useful for debugging)
+		int refundTimeout = (int) dataByteBuffer.getLong();
+
+		// Trade-mode refund timestamp (AT 'timestamp' converted to Qortal block height)
+		long tradeRefundTimestamp = dataByteBuffer.getLong();
+
+		// Skip last transaction timestamp
+		dataByteBuffer.position(dataByteBuffer.position() + 8);
+
+		// Skip block timestamp
+		dataByteBuffer.position(dataByteBuffer.position() + 8);
+
+		// Skip transaction type
+		dataByteBuffer.position(dataByteBuffer.position() + 8);
+
+		// Skip temporary result
+		dataByteBuffer.position(dataByteBuffer.position() + 8);
+
+		// Skip temporary message sender
+		dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
+
+		// Skip message length
+		dataByteBuffer.position(dataByteBuffer.position() + 8);
+
+		// Skip temporary message data
+		dataByteBuffer.position(dataByteBuffer.position() + 8 * 4);
+
+		// Potential hash160 of secret A
+		byte[] hashOfSecretA = new byte[20];
+		dataByteBuffer.get(hashOfSecretA);
+		dataByteBuffer.position(dataByteBuffer.position() + 32 - hashOfSecretA.length); // skip to 32 bytes
+
+		// Potential partner's Bitcoin PKH
+		byte[] partnerBitcoinPKH = new byte[20];
+		dataByteBuffer.get(partnerBitcoinPKH);
+		dataByteBuffer.position(dataByteBuffer.position() + 32 - partnerBitcoinPKH.length); // skip to 32 bytes
+
+		// Partner's receiving address (if present)
+		byte[] partnerReceivingAddress = new byte[25];
+		dataByteBuffer.get(partnerReceivingAddress);
+		dataByteBuffer.position(dataByteBuffer.position() + 32 - partnerReceivingAddress.length); // skip to 32 bytes
+
+		// Trade AT's 'mode'
+		long modeValue = dataByteBuffer.getLong();
+		AcctMode mode = AcctMode.valueOf((int) (modeValue & 0xffL));
+
+		/* End of variables */
+
+		if (mode != null && mode != AcctMode.OFFERING) {
+			tradeData.mode = mode;
+			tradeData.refundTimeout = refundTimeout;
+			tradeData.tradeRefundHeight = new Timestamp(tradeRefundTimestamp).blockHeight;
+			tradeData.qortalPartnerAddress = qortalRecipient;
+			tradeData.hashOfSecretA = hashOfSecretA;
+			tradeData.partnerForeignPKH = partnerBitcoinPKH;
+			tradeData.lockTimeA = lockTimeA;
+
+			if (mode == AcctMode.REDEEMED)
+				tradeData.qortalPartnerReceivingAddress = Base58.encode(partnerReceivingAddress);
+		} else {
+			tradeData.mode = AcctMode.OFFERING;
+		}
+
+		tradeData.duplicateDeprecated();
+
+		return tradeData;
+	}
+
+	/** Returns 'offer' MESSAGE payload for trade partner to send to AT creator's trade address. */
+	public static byte[] buildOfferMessage(byte[] partnerBitcoinPKH, byte[] hashOfSecretA, int lockTimeA) {
+		byte[] lockTimeABytes = BitTwiddling.toBEByteArray((long) lockTimeA);
+		return Bytes.concat(partnerBitcoinPKH, hashOfSecretA, lockTimeABytes);
+	}
+
+	/** Returns info extracted from 'offer' MESSAGE payload sent by trade partner to AT creator's trade address, or null if not valid. */
+	public static OfferMessageData extractOfferMessageData(byte[] messageData) {
+		if (messageData == null || messageData.length != OFFER_MESSAGE_LENGTH)
+			return null;
+
+		OfferMessageData offerMessageData = new OfferMessageData();
+		offerMessageData.partnerBitcoinPKH = Arrays.copyOfRange(messageData, 0, 20);
+		offerMessageData.hashOfSecretA = Arrays.copyOfRange(messageData, 20, 40);
+		offerMessageData.lockTimeA = BitTwiddling.longFromBEBytes(messageData, 40);
+
+		return offerMessageData;
+	}
+
+	/** Returns 'trade' MESSAGE payload for AT creator to send to AT. */
+	public static byte[] buildTradeMessage(String partnerQortalTradeAddress, byte[] partnerBitcoinPKH, byte[] hashOfSecretA, int lockTimeA, int refundTimeout) {
+		byte[] data = new byte[TRADE_MESSAGE_LENGTH];
+		byte[] partnerQortalAddressBytes = Base58.decode(partnerQortalTradeAddress);
+		byte[] lockTimeABytes = BitTwiddling.toBEByteArray((long) lockTimeA);
+		byte[] refundTimeoutBytes = BitTwiddling.toBEByteArray((long) refundTimeout);
+
+		System.arraycopy(partnerQortalAddressBytes, 0, data, 0, partnerQortalAddressBytes.length);
+		System.arraycopy(partnerBitcoinPKH, 0, data, 32, partnerBitcoinPKH.length);
+		System.arraycopy(refundTimeoutBytes, 0, data, 56, refundTimeoutBytes.length);
+		System.arraycopy(hashOfSecretA, 0, data, 64, hashOfSecretA.length);
+		System.arraycopy(lockTimeABytes, 0, data, 88, lockTimeABytes.length);
+
+		return data;
+	}
+
+	/** Returns 'cancel' MESSAGE payload for AT creator to cancel trade AT. */
+	@Override
+	public byte[] buildCancelMessage(String creatorQortalAddress) {
+		byte[] data = new byte[CANCEL_MESSAGE_LENGTH];
+		byte[] creatorQortalAddressBytes = Base58.decode(creatorQortalAddress);
+
+		System.arraycopy(creatorQortalAddressBytes, 0, data, 0, creatorQortalAddressBytes.length);
+
+		return data;
+	}
+
+	/** Returns 'redeem' MESSAGE payload for trade partner to send to AT. */
+	public static byte[] buildRedeemMessage(byte[] secretA, String qortalReceivingAddress) {
+		byte[] data = new byte[REDEEM_MESSAGE_LENGTH];
+		byte[] qortalReceivingAddressBytes = Base58.decode(qortalReceivingAddress);
+
+		System.arraycopy(secretA, 0, data, 0, secretA.length);
+		System.arraycopy(qortalReceivingAddressBytes, 0, data, 32, qortalReceivingAddressBytes.length);
+
+		return data;
+	}
+
+	/** Returns refund timeout (minutes) based on trade partner's 'offer' MESSAGE timestamp and P2SH-A locktime. */
+	public static int calcRefundTimeout(long offerMessageTimestamp, int lockTimeA) {
+		// refund should be triggered halfway between offerMessageTimestamp and lockTimeA
+		return (int) ((lockTimeA - (offerMessageTimestamp / 1000L)) / 2L / 60L);
+	}
+
+	@Override
+	public byte[] findSecretA(Repository repository, CrossChainTradeData crossChainTradeData) throws DataException {
+		String atAddress = crossChainTradeData.qortalAtAddress;
+		String redeemerAddress = crossChainTradeData.qortalPartnerAddress;
+
+		// We don't have partner's public key so we check every message to AT
+		List<MessageTransactionData> messageTransactionsData = repository.getMessageRepository().getMessagesByParticipants(null, atAddress, null, null, null);
+		if (messageTransactionsData == null)
+			return null;
+
+		// Find 'redeem' message
+		for (MessageTransactionData messageTransactionData : messageTransactionsData) {
+			// Check message payload type/encryption
+			if (messageTransactionData.isText() || messageTransactionData.isEncrypted())
+				continue;
+
+			// Check message payload size
+			byte[] messageData = messageTransactionData.getData();
+			if (messageData.length != REDEEM_MESSAGE_LENGTH)
+				// Wrong payload length
+				continue;
+
+			// Check sender
+			if (!Crypto.toAddress(messageTransactionData.getSenderPublicKey()).equals(redeemerAddress))
+				// Wrong sender;
+				continue;
+
+			// Extract secretA
+			byte[] secretA = new byte[32];
+			System.arraycopy(messageData, 0, secretA, 0, secretA.length);
+
+			byte[] hashOfSecretA = Crypto.hash160(secretA);
+			if (!Arrays.equals(hashOfSecretA, crossChainTradeData.hashOfSecretA))
+				continue;
+
+			return secretA;
+		}
+
+		return null;
+	}
+
+}

src/main/java/org/qortal/crosschain/Bitcoiny.java

@@ -1,12 +1,6 @@
 package org.qortal.crosschain;
 
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.Comparator;
-import java.util.HashSet;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
+import java.util.*;
 import java.util.stream.Collectors;
 
 import org.apache.logging.log4j.LogManager;
@@ -35,10 +29,12 @@ import org.bitcoinj.wallet.SendRequest;
 import org.bitcoinj.wallet.Wallet;
 import org.qortal.api.model.SimpleForeignTransaction;
 import org.qortal.crypto.Crypto;
+import org.qortal.settings.Settings;
 import org.qortal.utils.Amounts;
 import org.qortal.utils.BitTwiddling;
 
 import com.google.common.hash.HashCode;
+import org.qortal.utils.NTP;
 
 /** Bitcoin-like (Bitcoin, Litecoin, etc.) support */
 public abstract class Bitcoiny implements ForeignBlockchain {
@@ -47,17 +43,23 @@ public abstract class Bitcoiny implements ForeignBlockchain {
 
 	public static final int HASH160_LENGTH = 20;
 
-	protected final BitcoinyBlockchainProvider blockchain;
+	protected final BitcoinyBlockchainProvider blockchainProvider;
 	protected final Context bitcoinjContext;
 	protected final String currencyCode;
 
 	protected final NetworkParameters params;
 
+	/** Cache recent transactions to speed up subsequent lookups */
+	protected List<SimpleTransaction> transactionsCache;
+	protected Long transactionsCacheTimestamp;
+	protected String transactionsCacheXpub;
+	protected static long TRANSACTIONS_CACHE_TIMEOUT = 2 * 60 * 1000L; // 2 minutes
+
 	/** Keys that have been previously marked as fully spent,<br>
 	 * i.e. keys with transactions but with no unspent outputs. */
 	protected final Set<ECKey> spentKeys = Collections.synchronizedSet(new HashSet<>());
 
-	/** How many bitcoinj wallet keys to generate in each batch. */
+	/** How many wallet keys to generate in each batch. */
 	private static final int WALLET_KEY_LOOKAHEAD_INCREMENT = 3;
 
 	/** Byte offset into raw block headers to block timestamp. */
@@ -65,8 +67,8 @@ public abstract class Bitcoiny implements ForeignBlockchain {
 
 	// Constructors and instance
 
-	protected Bitcoiny(BitcoinyBlockchainProvider blockchain, Context bitcoinjContext, String currencyCode) {
-		this.blockchain = blockchain;
+	protected Bitcoiny(BitcoinyBlockchainProvider blockchainProvider, Context bitcoinjContext, String currencyCode) {
+		this.blockchainProvider = blockchainProvider;
 		this.bitcoinjContext = bitcoinjContext;
 		this.currencyCode = currencyCode;
 
@@ -76,7 +78,7 @@ public abstract class Bitcoiny implements ForeignBlockchain {
 	// Getters & setters
 
 	public BitcoinyBlockchainProvider getBlockchainProvider() {
-		return this.blockchain;
+		return this.blockchainProvider;
 	}
 
 	public Context getBitcoinjContext() {
@@ -98,8 +100,9 @@ public abstract class Bitcoiny implements ForeignBlockchain {
 		try {
 			ScriptType addressType = Address.fromString(this.params, address).getOutputScriptType();
 
-			return addressType == ScriptType.P2PKH || addressType == ScriptType.P2SH;
+			return addressType == ScriptType.P2PKH || addressType == ScriptType.P2SH || addressType == ScriptType.P2WPKH;
 		} catch (AddressFormatException e) {
+			LOGGER.error(String.format("Unrecognised address format: %s", address));
 			return false;
 		}
 	}
@@ -148,10 +151,10 @@ public abstract class Bitcoiny implements ForeignBlockchain {
 	 * @throws ForeignBlockchainException if error occurs
 	 */
 	public int getMedianBlockTime() throws ForeignBlockchainException {
-		int height = this.blockchain.getCurrentHeight();
+		int height = this.blockchainProvider.getCurrentHeight();
 
 		// Grab latest 11 blocks
-		List<byte[]> blockHeaders = this.blockchain.getRawBlockHeaders(height - 11, 11);
+		List<byte[]> blockHeaders = this.blockchainProvider.getRawBlockHeaders(height - 11, 11);
 		if (blockHeaders.size() < 11)
 			throw new ForeignBlockchainException("Not enough blocks to determine median block time");
 
@@ -190,7 +193,7 @@ public abstract class Bitcoiny implements ForeignBlockchain {
 	 * @throws ForeignBlockchainException if there was an error
 	 */
 	public long getConfirmedBalance(String base58Address) throws ForeignBlockchainException {
-		return this.blockchain.getConfirmedBalance(addressToScriptPubKey(base58Address));
+		return this.blockchainProvider.getConfirmedBalance(addressToScriptPubKey(base58Address));
 	}
 
 	/**
@@ -201,7 +204,7 @@ public abstract class Bitcoiny implements ForeignBlockchain {
 	 */
 	// TODO: don't return bitcoinj-based objects like TransactionOutput, use BitcoinyTransaction.Output instead
 	public List<TransactionOutput> getUnspentOutputs(String base58Address) throws ForeignBlockchainException {
-		List<UnspentOutput> unspentOutputs = this.blockchain.getUnspentOutputs(addressToScriptPubKey(base58Address), false);
+		List<UnspentOutput> unspentOutputs = this.blockchainProvider.getUnspentOutputs(addressToScriptPubKey(base58Address), false);
 
 		List<TransactionOutput> unspentTransactionOutputs = new ArrayList<>();
 		for (UnspentOutput unspentOutput : unspentOutputs) {
@@ -221,13 +224,32 @@ public abstract class Bitcoiny implements ForeignBlockchain {
 	 */
 	// TODO: don't return bitcoinj-based objects like TransactionOutput, use BitcoinyTransaction.Output instead
 	public List<TransactionOutput> getOutputs(byte[] txHash) throws ForeignBlockchainException {
-		byte[] rawTransactionBytes = this.blockchain.getRawTransaction(txHash);
+		byte[] rawTransactionBytes = this.blockchainProvider.getRawTransaction(txHash);
 
 		Context.propagate(bitcoinjContext);
 		Transaction transaction = new Transaction(this.params, rawTransactionBytes);
 		return transaction.getOutputs();
 	}
 
+	/**
+	 * Returns transactions for passed script
+	 * <p>
+	 * @throws ForeignBlockchainException if error occurs
+	 */
+	public List<TransactionHash> getAddressTransactions(byte[] scriptPubKey, boolean includeUnconfirmed) throws ForeignBlockchainException {
+		int retries = 0;
+		ForeignBlockchainException e2 = null;
+		while (retries <= 3) {
+			try {
+				return this.blockchainProvider.getAddressTransactions(scriptPubKey, includeUnconfirmed);
+			} catch (ForeignBlockchainException e) {
+				e2 = e;
+				retries++;
+			}
+		}
+		throw(e2);
+	}
+
 	/**
 	 * Returns list of transaction hashes pertaining to passed address.
 	 * <p>
@@ -235,7 +257,7 @@ public abstract class Bitcoiny implements ForeignBlockchain {
 	 * @throws ForeignBlockchainException if there was an error.
 	 */
 	public List<TransactionHash> getAddressTransactions(String base58Address, boolean includeUnconfirmed) throws ForeignBlockchainException {
-		return this.blockchain.getAddressTransactions(addressToScriptPubKey(base58Address), includeUnconfirmed);
+		return this.blockchainProvider.getAddressTransactions(addressToScriptPubKey(base58Address), includeUnconfirmed);
 	}
 
 	/**
@@ -244,11 +266,11 @@ public abstract class Bitcoiny implements ForeignBlockchain {
 	 * @throws ForeignBlockchainException if there was an error
 	 */
 	public List<byte[]> getAddressTransactions(String base58Address) throws ForeignBlockchainException {
-		List<TransactionHash> transactionHashes = this.blockchain.getAddressTransactions(addressToScriptPubKey(base58Address), false);
+		List<TransactionHash> transactionHashes = this.blockchainProvider.getAddressTransactions(addressToScriptPubKey(base58Address), false);
 
 		List<byte[]> rawTransactions = new ArrayList<>();
 		for (TransactionHash transactionInfo : transactionHashes) {
-			byte[] rawTransaction = this.blockchain.getRawTransaction(HashCode.fromString(transactionInfo.txHash).asBytes());
+			byte[] rawTransaction = this.blockchainProvider.getRawTransaction(HashCode.fromString(transactionInfo.txHash).asBytes());
 			rawTransactions.add(rawTransaction);
 		}
 
@@ -262,7 +284,17 @@ public abstract class Bitcoiny implements ForeignBlockchain {
 	 * @throws ForeignBlockchainException if error occurs
 	 */
 	public BitcoinyTransaction getTransaction(String txHash) throws ForeignBlockchainException {
-		return this.blockchain.getTransaction(txHash);
+		int retries = 0;
+		ForeignBlockchainException e2 = null;
+		while (retries <= 3) {
+			try {
+				return this.blockchainProvider.getTransaction(txHash);
+			} catch (ForeignBlockchainException e) {
+				e2 = e;
+				retries++;
+			}
+		}
+		throw(e2);
 	}
 
 	/**
@@ -271,7 +303,7 @@ public abstract class Bitcoiny implements ForeignBlockchain {
 	 * @throws ForeignBlockchainException if error occurs
 	 */
 	public void broadcastTransaction(Transaction transaction) throws ForeignBlockchainException {
-		this.blockchain.broadcastTransaction(transaction.bitcoinSerialize());
+		this.blockchainProvider.broadcastTransaction(transaction.bitcoinSerialize());
 	}
 
 	/**
@@ -324,83 +356,112 @@ public abstract class Bitcoiny implements ForeignBlockchain {
 	 * @param key58 BIP32/HD extended Bitcoin private/public key
 	 * @return unspent BTC balance, or null if unable to determine balance
 	 */
-	public Long getWalletBalance(String key58) {
-		Context.propagate(bitcoinjContext);
+	public Long getWalletBalance(String key58) throws ForeignBlockchainException {
+		// It's more accurate to calculate the balance from the transactions, rather than asking Bitcoinj
+		return this.getWalletBalanceFromTransactions(key58);
 
-		Wallet wallet = walletFromDeterministicKey58(key58);
-		wallet.setUTXOProvider(new WalletAwareUTXOProvider(this, wallet));
+//		Context.propagate(bitcoinjContext);
+//
+//		Wallet wallet = walletFromDeterministicKey58(key58);
+//		wallet.setUTXOProvider(new WalletAwareUTXOProvider(this, wallet));
+//
+//		Coin balance = wallet.getBalance();
+//		if (balance == null)
+//			return null;
+//
+//		return balance.value;
+	}
 
-		Coin balance = wallet.getBalance();
-		if (balance == null)
-			return null;
-
-		return balance.value;
+	public Long getWalletBalanceFromTransactions(String key58) throws ForeignBlockchainException {
+		long balance = 0;
+		Comparator<SimpleTransaction> oldestTimestampFirstComparator = Comparator.comparingLong(SimpleTransaction::getTimestamp);
+		List<SimpleTransaction> transactions = getWalletTransactions(key58).stream().sorted(oldestTimestampFirstComparator).collect(Collectors.toList());
+		for (SimpleTransaction transaction : transactions) {
+			balance += transaction.getTotalAmount();
+		}
+		return balance;
 	}
 
 	public List<SimpleTransaction> getWalletTransactions(String key58) throws ForeignBlockchainException {
-		Context.propagate(bitcoinjContext);
-
-		Wallet wallet = walletFromDeterministicKey58(key58);
-		DeterministicKeyChain keyChain = wallet.getActiveKeyChain();
-
-		keyChain.setLookaheadSize(Bitcoiny.WALLET_KEY_LOOKAHEAD_INCREMENT);
-		keyChain.maybeLookAhead();
-
-		List<DeterministicKey> keys = new ArrayList<>(keyChain.getLeafKeys());
-
-		Set<BitcoinyTransaction> walletTransactions = new HashSet<>();
-		Set<String> keySet = new HashSet<>();
-
-		// Set the number of consecutive empty batches required before giving up
-		final int numberOfAdditionalBatchesToSearch = 5;
-
-		int unusedCounter = 0;
-		int ki = 0;
-		do {
-			boolean areAllKeysUnused = true;
-
-			for (; ki < keys.size(); ++ki) {
-				DeterministicKey dKey = keys.get(ki);
-
-				// Check for transactions
-				Address address = Address.fromKey(this.params, dKey, ScriptType.P2PKH);
-				keySet.add(address.toString());
-				byte[] script = ScriptBuilder.createOutputScript(address).getProgram();
-
-				// Ask for transaction history - if it's empty then key has never been used
-				List<TransactionHash> historicTransactionHashes = this.blockchain.getAddressTransactions(script, false);
-
-				if (!historicTransactionHashes.isEmpty()) {
-					areAllKeysUnused = false;
-
-					for (TransactionHash transactionHash : historicTransactionHashes)
-						walletTransactions.add(this.getTransaction(transactionHash.txHash));
+		synchronized (this) {
+			// Serve from the cache if it's recent, and matches this xpub
+			if (Objects.equals(transactionsCacheXpub, key58)) {
+				if (transactionsCache != null && transactionsCacheTimestamp != null) {
+					Long now = NTP.getTime();
+					boolean isCacheStale = (now != null && now - transactionsCacheTimestamp >= TRANSACTIONS_CACHE_TIMEOUT);
+					if (!isCacheStale) {
+						return transactionsCache;
+					}
 				}
 			}
 
-			if (areAllKeysUnused) {
-				// No transactions
-				if (unusedCounter >= numberOfAdditionalBatchesToSearch) {
-					// ... and we've hit our search limit
-					break;
+			Context.propagate(bitcoinjContext);
+
+			Wallet wallet = walletFromDeterministicKey58(key58);
+			DeterministicKeyChain keyChain = wallet.getActiveKeyChain();
+
+			keyChain.setLookaheadSize(Bitcoiny.WALLET_KEY_LOOKAHEAD_INCREMENT);
+			keyChain.maybeLookAhead();
+
+			List<DeterministicKey> keys = new ArrayList<>(keyChain.getLeafKeys());
+
+			Set<BitcoinyTransaction> walletTransactions = new HashSet<>();
+			Set<String> keySet = new HashSet<>();
+
+			int unusedCounter = 0;
+			int ki = 0;
+			do {
+				boolean areAllKeysUnused = true;
+
+				for (; ki < keys.size(); ++ki) {
+					DeterministicKey dKey = keys.get(ki);
+
+					// Check for transactions
+					Address address = Address.fromKey(this.params, dKey, ScriptType.P2PKH);
+					keySet.add(address.toString());
+					byte[] script = ScriptBuilder.createOutputScript(address).getProgram();
+
+					// Ask for transaction history - if it's empty then key has never been used
+					List<TransactionHash> historicTransactionHashes = this.getAddressTransactions(script, false);
+
+					if (!historicTransactionHashes.isEmpty()) {
+						areAllKeysUnused = false;
+
+						for (TransactionHash transactionHash : historicTransactionHashes)
+							walletTransactions.add(this.getTransaction(transactionHash.txHash));
+					}
 				}
-				// We haven't hit our search limit yet so increment the counter and keep looking
-				unusedCounter++;
-			}
-			else {
-				// Some keys in this batch were used, so reset the counter
-				unusedCounter = 0;
-			}
 
-			// Generate some more keys
-			keys.addAll(generateMoreKeys(keyChain));
+				if (areAllKeysUnused) {
+					// No transactions
+					if (unusedCounter >= Settings.getInstance().getGapLimit()) {
+						// ... and we've hit our search limit
+						break;
+					}
+					// We haven't hit our search limit yet so increment the counter and keep looking
+					unusedCounter += WALLET_KEY_LOOKAHEAD_INCREMENT;
+				} else {
+					// Some keys in this batch were used, so reset the counter
+					unusedCounter = 0;
+				}
 
-			// Process new keys
-		} while (true);
+				// Generate some more keys
+				keys.addAll(generateMoreKeys(keyChain));
 
-		Comparator<SimpleTransaction> newestTimestampFirstComparator = Comparator.comparingInt(SimpleTransaction::getTimestamp).reversed();
+				// Process new keys
+			} while (true);
 
-		return walletTransactions.stream().map(t -> convertToSimpleTransaction(t, keySet)).sorted(newestTimestampFirstComparator).collect(Collectors.toList());
+			Comparator<SimpleTransaction> newestTimestampFirstComparator = Comparator.comparingLong(SimpleTransaction::getTimestamp).reversed();
+
+			// Update cache and return
+			transactionsCacheTimestamp = NTP.getTime();
+			transactionsCacheXpub = key58;
+			transactionsCache = walletTransactions.stream()
+					.map(t -> convertToSimpleTransaction(t, keySet))
+					.sorted(newestTimestampFirstComparator).collect(Collectors.toList());
+
+			return transactionsCache;
+		}
 	}
 
 	protected SimpleTransaction convertToSimpleTransaction(BitcoinyTransaction t, Set<String> keySet) {
@@ -411,19 +472,27 @@ public abstract class Bitcoiny implements ForeignBlockchain {
 		List<SimpleTransaction.Input> inputs = new ArrayList<>();
 		List<SimpleTransaction.Output> outputs = new ArrayList<>();
 
+		boolean anyOutputAddressInWallet = false;
+		boolean transactionInvolvesExternalWallet = false;
+
 		for (BitcoinyTransaction.Input input : t.inputs) {
 			try {
 				BitcoinyTransaction t2 = getTransaction(input.outputTxHash);
 				List<String> senders = t2.outputs.get(input.outputVout).addresses;
 				long inputAmount = t2.outputs.get(input.outputVout).value;
 				totalInputAmount += inputAmount;
-				for (String sender : senders) {
-					boolean addressInWallet = false;
-					if (keySet.contains(sender)) {
-						total += inputAmount;
-						addressInWallet = true;
+				if (senders != null) {
+					for (String sender : senders) {
+						boolean addressInWallet = false;
+						if (keySet.contains(sender)) {
+							total += inputAmount;
+							addressInWallet = true;
+						}
+						else {
+							transactionInvolvesExternalWallet = true;
+						}
+						inputs.add(new SimpleTransaction.Input(sender, inputAmount, addressInWallet));
 					}
-					inputs.add(new SimpleTransaction.Input(sender, inputAmount, addressInWallet));
 				}
 			} catch (ForeignBlockchainException e) {
 				LOGGER.trace("Failed to retrieve transaction information {}", input.outputTxHash);
@@ -431,23 +500,41 @@ public abstract class Bitcoiny implements ForeignBlockchain {
 		}
 		if (t.outputs != null && !t.outputs.isEmpty()) {
 			for (BitcoinyTransaction.Output output : t.outputs) {
-				for (String address : output.addresses) {
-					boolean addressInWallet = false;
-					if (keySet.contains(address)) {
-						if (total > 0L) {
-							amount -= (total - output.value);
-						} else {
-							amount += output.value;
+				if (output.addresses != null) {
+					for (String address : output.addresses) {
+						boolean addressInWallet = false;
+						if (keySet.contains(address)) {
+							if (total > 0L) { // Change returned from sent amount
+								amount -= (total - output.value);
+							} else { // Amount received
+								amount += output.value;
+							}
+							addressInWallet = true;
+							anyOutputAddressInWallet = true;
 						}
-						addressInWallet = true;
+						else {
+							transactionInvolvesExternalWallet = true;
+						}
+						outputs.add(new SimpleTransaction.Output(address, output.value, addressInWallet));
 					}
-					outputs.add(new SimpleTransaction.Output(address, output.value, addressInWallet));
 				}
 				totalOutputAmount += output.value;
 			}
 		}
 		long fee = totalInputAmount - totalOutputAmount;
-		return new SimpleTransaction(t.txHash, t.timestamp, amount, fee, inputs, outputs);
+
+		if (!anyOutputAddressInWallet) {
+			// No outputs relate to this wallet - check if any inputs did (which is signified by a positive total)
+			if (total > 0) {
+				amount = total * -1;
+			}
+		}
+		else if (!transactionInvolvesExternalWallet) {
+			// All inputs and outputs relate to this wallet, so the balance should be unaffected
+			amount = 0;
+		}
+		long timestampMillis = t.timestamp * 1000L;
+		return new SimpleTransaction(t.txHash, timestampMillis, amount, fee, inputs, outputs, null);
 	}
 
 	/**
@@ -483,7 +570,7 @@ public abstract class Bitcoiny implements ForeignBlockchain {
 				Address address = Address.fromKey(this.params, dKey, ScriptType.P2PKH);
 				byte[] script = ScriptBuilder.createOutputScript(address).getProgram();
 
-				List<UnspentOutput> unspentOutputs = this.blockchain.getUnspentOutputs(script, false);
+				List<UnspentOutput> unspentOutputs = this.blockchainProvider.getUnspentOutputs(script, false);
 
 				/*
 				 * If there are no unspent outputs then either:
@@ -501,7 +588,7 @@ public abstract class Bitcoiny implements ForeignBlockchain {
 					}
 
 					// Ask for transaction history - if it's empty then key has never been used
-					List<TransactionHash> historicTransactionHashes = this.blockchain.getAddressTransactions(script, false);
+					List<TransactionHash> historicTransactionHashes = this.blockchainProvider.getAddressTransactions(script, false);
 
 					if (!historicTransactionHashes.isEmpty()) {
 						// Fully spent key - case (a)
@@ -539,7 +626,7 @@ public abstract class Bitcoiny implements ForeignBlockchain {
 			this.keyChain = this.wallet.getActiveKeyChain();
 
 			// Set up wallet's key chain
-			this.keyChain.setLookaheadSize(Bitcoiny.WALLET_KEY_LOOKAHEAD_INCREMENT);
+			this.keyChain.setLookaheadSize(Settings.getInstance().getBitcoinjLookaheadSize());
 			this.keyChain.maybeLookAhead();
 		}
 
@@ -560,7 +647,7 @@ public abstract class Bitcoiny implements ForeignBlockchain {
 
 					List<UnspentOutput> unspentOutputs;
 					try {
-						unspentOutputs = this.bitcoiny.blockchain.getUnspentOutputs(script, false);
+						unspentOutputs = this.bitcoiny.blockchainProvider.getUnspentOutputs(script, false);
 					} catch (ForeignBlockchainException e) {
 						throw new UTXOProviderException(String.format("Unable to fetch unspent outputs for %s", address));
 					}
@@ -584,7 +671,7 @@ public abstract class Bitcoiny implements ForeignBlockchain {
 						// Ask for transaction history - if it's empty then key has never been used
 						List<TransactionHash> historicTransactionHashes;
 						try {
-							historicTransactionHashes = this.bitcoiny.blockchain.getAddressTransactions(script, false);
+							historicTransactionHashes = this.bitcoiny.blockchainProvider.getAddressTransactions(script, false);
 						} catch (ForeignBlockchainException e) {
 							throw new UTXOProviderException(String.format("Unable to fetch transaction history for %s", address));
 						}
@@ -637,7 +724,7 @@ public abstract class Bitcoiny implements ForeignBlockchain {
 		@Override
 		public int getChainHeadHeight() throws UTXOProviderException {
 			try {
-				return this.bitcoiny.blockchain.getCurrentHeight();
+				return this.bitcoiny.blockchainProvider.getCurrentHeight();
 			} catch (ForeignBlockchainException e) {
 				throw new UTXOProviderException("Unable to determine Bitcoiny chain height");
 			}

src/main/java/org/qortal/crosschain/BitcoinyBlockchainProvider.java

@@ -1,5 +1,7 @@
 package org.qortal.crosschain;
 
+import cash.z.wallet.sdk.rpc.CompactFormats.*;
+
 import java.util.List;
 
 public abstract class BitcoinyBlockchainProvider {
@@ -7,18 +9,32 @@ public abstract class BitcoinyBlockchainProvider {
 	public static final boolean INCLUDE_UNCONFIRMED = true;
 	public static final boolean EXCLUDE_UNCONFIRMED = false;
 
+	/** Sets the blockchain using this provider instance */
+	public abstract void setBlockchain(Bitcoiny blockchain);
+
 	/** Returns ID unique to bitcoiny network (e.g. "Litecoin-TEST3") */
 	public abstract String getNetId();
 
 	/** Returns current blockchain height. */
 	public abstract int getCurrentHeight() throws ForeignBlockchainException;
 
+	/** Returns a list of compact blocks, starting at <tt>startHeight</tt> (inclusive), up to <tt>count</tt> max.
+	 * Used for Pirate/Zcash only. If ever needed for other blockchains, the response format will need to be
+	 * made generic. */
+	public abstract List<CompactBlock> getCompactBlocks(int startHeight, int count) throws ForeignBlockchainException;
+
 	/** Returns a list of raw block headers, starting at <tt>startHeight</tt> (inclusive), up to <tt>count</tt> max. */
 	public abstract List<byte[]> getRawBlockHeaders(int startHeight, int count) throws ForeignBlockchainException;
 
+	/** Returns a list of block timestamps, starting at <tt>startHeight</tt> (inclusive), up to <tt>count</tt> max. */
+	public abstract List<Long> getBlockTimestamps(int startHeight, int count) throws ForeignBlockchainException;
+
 	/** Returns balance of address represented by <tt>scriptPubKey</tt>. */
 	public abstract long getConfirmedBalance(byte[] scriptPubKey) throws ForeignBlockchainException;
 
+	/** Returns balance of base58 encoded address. */
+	public abstract long getConfirmedAddressBalance(String base58Address) throws ForeignBlockchainException;
+
 	/** Returns raw, serialized, transaction bytes given <tt>txHash</tt>. */
 	public abstract byte[] getRawTransaction(String txHash) throws ForeignBlockchainException;
 
@@ -31,6 +47,12 @@ public abstract class BitcoinyBlockchainProvider {
 	/** Returns list of transaction hashes (and heights) for address represented by <tt>scriptPubKey</tt>, optionally including unconfirmed transactions. */
 	public abstract List<TransactionHash> getAddressTransactions(byte[] scriptPubKey, boolean includeUnconfirmed) throws ForeignBlockchainException;
 
+	/** Returns list of BitcoinyTransaction objects for <tt>address</tt>, optionally including unconfirmed transactions. */
+	public abstract List<BitcoinyTransaction> getAddressBitcoinyTransactions(String address, boolean includeUnconfirmed) throws ForeignBlockchainException;
+
+		/** Returns list of unspent transaction outputs for <tt>address</tt>, optionally including unconfirmed transactions. */
+	public abstract List<UnspentOutput> getUnspentOutputs(String address, boolean includeUnconfirmed) throws ForeignBlockchainException;
+
 	/** Returns list of unspent transaction outputs for address represented by <tt>scriptPubKey</tt>, optionally including unconfirmed transactions. */
 	public abstract List<UnspentOutput> getUnspentOutputs(byte[] scriptPubKey, boolean includeUnconfirmed) throws ForeignBlockchainException;