Commit Graph

896 Commits

Author SHA1 Message Date
catbref
9e98ce220f Revert SysTray_ru.properties back to UTF8 form 2020-11-12 11:28:01 +00:00
catbref
10c3a0c056 Update AdvancedInstaller config file with v1.3.7 values 2020-11-12 11:27:36 +00:00
catbref
69ec654e4a Bump to version 1.3.7 2020-11-11 09:27:25 +00:00
catbref
a310e751bb Fix slow SQL query in HSQLDBATRepository.getBlockATStatesAtHeight() - mostly used during orphaning 2020-11-10 16:58:24 +00:00
catbref
3ef8b81e51 Fix incorrect column indexes when fetching frozen AT data 2020-11-10 15:38:15 +00:00
catbref
1f409235e4 Don't rebuild repository or export node-local data during repository build if repository was 'pristine'.
Under certain conditions, e.g. non-existent database files, the repository would be created
and then immediately be re-created.

Not only was this unnecessary, but HSQLDBDatabaseUpdates would attempt to export the node-local
data twice, which would cause an error due to existing .script files.

The fix is three-pronged:

1. Don't immediately rebuild the repository if it's only just been built
2. Don't export the empty node-local data if repository has only just been built
3. Don't export the node-local data if it's empty
2020-11-09 10:31:21 +00:00
catbref
806baa6ae4 Fix API call referenced in DB reshape 2020-11-06 11:23:35 +00:00
catbref
58ed72058f Another attempt to prevent "serialization failure" during trimming 2020-11-05 14:36:14 +00:00
catbref
253a994438 Add API POST /repository/checkpoint call. Renamed GET/POST /admin/repository calls to /admin/repository/data 2020-11-05 11:08:54 +00:00
catbref
5549eded38 Improve/fix use of latest block cache, for more cache hits, faster chain-tip response, etc. 2020-11-05 09:34:57 +00:00
catbref
20777363cf Split AT state storage into two HSQLDB table for better management
This involves a database reshape, but before this happens the node-local
data is exported to local files, giving the user the option to use a
bootstrap file instead of waiting.
2020-11-04 20:07:30 +00:00
catbref
b3f859f290 Don't use WITH COLUMN NAMES when exporting data from repository into local file 2020-11-04 15:48:52 +00:00
catbref
8c9f68a9c3 Add API calls for exporting node-local repository data & corresponding import to/from local files 2020-11-04 15:35:42 +00:00
catbref
41f178bf59 Add support for API key security, where X-API-KEY header must match apiKey from settings
apiKey in settings is null by default at this point, for backwards compatibility.
In the future, the Windows installer could generate a UUID for apiKey.
apiKey in settings needs to be at least 8 characters.

API calls in the documentation engine are now marked with an open/closed padlock
to show where API key might be required.
Add support for API key security, where X-API-KEY header must match apiKey from settings

apiKey in settings is null by default at this point, for backwards compatibility.
In the future, the Windows installer could generate a UUID for apiKey.
apiKey in settings needs to be at least 8 characters.

API calls in the documentation engine are now marked with an open/closed padlock
to show where API key might be required.
2020-11-04 15:35:20 +00:00
catbref
ad5050f92e Add support for exporting node-local repository data to .script files and corresponding import function 2020-11-04 15:29:10 +00:00
catbref
16397852ae Add synchronization around updating trim heights to prevent deadlock/rollback 2020-11-04 10:01:20 +00:00
catbref
c125a53655 More (optionally) logging when comparing chains with peers. Support for potential future minor consensus change. 2020-11-02 11:52:06 +00:00
catbref
7b056a832f Turn off HSQLDB redo-log "blockchain.log" and periodically call "CHECKPOINT" instead.
Checkpointing interval is 1 hour by default, changable in settings via
"repositoryCheckpointInterval"
plus corresponding "showCheckpointNotifications" SysTray flags (off by default).

Added entries to SysTray_en i18n properties, and converted SysTray_ru to ISO-8559-1.
2020-11-02 11:49:21 +00:00
catbref
6c40727027 More reporting for slow HSQLDB queries/commits 2020-11-02 11:16:40 +00:00
catbref
8f06765caf Networking performance improvements and message sending bugfix 2020-11-02 11:15:53 +00:00
catbref
de2fc78ad1 Add support for SHA256 digest of ByteBuffer in Crypto class 2020-11-02 10:47:27 +00:00
catbref
ee08410260 More trace-level debugging in Synchronizer to help diagnose chain reorg issues 2020-11-02 10:46:51 +00:00
catbref
88da8d949f Don't allow latest blocks cache to be empty 2020-11-02 10:45:21 +00:00
catbref
d2a92db921 More caching for GetBlockMessage. Added API call GET /admin/enginestats to monitor cache usage 2020-10-29 11:02:02 +00:00
catbref
9c18a33d7f Improve tools/build-auto-update.sh when working on detached HEAD 2020-10-28 09:08:16 +00:00
catbref
f3b8258067 Add more latest block caching to reduce repository accesses, especially for requests from remote peers 2020-10-28 08:46:30 +00:00
catbref
da78c73485 Remove extraneous call to Controller.onNewBlock() after synchronization, as this call is performed per-block inside Synchronizer 2020-10-28 08:42:59 +00:00
catbref
cec25ce279 Add API call POST /peers/commonblock <connected-peer> as debugging aid 2020-10-28 08:41:23 +00:00
catbref
0389007491 Skip trimming while performing synchronization 2020-10-28 08:38:11 +00:00
catbref
38a64bdd9e Prevent HSQLDB prepared statement cache invalidation when rebuilding latest AT states cache 2020-10-12 14:35:10 +01:00
catbref
6a24f787c4 Bump to v1.3.6 2020-10-07 14:56:58 +01:00
catbref
98564aa8bf Fix SQL logic error when fetching trade offers 2020-10-07 14:56:05 +01:00
catbref
9ceff90f42 Upgrade to CIYAM-AT v1.3.8 with slight performance improvements 2020-10-07 10:31:18 +01:00
catbref
6a4388fecc Use cached PreparedStatement for HSQLDB.assertEmptyTransaction + other minor HSQLDB fixes 2020-10-07 09:45:44 +01:00
catbref
1958444bc4 Add recipient indexes for payment/AT transactions to speed up AT processing 2020-10-06 14:09:42 +01:00
catbref
a2038274e1 Keep latest AT state, even if "finished", so we can produce historical trade data 2020-10-05 15:18:41 +01:00
catbref
532c697026 Moved AT State & online signatures trimming intervals, batch sizes, limits, etc. to Settings 2020-10-02 12:58:23 +01:00
catbref
5cf5c1e1f7 Take pressure off GC by not creating/destroying HSQLDB sub-repositories all the time 2020-10-01 13:18:24 +01:00
catbref
60621e8b81 Reworked AT-states and online signatures trimming
Instead of searching from block 0, we now keep a record of
base trim height in the DB itself.

Also, we no longer trim the latest AT state for non-finished ATs
in case they are in deep sleeping and we need their state for when
they awaken.
2020-10-01 13:17:00 +01:00
catbref
a6a1f65d3e Reduce block search size in AT state trimmer to reduce load 2020-09-29 11:41:30 +01:00
catbref
a681f741dd Add initial delay before trimming online accounts signatures 2020-09-29 11:40:41 +01:00
catbref
bed9837967 Added settings entry "localeLang" for controlling core language (not-API) 2020-09-29 10:56:27 +01:00
catbref
855cb2226a Aggressively trim old AT state data and online accounts signatures.
Two new classes/threads made to quickly find first trimmable row
then repeatedly trim rows in small batches after that.
2020-09-28 14:34:00 +01:00
catbref
d85a3d17c8 Fix for HSQLDB deadlock during CHECKPOINT.
Symptoms are:

* db/blockchain.log is pretty much exactly 50MB - the checkpoint-triggering size.

* Loads of threads are stuck waiting for HSQLDB's CountUpDownLatch$Sync.await()

* Synchronizer, or some other thread, possibly orphaning blocks.

The cause seems to be method A, which has a repository session,
calls EventBus.INSTANCE.notify() and one of the event listeners
then obtains their own repository session to do repository 'work'.

In the meantime, the HSQLDB log has reached 50MB, triggering auto-checkpoint.

HSQLDB attempts to CHECKPOINT, but waits for existing transactions
to complete, and also blocks starting new transactions.

Thus, one of the event listeners is blocked when they try to obtain
a new repository session, but HSQLDB never performs CHECKPOINT
because the event notifier (method A) still has an unfinished
transaction - hence deadlock.
2020-09-28 14:22:18 +01:00
catbref
81a5b154c2 Add API call DELETE /admin/repository which actually performs repository maintenance (takes several minutes) 2020-09-25 17:06:06 +01:00
catbref
a6f42df9d6 Add isTestNet to API call GET /admin/info 2020-09-25 16:35:53 +01:00
catbref
17ae7acc6d Reduce DB storage of AT states
Drop created_when column from ATStates as it never changes
and can be fetched from ATs table.
This takes about 50s on a fast machine.

Correspondingly rebuild height-based index on ATStates.
This takes about 3 minutes on a fast machine.

Modify AT-related repository methods and callers.

Aggressively remove 'old' (> 2 weeks) actual AT
state binary data, leaving only the hash in DB
(for syncing purposes). Seems to keep up with
syncing from another node on localhost.
2020-09-25 15:25:57 +01:00
catbref
3d5fec3c30 Bump to HSQLDB v2.5.1 as we seem clear of OOM issue 2020-09-25 15:25:15 +01:00
catbref
21f48fba5f HSQLDB PreparedStatement caching improvements 2020-09-24 12:46:30 +01:00
catbref
d0da5d7c48 ATs: only call MachineState.getCodeBytes() once in preparation for using newer AT lib 2020-09-24 12:46:17 +01:00