AlphaX/qortal
1
0
Fork 0
forked from Qortal/qortal
Code Issues Pull Requests Projects Releases Wiki Activity
1,302 Commits 77 Branches 112 Tags 430 MiB
8a654834ac
Commit Graph

1302 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
CalDescent
39f5dce51c Moved "directory" data uploads to new POST /site/upload API. 
Directory uploads don't make much sense outside of website hosting, so it's best to make this API specific to that purpose.
 2021-06-23 09:10:06 +01:00
CalDescent
f77ec1faf6 A very crude proof of concept which serves a website from a zipped (and in future, chunked) data blob. This forms the beginnings of the "website hosting" layer on top of the data storage. It needs a significant rework - most importantly so that we aren't serving every asset from memory, and also so that the correct content-type headers are returned, etc. 2021-06-23 08:43:21 +01:00
CalDescent
cd3a1e0159 Increased max file size to 1GiB. Will review this again later. 2021-06-23 08:33:53 +01:00
CalDescent
3f20fadb81 When zipping files, rename the outer folder to "data" instead of using the original folder name. This means that the data can be accessed deterministically without the need to first lookup the folder name. 2021-06-23 08:09:59 +01:00
CalDescent
1c6428dd3b Added equivalent split and join test but this time using a 5.5MiB source file. 2021-06-23 08:07:35 +01:00
CalDescent
aca620241a More work on data file split/join, and added a test. 2021-06-22 08:58:16 +01:00
CalDescent
808b36e088 Specify chunk size when splitting. 2021-06-22 07:44:34 +01:00
CalDescent
1613375cc0 Added more validation of files received in GET /data/file/frompeer 2021-06-21 19:03:34 +01:00
CalDescent
787ef957d2 Added support for uploading an entire directory via POST /data/upload/path 
If a directory is specified instead of a file, the directory is automatically zipped before being split into chunks.
 2021-06-21 19:02:49 +01:00
CalDescent
b915d0aed5 Only create the output file directories when we are actually writing a file there. This should prevent empty directories being created when initializing a nonexistent DataFile using a hash. 2021-06-21 08:40:52 +01:00
CalDescent
16dc5b5327 Include the data length in DataFileMessage, which is more similar to the approach used by ArbitraryDataMessage. These two message types are very similar, except arbitrary code currently has a requirement of one piece of data per signature, whereas DataFile code is independent and can support multiple files per transaction. Maybe the two can be combined at some point, but for now I'll keep them separate. 2021-06-20 19:49:10 +01:00
CalDescent
c2d0c63db0 Improved error logging. 2021-06-19 20:31:04 +01:00
CalDescent
f5c9807a48 Use contains() rather than equals() when matching a peer in /data/file/frompeer, so that the port can be optionally left out. 2021-06-19 20:29:05 +01:00
CalDescent
7e9b1d5e16 Rework of DataFile.base58Digest() 
This fixes an NPE when trying to send a file that doesn't exist. It also removes the caching, which we can add again later if it turns out to be needed.
 2021-06-19 20:25:25 +01:00
CalDescent
5070c4eea9 Better handling of data file responses in the /data/file/frompeer API endpoint. 2021-06-19 20:23:33 +01:00
CalDescent
33d9c51b6f Validate supplied base58 string in /data/file/frompeer API endpoint 2021-06-19 19:26:13 +01:00
CalDescent
d0f9d478c2 Fixed bug which prevented the DATA_FILE message ID from making it through to the reply queue. 2021-06-19 19:05:11 +01:00
CalDescent
f296ec46c8 Removed unused headers. 2021-06-19 17:32:55 +01:00
CalDescent
64d19e480b Chunk size set to 1MB for now, as it seems that our networking code has problems when transferring 2MB chunks. We can increase this later once that problem has been fixed. 2021-06-19 17:32:48 +01:00
CalDescent
8e35f131d5 Removed debugging log that wasn't intended to be committed. 2021-06-18 08:33:51 +01:00
CalDescent
aafb9d7e4f Include running total in "Sent X bytes" log entry. 2021-06-18 08:05:35 +01:00
CalDescent
652f30bdbd Added DATA_FILE and GET_DATA_FILE message types. 2021-06-18 08:02:57 +01:00
CalDescent
f4ba7b2a0c Added onNetworkGetDataFileMessage() handler 2021-06-18 08:02:13 +01:00
CalDescent
592490d709 Added GET /data/file/frompeer API endpoint 
This requests a file from the supplied peer address, and stores a copy locally if successful. Still a work in progress.
 2021-06-18 07:59:46 +01:00
CalDescent
5ac676d201 Added DataFileMessage and GetDataFileMessage, used for requesting and sending files between peers. 2021-06-17 19:05:00 +01:00
CalDescent
abfe0a925a More DataFile methods and improvements 2021-06-17 18:20:42 +01:00
CalDescent
fa11f4f45b Moved DataFileChunk(byte[] fileContent) constructor to the superclass so it can be used by regular data files too. 2021-06-17 18:20:11 +01:00
CalDescent
1e8dbfe4b7 Delete chunk if it fails the hash validation in the constructor. 2021-06-17 18:18:19 +01:00
CalDescent
f82f2bd287 Added DELETE /data/file API endpoint 
This deletes a file referenced by a user supplied SHA256 digest string (which we will use as the file's "ID" in the Qortal data system). In the future this could be extended to delete all associated chunks, but first we need to build out the data chain so we have a way to look up chunks associated with a file hash.
 2021-06-16 20:03:15 +01:00
CalDescent
76742c3869 Removed .dat extension, and use an extra level in the directory structure (data/aB/cD/aBcDeF... etc) 2021-06-16 19:49:12 +01:00
CalDescent
9407e7e418 Data storage location moved to settings ("dataPath") 2021-06-16 19:22:50 +01:00
CalDescent
120552b36e Added resource file missing from last commit. 2021-06-16 19:13:24 +01:00
CalDescent
9fb58c7ae3 Added the beginnings of an upload API, with some basic data file management. 2021-06-16 19:10:35 +01:00
CalDescent
b917da765c Removed block 212937 2021-06-15 09:29:07 +01:00
CalDescent
cc59510cd0 Removed all cross-chain code. It's not needed for data nodes. 2021-06-15 09:27:05 +01:00
CalDescent
86aab7023c Initial settings for data node development. Most to be decided later. 2021-06-14 19:40:37 +01:00
CalDescent
e3b0a41ba9 Merge branch 'sync-multiple-blocks' 2021-06-14 19:01:17 +01:00
CalDescent
802c55dfc8 Merge branch 'networking' 2021-06-14 19:00:17 +01:00
CalDescent
280f7814aa Merge branch 'master' of github.com:Qortal/qortal 2021-06-14 18:49:55 +01:00
CalDescent
3174681bd8 Removed /src/main/resources/log*.properties from .gitignore 
We must be careful not to add files to the resources folder accidentally, given that a bundled log4j2.properties file is used in preference to the user's copy. By keeping this out of gitignore, it becomes more obvious if a file is added, and it can then be caught and removed before a release.
 2021-06-14 18:49:24 +01:00
CalDescent
853f80b928 Updates to build-zip.sh and build-release.sh 
There were necessary for these scripts to function in my build environment (Mac OSX). This may give errors when running in other environments, but we can deal with that in future, when others need to use these scripts.
 2021-06-14 18:46:21 +01:00
CalDescent
8bdad377d7 Removed outdated qortal.jar from "WindowsInstaller/Install Files" directory. 
Including an older JAR in the source code only leads to confusion, because a zip of the source code is automatically included with each github release. From what I can see, there is no need for it to be here. Added to .gitignore so we have the option of keeping a local copy.
 2021-06-14 18:41:27 +01:00
CalDescent
9e1c2a5bd1 Updated AdvancedInstaller project for v1.5.4 2021-06-14 18:39:46 +01:00
CalDescent
b1777b6011 Bump version to 1.5.4 2021-06-13 19:22:16 +01:00
CalDescent
904be3005f Enable fast sync by default. 2021-06-12 13:07:25 +01:00
CalDescent
95eaf4c887 Merge branch 'master' into sync-multiple-blocks 2021-06-12 11:15:28 +01:00
CalDescent
e3923b7b22 Fixed issue causing frequent disconnects (found by szisti) 
When sending or requesting more than 1000 online accounts, peers would be disconnected with an EOF or connection reset error due to an intentional null response. This response has been removed and it will instead now only send the first 1000 accounts, which prevents the disconnections from occurring.

In theory, these accounts should be in a different order on each node, so the 1000 limit should still result in a fairly even propagation of accounts. However, we may want to consider increasing this limit, to maximise the propagation speed.

Thanks to szisti for tracking this one down.
 2021-06-11 19:10:04 +01:00
CalDescent
a43993e3ec Use placeholder build timestamp and build version when building without mvn package (e.g. from within an IDE) 
This fixes an exception thrown when running directly in IntelliJ, as previously we relied on mvn package to parse the commit hash and timestamp.
 2021-06-11 19:01:35 +01:00
CalDescent
bc6b3fb5f4 Include timestamps in block-timings.sh 2021-06-09 13:04:49 +01:00
CalDescent
df47f5d47b Merge branch 'master' into sync-multiple-blocks 2021-06-06 10:35:47 +01:00