crowetic/qortal
1
0
Fork 0
forked from Qortal/qortal
Code Pull Requests Activity
1,274 Commits 85 Branches 118 Tags 530 MiB
49eddc9da5
Commit Graph

1274 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
CalDescent
49eddc9da5 Allow zero fee transactions if the fee is zero in blockchain.json 
Until now it wasn't possible to set up a chain with zero transaction fees due to a hardcoded zero check in Payment.isValid(), and a divide by zero error in Transaction.hasMinimumFeePerByte()
 2021-07-04 09:31:51 +01:00
CalDescent
4b1de108d1 Fixed bug in expected chunk count. 2021-07-03 18:42:42 +01:00
CalDescent
e46c735efa Fixed recently introduced bugs with file management. 2021-07-03 18:05:17 +01:00
CalDescent
56da7deb4c DataFile updates to simplify integration with arbitrary transactions. 2021-07-03 17:41:52 +01:00
CalDescent
5f4649ee2b Major upgrade of arbitrary transactions 
- Adds support for files up 500MiB per transaction (at 2MiB chunk sizes). Previously, the max data size was 4000 bytes.
- Adds a nonce, giving us the option to remove the transaction fees altogether on the data chain.

These features become enabled in version 5 of arbitrary transactions.
 2021-07-03 17:40:02 +01:00
CalDescent
7cc2c4f621 Progress on website API endpoints. 2021-07-02 08:57:55 +01:00
CalDescent
cc449f9304 Added DataFile.chunkHashes() method which appends all hashes into a single byte array (32 bytes / 256 bits each). 2021-07-02 08:54:05 +01:00
CalDescent
28425efbe7 Added jsoup dependency - this was missing from an earlier commit. 2021-07-02 08:52:38 +01:00
CalDescent
8c3a22aa5c Improved link replacement criteria. 2021-07-01 11:21:40 +01:00
CalDescent
f3e5933599 Fixed naming error in joinFiles API. 2021-07-01 11:18:46 +01:00
CalDescent
aac4fe37e8 Fixed API response description. 2021-06-26 10:28:22 +01:00
CalDescent
ebfa941a4f Fixed some file separators. 2021-06-26 10:27:32 +01:00
CalDescent
47c70eea9e Use system temp directory instead of making a "temp" subfolder when zipping files. 2021-06-26 10:11:34 +01:00
CalDescent
fe7c40cb7c Reduced log spam. 2021-06-25 19:31:28 +01:00
CalDescent
8973626a4b Fixed issue with temp directory on Linux. 2021-06-25 19:31:13 +01:00
CalDescent
ace5d999e2 Log a comma separated list of hashes after splitting a file into chunks, so they can easily be requested from another node using the //data/files/frompeer/{peer} API endpoint. Again temporary until the sync happens automatically. 2021-06-25 19:30:45 +01:00
CalDescent
52829a244b More work on APIs to request files from peers. I won't spend too long making these perfect as they are mostly a temporary measure. 2021-06-25 19:28:39 +01:00
CalDescent
71c247fe56 Added POST /data/file/{hash}/build API, used to join multiple chunks together. 2021-06-25 19:28:01 +01:00
CalDescent
b34066f579 More work on HTML parsing. 
The style tag parsing ideally needs rewriting using an actual CSS parser, but we can get away with this hacky approach in the short term.
 2021-06-25 08:34:44 +01:00
CalDescent
b286c15c51 Optimized website serving, and added code to return the correct content types. 
This is probably the most efficient way to process the data on the fly, but it's still not very scalable. A better approach would be to pre-process the HTML when building the file structure, and then serve them completely statically (i.e. using a standard webserver rather than via application memory). But it makes sense to keep it this way for development and maybe early beta testing.
 2021-06-25 08:32:22 +01:00
CalDescent
ea5e2f5580 Added POST /site/preview API 
This can be used to preview a site before signing a transaction and announcing it to the network. The response will need reworking to return JSON (along with most of the other new APIs)
 2021-06-23 09:28:38 +01:00
CalDescent
b65c7a75fe Handle relative links when parsing HTML. 2021-06-23 09:26:41 +01:00
CalDescent
39f5dce51c Moved "directory" data uploads to new POST /site/upload API. 
Directory uploads don't make much sense outside of website hosting, so it's best to make this API specific to that purpose.
 2021-06-23 09:10:06 +01:00
CalDescent
f77ec1faf6 A very crude proof of concept which serves a website from a zipped (and in future, chunked) data blob. This forms the beginnings of the "website hosting" layer on top of the data storage. It needs a significant rework - most importantly so that we aren't serving every asset from memory, and also so that the correct content-type headers are returned, etc. 2021-06-23 08:43:21 +01:00
CalDescent
cd3a1e0159 Increased max file size to 1GiB. Will review this again later. 2021-06-23 08:33:53 +01:00
CalDescent
3f20fadb81 When zipping files, rename the outer folder to "data" instead of using the original folder name. This means that the data can be accessed deterministically without the need to first lookup the folder name. 2021-06-23 08:09:59 +01:00
CalDescent
1c6428dd3b Added equivalent split and join test but this time using a 5.5MiB source file. 2021-06-23 08:07:35 +01:00
CalDescent
aca620241a More work on data file split/join, and added a test. 2021-06-22 08:58:16 +01:00
CalDescent
808b36e088 Specify chunk size when splitting. 2021-06-22 07:44:34 +01:00
CalDescent
1613375cc0 Added more validation of files received in GET /data/file/frompeer 2021-06-21 19:03:34 +01:00
CalDescent
787ef957d2 Added support for uploading an entire directory via POST /data/upload/path 
If a directory is specified instead of a file, the directory is automatically zipped before being split into chunks.
 2021-06-21 19:02:49 +01:00
CalDescent
b915d0aed5 Only create the output file directories when we are actually writing a file there. This should prevent empty directories being created when initializing a nonexistent DataFile using a hash. 2021-06-21 08:40:52 +01:00
CalDescent
16dc5b5327 Include the data length in DataFileMessage, which is more similar to the approach used by ArbitraryDataMessage. These two message types are very similar, except arbitrary code currently has a requirement of one piece of data per signature, whereas DataFile code is independent and can support multiple files per transaction. Maybe the two can be combined at some point, but for now I'll keep them separate. 2021-06-20 19:49:10 +01:00
CalDescent
c2d0c63db0 Improved error logging. 2021-06-19 20:31:04 +01:00
CalDescent
f5c9807a48 Use contains() rather than equals() when matching a peer in /data/file/frompeer, so that the port can be optionally left out. 2021-06-19 20:29:05 +01:00
CalDescent
7e9b1d5e16 Rework of DataFile.base58Digest() 
This fixes an NPE when trying to send a file that doesn't exist. It also removes the caching, which we can add again later if it turns out to be needed.
 2021-06-19 20:25:25 +01:00
CalDescent
5070c4eea9 Better handling of data file responses in the /data/file/frompeer API endpoint. 2021-06-19 20:23:33 +01:00
CalDescent
33d9c51b6f Validate supplied base58 string in /data/file/frompeer API endpoint 2021-06-19 19:26:13 +01:00
CalDescent
d0f9d478c2 Fixed bug which prevented the DATA_FILE message ID from making it through to the reply queue. 2021-06-19 19:05:11 +01:00
CalDescent
f296ec46c8 Removed unused headers. 2021-06-19 17:32:55 +01:00
CalDescent
64d19e480b Chunk size set to 1MB for now, as it seems that our networking code has problems when transferring 2MB chunks. We can increase this later once that problem has been fixed. 2021-06-19 17:32:48 +01:00
CalDescent
8e35f131d5 Removed debugging log that wasn't intended to be committed. 2021-06-18 08:33:51 +01:00
CalDescent
aafb9d7e4f Include running total in "Sent X bytes" log entry. 2021-06-18 08:05:35 +01:00
CalDescent
652f30bdbd Added DATA_FILE and GET_DATA_FILE message types. 2021-06-18 08:02:57 +01:00
CalDescent
f4ba7b2a0c Added onNetworkGetDataFileMessage() handler 2021-06-18 08:02:13 +01:00
CalDescent
592490d709 Added GET /data/file/frompeer API endpoint 
This requests a file from the supplied peer address, and stores a copy locally if successful. Still a work in progress.
 2021-06-18 07:59:46 +01:00
CalDescent
5ac676d201 Added DataFileMessage and GetDataFileMessage, used for requesting and sending files between peers. 2021-06-17 19:05:00 +01:00
CalDescent
abfe0a925a More DataFile methods and improvements 2021-06-17 18:20:42 +01:00
CalDescent
fa11f4f45b Moved DataFileChunk(byte[] fileContent) constructor to the superclass so it can be used by regular data files too. 2021-06-17 18:20:11 +01:00
CalDescent
1e8dbfe4b7 Delete chunk if it fails the hash validation in the constructor. 2021-06-17 18:18:19 +01:00