Implemented automatic PUT/PATCH detection

When using POST /arbitrary/{service}/{name}... it will now automatically decide which method to use (PUT/PATCH) based on a few factors:

- If there are already 10 or more layers, use PUT to reset back to a single layer
- If the next layer's patch is more than 20% of the total resource file size, use PUT
- If the next layer modifies more than 50% of the total file count, use PUT
- Otherwise, use PATCH

The PUT method causes a new base layer to be created and all previous update history for that resource becomes obsolete. The PATCH method adds a small delta layer on top of the existing layer(s).

The idea is to wipe the slate clean with a new base layer once the patches start to get demanding for the network to apply. Nodes which view the content will ultimately have build timeouts to prevent someone from deploying a resource with hundreds of complex layers for example, so this approach is there to maximize the chances of the resource being buildable.

The constants above (10 layers, 20% total size, 50% file count) will most likely need tweaking once we have some real world data.
This commit is contained in:
CalDescent
2021-11-13 09:56:13 +00:00
parent e994d501b0
commit c069c39ce1
9 changed files with 158 additions and 11 deletions

View File

@@ -8,7 +8,7 @@ if [ -z "$*" ]; then
echo "Usage:"
echo
echo "Host/update data:"
echo "qdata [PUT/PATCH] [service] [name] [dirpath] <identifier>"
echo "qdata [POST/PUT/PATCH] [service] [name] [dirpath] <identifier>"
echo
echo "Fetch data:"
echo "qdata GET [service] [name] <identifier-or-default> <filepath-or-default> <rebuild>"
@@ -36,7 +36,7 @@ if [ -z "${name}" ]; then
fi
if [[ "${method}" == "PUT" || "${method}" == "PATCH" ]]; then
if [[ "${method}" == "POST" || "${method}" == "PUT" || "${method}" == "PATCH" ]]; then
directory=$4
identifier=$5