Default thread limits per message type can be specified in Settings.setAdditionalDefaults(), e.g
maxThreadsPerMessageType.add(new ThreadLimit("GET_ARBITRARY_DATA_FILE", 5));
These can also be overridden on a per-node basis in settings.json, e.g
"maxThreadsPerMessageType": [
{ "messageType": "GET_ARBITRARY_DATA_FILE", "limit": 3 },
{ "messageType": "GET_ARBITRARY_DATA_FILE_LIST", "limit": 3 }
]
settings.json values take priority, but any message types that aren't specified in settings.json will still be included from the Settings.java defaults. This allows single message types to be overridden in settings.json without removing the limits for all of the other message types.
Any messages that arrive are discarded if the node is already at the thread limit for that message type.
Warnings are now shown in the logs if the total number of active threads reaches 90% of the allocated thread pool size. Additionally, it can warn per message type by specifying a per-message-type warning threshold in settings.json, e.g
"threadCountPerMessageTypeWarningThreshold": 20
The above setting would warn in the logs if a single message type was consuming more than 20 threads at once, therefore making it a candidate to be limited in maxThreadsPerMessageType.
Initial values of maxThreadsPerMessageType are guesses and may need modifying based on real world results. Limiting threads may impact functionality, so this should be carefully tested.
Also be aware that the thread tracking may reduce network performance slightly, so be sure to test thoroughly on slower hardware.
