Faster Processing Throughput with Concurrent Virus Scanning

Background

During peak usage, large Jira instances can experience over 1000 attachment uploads per minute.

However, the scanning capabilities are significantly slower, for example:

• clamscan can only process 8 to 10 attachments per minute.

• clamdscan, while faster, can only handle up to 100 attachments per minute.

This discrepancy between the upload rate and the scanning rate leads to a snowball effect in the queue, as the number of unscanned attachments accumulates rapidly.

How it works

To address this, we've implemented a "smart assignment" module that:

• Assigns attachments pending scan to active Jira nodes.

• Reassigns attachments to other active nodes if any Jira node goes down.

Benefits

This significantly increases processing throughput by:

• 2x if there are 2 nodes

• 3x if there are 3 nodes

Furthermore, it reduces the latency between an attachment upload and the warning comment added on the Jira work item for infected attachments.

How many nodes do you need

The number of nodes required will depend on:

• The speed of virus scans

• The rate at which users upload new files