Avoid having large requests go through Redis

[MichaelRipa]

This addresses a known issue of Redis having an implicit upperbound of 512 MB key/values, and of Redis being slow with large payloads. Before this fix, the API would throw a cryptic error if a request exceeded said upperbound. We now enforce an explicit upperbound, and route larger requests from api to queue by dumping to /tmp instead of through Redis.

Introduced Environment variables

• NDIF_QUEUE_TEMP_FILE_THRESHOLD : Determines when to route requests to /tmp (rather then redis, defaults to 16MB)
• NDIF_MAX_REQUEST_SIZE : Maximum content-length of request (defaults to 1GB)

[MichaelRipa]

Benchmarks on dev stack:

With tmp file routing (starting at 16 MB):

================================================================================
Size         Bytes        Mean (s)   Std (s)    Success    Times
--------------------------------------------------------------------------------
1.00 KB      1024         0.56       0.37       5/5        1.21, 0.39, 0.39, 0.39, 0.40
2.00 KB      2048         0.45       0.09       5/5        0.60, 0.44, 0.40, 0.40, 0.39
4.00 KB      4096         0.46       0.02       5/5        0.49, 0.45, 0.43, 0.45, 0.46
8.00 KB      8192         0.45       0.02       5/5        0.45, 0.43, 0.48, 0.46, 0.44
16.00 KB     16384        0.44       0.02       5/5        0.46, 0.47, 0.43, 0.41, 0.44
32.00 KB     32768        0.42       0.02       5/5        0.42, 0.41, 0.41, 0.41, 0.45
64.00 KB     65536        0.43       0.02       5/5        0.42, 0.41, 0.46, 0.44, 0.43
128.00 KB    131072       0.44       0.01       5/5        0.45, 0.44, 0.44, 0.45, 0.42
256.00 KB    262144       0.47       0.02       5/5        0.48, 0.44, 0.48, 0.47, 0.48
512.00 KB    524288       0.51       0.02       5/5        0.52, 0.50, 0.50, 0.54, 0.48
1.00 MB      1048576      0.53       0.02       5/5        0.51, 0.52, 0.55, 0.50, 0.56
2.00 MB      2097152      0.56       0.02       5/5        0.56, 0.55, 0.59, 0.57, 0.54
4.00 MB      4194304      0.65       0.04       5/5        0.65, 0.58, 0.63, 0.67, 0.70
8.00 MB      8388608      0.77       0.04       5/5        0.81, 0.78, 0.77, 0.78, 0.71
16.00 MB     16777216     1.08       0.19       5/5        1.31, 1.12, 0.90, 0.87, 1.20
32.00 MB     33554432     1.29       0.05       5/5        1.27, 1.31, 1.27, 1.37, 1.25
64.00 MB     67108864     2.50       0.23       5/5        2.60, 2.34, 2.87, 2.36, 2.33
128.00 MB    134217728    4.76       0.09       5/5        4.80, 4.83, 4.61, 4.76, 4.83
256.00 MB    268435456    11.33      0.24       5/5        11.02, 11.64, 11.34, 11.20, 11.47
512.00 MB    536870912    32.80      0.77       5/5        31.91, 33.65, 32.56, 33.56, 32.33
1.00 GB      1073741824   108.95     7.99       5/5        103.92, 106.10, 123.14, 105.28, 106.28
================================================================================

Without tmp file routing:

================================================================================
Size         Bytes        Mean (s)   Std (s)    Success    Times
--------------------------------------------------------------------------------
1.00 KB      1024         0.55       0.31       5/5        1.11, 0.39, 0.41, 0.39, 0.46
2.00 KB      2048         0.41       0.03       5/5        0.46, 0.42, 0.39, 0.40, 0.40
4.00 KB      4096         0.44       0.11       5/5        0.40, 0.40, 0.39, 0.39, 0.64
8.00 KB      8192         0.39       0.01       5/5        0.39, 0.40, 0.38, 0.41, 0.39
16.00 KB     16384        0.42       0.02       5/5        0.41, 0.42, 0.45, 0.39, 0.41
32.00 KB     32768        0.40       0.01       5/5        0.40, 0.39, 0.40, 0.41, 0.41
64.00 KB     65536        0.44       0.03       5/5        0.49, 0.43, 0.43, 0.43, 0.43
128.00 KB    131072       0.44       0.01       5/5        0.45, 0.44, 0.42, 0.44, 0.45
256.00 KB    262144       0.49       0.02       5/5        0.49, 0.51, 0.48, 0.51, 0.48
512.00 KB    524288       0.57       0.11       5/5        0.53, 0.51, 0.76, 0.50, 0.56
1.00 MB      1048576      0.57       0.03       5/5        0.56, 0.62, 0.59, 0.54, 0.55
2.00 MB      2097152      0.58       0.02       5/5        0.61, 0.57, 0.57, 0.56, 0.58
4.00 MB      4194304      0.60       0.03       5/5        0.64, 0.58, 0.61, 0.58, 0.58
8.00 MB      8388608      0.86       0.16       5/5        0.79, 0.95, 1.09, 0.74, 0.70
16.00 MB     16777216     0.99       0.10       5/5        0.97, 1.16, 0.91, 1.01, 0.92
32.00 MB     33554432     1.52       0.20       5/5        1.47, 1.88, 1.46, 1.40, 1.42
64.00 MB     67108864     2.65       0.03       5/5        2.65, 2.63, 2.69, 2.63, 2.62
128.00 MB    134217728    5.42       0.10       5/5        5.34, 5.56, 5.37, 5.47, 5.34
256.00 MB    268435456    12.62      0.18       5/5        12.85, 12.74, 12.41, 12.58, 12.52
512.00 MB    536870912    37.92      2.12       5/5        37.13, 40.82, 36.94, 39.27, 35.43
1.00 GB      1073741824   N/A        N/A        0/5        none
================================================================================