ExecutorCompletionService produces OOM

[marcosnils]

If consumers process slower than producers then you'll end up with a memory leak as ExecutorCompletionService created an unbounded LinkedBlockingQueue by default.

https://github.com/awslabs/dynamodb-import-export-tool/blob/623c333a7726cb80b01f2723a44a00951ec3cc64/src/main/java/com/amazonaws/dynamodb/bootstrap/BlockingQueueConsumer.java#L42

Also, the underlying ThreadPoolExecutor doesn't have a bounded blocking queue, which also causes OOM as infinite tasks can be scheduled into the executor:

https://github.com/awslabs/dynamodb-import-export-tool/blob/623c333a7726cb80b01f2723a44a00951ec3cc64/src/main/java/com/amazonaws/dynamodb/bootstrap/BlockingQueueConsumer.java#L41

Final question:

What's the purpose of having an ExecutorCompletionService here (https://github.com/awslabs/dynamodb-import-export-tool/blob/623c333a7726cb80b01f2723a44a00951ec3cc64/src/main/java/com/amazonaws/dynamodb/bootstrap/BlockingQueueConsumer.java#L42) if take or poll is never called?

[jseed]

+1 to this issue.
Using this tool on any of our larger tables generates an OutOfMemoryError, rendering it pretty much useless