Failure: Received message larger than max

[pjweinb]

MacOS Catalina, 10.15.5, bazel 3.4.1. (setup was perfectly smooth)
Created example with
bazel-bin/generate_dummy_data
--server_data_file=/tmp/dummy_server_data.csv
--client_data_file=/tmp/dummy_client_data.csv --server_data_size=10000
--client_data_size=10000 --intersection_size=800 --max_associated_value=100

Client failed, saying:
Client: Loading data...
Client: Generating keys...
Client: Starting the protocol.
Client: Waiting for response and encrypted set from the server...
Client: Received encrypted set from the server, double encrypting...
Client: Sending double encrypted server data and single-encrypted client data to the server.
Client: Waiting for encrypted intersection sum...
Client::ExecuteProtocol: failed to ReEncryptSet: 13: GrpcClientMessageSink: Failed to send message, error code: 8, error_message: Received message larger than max (6540031 vs. 4194304)

[karnseth]

Hi pjweinb,

Our demo tool has a limit on the input sizes, because GRPC only supports messages of upto 4 MB. If you need larger sizes,one solution modifying the code to use files tp transfer data instead of using GRPC, or to use streaming (i.e. split up the message into smaller messages).

We are thinking about modifying the code to support larger files, but haven't gotten around to it yet.

[chorin1]

@karnseth @pjweinb
gRPC supports larger messages (just need to change the default settings).
If still needed, the above PR fixes this issue.

[karnseth]

Thanks @chorin1 . We're going to keep the code in the repo with the 4MB limit for now, but users are free to use a custom RPC channel and apply the modification you show in the pull request.

In addition, we've recently added some libraries in util/ that make it easy to serialize/deserialize protocol buffers to/from files, which may be used to enable larger exchanges.