When I run the run_grpo.sh on my own dataset, I get the error pyarrow.lib.ArrowInvalid: offset overflow while concatenating arrays. It seems that the size of the picture is too large. If I compress my pictures, I may lose the features. How to fix the error?
[rank7]: Traceback (most recent call last):
[rank7]: File "/data/lilumin/anaconda3/envs/r1-v/lib/python3.11/site-packages/datasets/arrow_dataset.py", line 3547, in _map_singl
e
[rank7]: writer.finalize() # close_stream=bool(buf_writer is None)) # We only close if we are writing in a file
[rank7]: ^^^^^^^^^^^^^^^^^
[rank7]: File "/data/lilumin/anaconda3/envs/r1-v/lib/python3.11/site-packages/datasets/arrow_writer.py", line 657, in finalize
[rank7]: self.write_examples_on_file()
[rank7]: File "/data/lilumin/anaconda3/envs/r1-v/lib/python3.11/site-packages/datasets/arrow_writer.py", line 504, in write_exampl
es_on_file
[rank7]: batch_examples[col] = pa.concat_arrays(arrays)
[rank7]: ^^^^^^^^^^^^^^^^^^^^^^^^
[rank7]: File "pyarrow/array.pxi", line 4954, in pyarrow.lib.concat_arrays
[rank7]: File "pyarrow/error.pxi", line 155, in pyarrow.lib.pyarrow_internal_check_status
[rank7]: File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status
[rank7]: pyarrow.lib.ArrowInvalid: offset overflow while concatenating arrays
[rank7]: During handling of the above exception, another exception occurred:
[rank7]: Traceback (most recent call last):
[rank7]: File "/data/lilumin/R1-V/src/r1-v/src/open_r1/grpo.py", line 208, in <module>
[rank7]: main(script_args, training_args, model_args)
[rank7]: File "/data/lilumin/R1-V/src/r1-v/src/open_r1/grpo.py", line 166, in main
[rank7]: dataset = dataset.map(make_conversation_image) # Utilize multiprocessing for faster mapping
[rank7]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank7]: File "/data/lilumin/anaconda3/envs/r1-v/lib/python3.11/site-packages/datasets/dataset_dict.py", line 941, in map
[rank7]: dataset_dict[split] = dataset.map(
[rank7]: ^^^^^^^^^^^^
[rank7]: File "/data/lilumin/anaconda3/envs/r1-v/lib/python3.11/site-packages/datasets/arrow_dataset.py", line 557, in wrapper
[rank7]: out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
[rank7]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank7]: File "/data/lilumin/anaconda3/envs/r1-v/lib/python3.11/site-packages/datasets/arrow_dataset.py", line 3079, in map
[rank7]: for rank, done, content in Dataset._map_single(**dataset_kwargs):
[rank7]: File "/data/lilumin/anaconda3/envs/r1-v/lib/python3.11/site-packages/datasets/arrow_dataset.py", line 3552, in _map_singl
e
[rank7]: writer.finalize()
[rank7]: File "/data/lilumin/anaconda3/envs/r1-v/lib/python3.11/site-packages/datasets/arrow_writer.py", line 657, in finalize
[rank7]: self.write_examples_on_file()
[rank7]: File "/data/lilumin/anaconda3/envs/r1-v/lib/python3.11/site-packages/datasets/arrow_writer.py", line 504, in write_exampl
es_on_file
[rank7]: batch_examples[col] = pa.concat_arrays(arrays)
[rank7]: ^^^^^^^^^^^^^^^^^^^^^^^^
[rank7]: File "pyarrow/array.pxi", line 4954, in pyarrow.lib.concat_arrays
[rank7]: File "pyarrow/error.pxi", line 155, in pyarrow.lib.pyarrow_internal_check_status
[rank7]: File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status
[rank7]: pyarrow.lib.ArrowInvalid: offset overflow while concatenating arrays
When I run the run_grpo.sh on my own dataset, I get the error pyarrow.lib.ArrowInvalid: offset overflow while concatenating arrays. It seems that the size of the picture is too large. If I compress my pictures, I may lose the features. How to fix the error?