rayextract trees - malloc() invalid size (unsorted)

[hagreven]

Hello Thomas,

rayextract trees recently crashed for me with malloc(): invalid size (unsorted).

rayextract trees c13_r06_subsampled_9cm.ply c13_r06_subsampled_9cm_mesh.ply
reading: c13_r06_subsampled_9cm.ply
warning: very large data in point 0, suspicious:      652470 5.77289e+06      242.11
read and process 19 / 20 27.984s    
row size: 28
reading from c13_r06_subsampled_9cm_mesh.ply, 1941516 triangles.
dims for low: 100 100   1, rows: 100, cols: 100
malloc(): invalid size (unsorted)
Aborted (core dumped)

I looked into it with valgrind. The issue seems to be caused by ray::getRootsAndSegment().

valgrind --tool=memcheck rayextract trees c13_r06_subsampled_9cm.ply c13_r06_subsampled_9cm_mesh.ply 
==966== Memcheck, a memory error detector
==966== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==966== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==966== Command: rayextract trees c13_r06_subsampled_9cm.ply c13_r06_subsampled_9cm_mesh.ply
==966== 
reading: c13_r06_subsampled_9cm.ply
warning: very large data in point 0, suspicious:      652470 5.77289e+06      242.11
read and process 8 / 20==966== Warning: set address range perms: large range [0x366eb040, 0x4d521040) (undefined)
==966== Warning: set address range perms: large range [0x59c87040, 0x70abd040) (undefined)
read and process 16 / 20==966== Warning: set address range perms: large range [0x70abe040, 0x9e72a040) (undefined)
==966== Warning: set address range perms: large range [0x366eb028, 0x4d521058) (noaccess)
==966== Warning: set address range perms: large range [0x9e72b040, 0xcc397040) (undefined)
==966== Warning: set address range perms: large range [0x59c87028, 0x70abd058) (noaccess)
read and process 19 / 20 76.516s    
row size: 28
reading from c13_r06_subsampled_9cm_mesh.ply, 1941516 triangles.
==966== Warning: set address range perms: large range [0xcc398040, 0x13491d110) (undefined)
==966== Warning: set address range perms: large range [0x13491e040, 0x2054281e0) (undefined)
==966== Warning: set address range perms: large range [0xcc398028, 0x13491d128) (noaccess)
dims for low: 100 100   1, rows: 100, cols: 100
==966== Invalid read of size 8
==966==    at 0x4998952: ray::getRootsAndSegment(std::vector<ray::Vertex, std::allocator<ray::Vertex> >&, ray::Cloud const&, ray::Mesh const&, double, double, double, double) (in /usr/local/lib/libraylib.so.0.0.1)
==966==    by 0x498866D: ray::Trees::Trees(ray::Cloud&, Eigen::Matrix<double, 3, 1, 0, 3, 1> const&, ray::Mesh const&, ray::TreesParams const&, bool) (in /usr/local/lib/libraylib.so.0.0.1)
==966==    by 0x127D96: rayExtract(int, char**) (rayextract.cpp:233)
==966==    by 0x12F097: int std::__invoke_impl<int, int (*&)(int, char**), int, char**>(std::__invoke_other, int (*&)(int, char**), int&&, char**&&) (invoke.h:61)
==966==    by 0x12F105: std::enable_if<std::__and_<std::__not_<std::is_void<bool> >, std::is_convertible<std::__invoke_result<int (*&)(int, char**), int, char**>::type, bool> >::value, bool>::type std::__invoke_r<bool, int (*&)(int, char**), int, char**>(int (*&)(int, char**), int&&, char**&&) (invoke.h:142)
==966==    by 0x12DC64: std::_Function_handler<bool (int, char**), int (*)(int, char**)>::_M_invoke(std::_Any_data const&, int&&, char**&&) (std_function.h:290)
==966==    by 0x12AF67: std::function<bool (int, char**)>::operator()(int, char**) const (std_function.h:590)
==966==    by 0x129C55: ray::runWithMemoryCheck(std::function<bool (int, char**)>, int, char**) (rayutils.h:38)
==966==    by 0x1297E5: main (rayextract.cpp:320)
==966==  Address 0xd036e80 is 256 bytes inside an unallocated block of size 426,592 in arena "client"
==966== 
==966== Invalid write of size 8
==966==    at 0x4998956: ray::getRootsAndSegment(std::vector<ray::Vertex, std::allocator<ray::Vertex> >&, ray::Cloud const&, ray::Mesh const&, double, double, double, double) (in /usr/local/lib/libraylib.so.0.0.1)
==966==    by 0x498866D: ray::Trees::Trees(ray::Cloud&, Eigen::Matrix<double, 3, 1, 0, 3, 1> const&, ray::Mesh const&, ray::TreesParams const&, bool) (in /usr/local/lib/libraylib.so.0.0.1)
==966==    by 0x127D96: rayExtract(int, char**) (rayextract.cpp:233)
==966==    by 0x12F097: int std::__invoke_impl<int, int (*&)(int, char**), int, char**>(std::__invoke_other, int (*&)(int, char**), int&&, char**&&) (invoke.h:61)
==966==    by 0x12F105: std::enable_if<std::__and_<std::__not_<std::is_void<bool> >, std::is_convertible<std::__invoke_result<int (*&)(int, char**), int, char**>::type, bool> >::value, bool>::type std::__invoke_r<bool, int (*&)(int, char**), int, char**>(int (*&)(int, char**), int&&, char**&&) (invoke.h:142)
==966==    by 0x12DC64: std::_Function_handler<bool (int, char**), int (*)(int, char**)>::_M_invoke(std::_Any_data const&, int&&, char**&&) (std_function.h:290)
==966==    by 0x12AF67: std::function<bool (int, char**)>::operator()(int, char**) const (std_function.h:590)
==966==    by 0x129C55: ray::runWithMemoryCheck(std::function<bool (int, char**)>, int, char**) (rayutils.h:38)
==966==    by 0x1297E5: main (rayextract.cpp:320)
==966==  Address 0xd036e80 is 256 bytes inside an unallocated block of size 426,592 in arena "client"
==966== 

valgrind: m_mallocfree.c:303 (get_bszB_as_is): Assertion 'bszB_lo == bszB_hi' failed.
valgrind: Heap block lo/hi size mismatch: lo = 80064, hi = 4625219335307636240.
This is probably caused by your program erroneously writing past the
end of a heap block and corrupting heap metadata.  If you fix any
invalid writes reported by Memcheck, this assertion failure will
probably go away.  Please try that before reporting this as a bug.


host stacktrace:
==966==    at 0x5804284A: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
==966==    by 0x58042977: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
==966==    by 0x58042B1B: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
==966==    by 0x5804C8CF: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
==966==    by 0x5803AE9A: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
==966==    by 0x580395B7: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
==966==    by 0x5803DF3D: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
==966==    by 0x58038868: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
==966==    by 0x5800F344: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
==966==    by 0x10032042D3: ???
==966==    by 0x1002DB1F2F: ???
==966==    by 0x581FCD83: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
==966==    by 0x1002DB1F17: ???
==966==    by 0x1002DB1F2F: ???

sched status:
  running_tid=1

Thread 1: status = VgTs_Runnable (lwpid 966)
==966==    at 0x4998461: ray::getRootsAndSegment(std::vector<ray::Vertex, std::allocator<ray::Vertex> >&, ray::Cloud const&, ray::Mesh const&, double, double, double, double) (in /usr/local/lib/libraylib.so.0.0.1)
==966==    by 0x498866D: ray::Trees::Trees(ray::Cloud&, Eigen::Matrix<double, 3, 1, 0, 3, 1> const&, ray::Mesh const&, ray::TreesParams const&, bool) (in /usr/local/lib/libraylib.so.0.0.1)
==966==    by 0x127D96: rayExtract(int, char**) (rayextract.cpp:233)
==966==    by 0x12F097: int std::__invoke_impl<int, int (*&)(int, char**), int, char**>(std::__invoke_other, int (*&)(int, char**), int&&, char**&&) (invoke.h:61)
==966==    by 0x12F105: std::enable_if<std::__and_<std::__not_<std::is_void<bool> >, std::is_convertible<std::__invoke_result<int (*&)(int, char**), int, char**>::type, bool> >::value, bool>::type std::__invoke_r<bool, int (*&)(int, char**), int, char**>(int (*&)(int, char**), int&&, char**&&) (invoke.h:142)
==966==    by 0x12DC64: std::_Function_handler<bool (int, char**), int (*)(int, char**)>::_M_invoke(std::_Any_data const&, int&&, char**&&) (std_function.h:290)
==966==    by 0x12AF67: std::function<bool (int, char**)>::operator()(int, char**) const (std_function.h:590)
==966==    by 0x129C55: ray::runWithMemoryCheck(std::function<bool (int, char**)>, int, char**) (rayutils.h:38)
==966==    by 0x1297E5: main (rayextract.cpp:320)
client stack range: [0x1FFEFFD000 0x1FFF000FFF] client SP: 0x1FFEFFE740
valgrind stack range: [0x1002CB2000 0x1002DB1FFF] top usage: 18984 of 1048576

This is what the input looks like

input raycloud 0.91 GB

Do you have an idea why this happens?

I have been working with rayextract trees before but never saw this error.

Kind regards,
Hannah

[thomas-lowe]

Hi Hannah,
I've not seen it before but there are two things that might cause it.

1. the very point that createst box_max may produce an index that is 1 outside the ArrayXXd ranges. Simplest test is just after calcBounds in getRootsAndSegment (raylib/extraction/raysegment.cpp) add: box_max += 0.0001*Eigen::Vector3d(1,1,1); and see if it fixes it.
2. it may have bad input, i.e. if the points vector passed into getRootsAndSegment is empty, so you could add a warning/assert/breakpoint on this bad input.

I can't debug it without accessing the input raycloud, but most likely it needs some guardrails around those index array lookups or it is bad input suggesting the problem happens earlier.

[hagreven]

Thank you for the prompt reply! If it would be helpful, I’d be happy to share the input data with you.

[hagreven]

Your first suggestion fixes it! With the added line box_max += 0.0001*Eigen::Vector3d(1,1,1); rayextract trees finishes successfully.