Skip to content

Conversation

@nathanb2111
Copy link

UCP Worker Thread Modes in OpenUCX define how multiple threads interact with a UCP worker. UCS_THREAD_MODE_SINGLE indicates that only one unique thread can use this worker throughout the whole application.
In a multithreaded application, OMPI may reuse winfo objects among different threads, conflicting with hard-coded UCS_THREAD_MODE_SINGLE, and resulting in an assert failure

If an application use multiple threads, we want to use parameter UCS_THREAD_MODE_SERIALIZED that allows different threads to use the same worker successively

Signed-off-by: Nathan Bellalou <nbellalou@nvidia.com>
@nbellalou
Copy link

@gleon99

memset(&worker_params, 0, sizeof(worker_params));
worker_params.field_mask = UCP_WORKER_PARAM_FIELD_THREAD_MODE;
worker_params.thread_mode = UCS_THREAD_MODE_SINGLE;
worker_params.thread_mode = ompi_mpi_thread_provided == MPI_THREAD_SINGLE ? UCS_THREAD_MODE_SINGLE : UCS_THREAD_MODE_SERIALIZED;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the user initialized MPI with MPI_THREAD_MULTIPLE, will this be passed correctly to the initialization of the UCX worker when creating a window? I am not sure this fix here correctly captures this (or fixes it), see #13347

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants