[QNN EP] Added sustained high performance mode implementation

[qti-monumeen]

Description

Added sustained high performance mode

Motivation and Context

[qti-jkilpatrick]

It looks like this needs to be rebased on main and CI isn't clean yet. Please consider converting this to a draft until it's ready so reviewers can see they should wait. Thanks!

[huaychou]

Please also fix this

[huaychou]

[N-2] (Minor) try/catch in BkgTimer() is dead code: cv_.notify_all() is noexcept

In the initialization block of BkgTimer() (timer.cc lines 25–32), cv_.notify_all() is wrapped in a try { ... } catch (const std::system_error& e). The C++11 standard specifies that std::condition_variable::notify_all() is noexcept and can never throw std::system_error. The catch block is therefore dead code, and the thread_status_ = FAILED path can never execute.

Impact: The FAILED error-handling path in Initialize() can never be triggered, making the error handling misleading.

Suggested fix:

Remove the try/catch and write directly: thread_status_ = threadState::IDLE; cv_.notify_all();. To catch thread-startup errors, handle std::system_error around the std::thread constructor call instead.

[huaychou]

[N-3] (Minor) SetPostRunHtpPerf() sets finalized_=true before calling SetState — destructor fallback skipped on failure

In HtpPowerStateGuard::SetPostRunHtpPerf() (lines 65–70), finalized_ = true is set before calling power_manager_->SetState(done_state_, ...).

If SetState(done_state_) fails and returns an error, finalized_ is already true, so the destructor's if (pre_run_called_ && !finalized_) condition is false and the destructor will not retry the done-state transition. The HTP power config is left in start_state (high-performance) until the next call.

Impact: In the edge case where SetPostRunHtpPerf() fails (power-config API error), HTP stays in high-performance mode instead of being relaxed.

Suggested fix:

Move finalized_ = true to after the SetState(done_state_) call, setting it only on success. Alternatively, keep the current behavior (always set finalized_) but log a WARNING on failure to indicate that the HTP power state may not have been correctly relaxed.

[huaychou]

Just wondering, what is the benefit of this PR? Should it reduce the init/deinit time of each execution?

[qti-monumeen]

Refer to this page for reason to add this change:
https://qualcomm-confluence.atlassian.net/wiki/spaces/ML/pages/434806935/Sustained+High+Performance+enablement+on+ORT