Conversation
commit d142df1 upstream. New CPU #defines encode vendor and family as well as model. Signed-off-by: Tony Luck <tony.luck@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Link: https://lore.kernel.org/all/20240520224620.9480-32-tony.luck%40intel.com Conflicts: arch/x86/events/intel/core.c [jz: resolve context conflicts] Signed-off-by: Jason Zeng <jason.zeng@intel.com>
commit 4e955c08d6dc76fb60cda9af955ddcebedaa7f69 upstream. Diamond Rapids (DMR) and Nova Lake (NVL) introduce an enhanced Off-Module Response (OMR) facility, replacing the Off-Core Response (OCR) Performance Monitoring of previous processors. Legacy microarchitectures used the OCR facility to evaluate off-core and multi-core off-module transactions. The newly named OMR facility improves OCR capabilities for scalable coverage of new memory systems in multi-core module systems. Similar to OCR, 4 additional off-module configuration MSRs (OFFMODULE_RSP_0 to OFFMODULE_RSP_3) are introduced to specify attributes of off-module transactions. When multiple identical OMR events are created, they need to occupy the same OFFMODULE_RSP_x MSR. To ensure these multiple identical OMR events can work simultaneously, the intel_alt_er() and intel_fixup_er() helpers are enhanced to rotate these OMR events across different OFFMODULE_RSP_* MSRs, similar to previous OCR events. For more details about OMR, please refer to section 16.1 "OFF-MODULE RESPONSE (OMR) FACILITY" in ISE documentation. Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://patch.msgid.link/20260114011750.350569-2-dapeng1.mi@linux.intel.com Signed-off-by: Jason Zeng <jason.zeng@intel.com>
commit d2bdcde9626cbea0c44a6aaa33b440c8adf81e09 upstream. With the introduction of the OMR feature, the PEBS memory auxiliary info field for load and store latency events has been restructured for DMR. The memory auxiliary info field's bit[8] indicates whether a L2 cache miss occurred for a memory load or store instruction. If bit[8] is 0, it signifies no L2 cache miss, and bits[7:0] specify the exact cache data source (up to the L2 cache level). If bit[8] is 1, bits[7:0] represent the OMR encoding, indicating the specific L3 cache or memory region involved in the memory access. A significant enhancement is OMR encoding provides up to 8 fine-grained memory regions besides the cache region. A significant enhancement for OMR encoding is the ability to provide up to 8 fine-grained memory regions in addition to the cache region, offering more detailed insights into memory access regions. For detailed information on the memory auxiliary info encoding, please refer to section 16.2 "PEBS LOAD LATENCY AND STORE LATENCY FACILITY" in the ISE documentation. This patch ensures that the PEBS memory auxiliary info field is correctly interpreted and utilized in DMR. Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://patch.msgid.link/20260114011750.350569-3-dapeng1.mi@linux.intel.com Conflicts: arch/x86/events/intel/ds.c arch/x86/events/perf_event.h [jz: resolve context Conflicts] Signed-off-by: Jason Zeng <jason.zeng@intel.com>
commit d345b6bb886004ac1018da0348b5da7d9906071b upstream. This patch enables core PMU features for Diamond Rapids (Panther Cove microarchitecture), including Panther Cove specific counter and PEBS constraints, a new cache events ID table, and the model-specific OMR events extra registers table. For detailed information about counter constraints, please refer to section 16.3 "COUNTER RESTRICTIONS" in the ISE documentation. Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://patch.msgid.link/20260114011750.350569-4-dapeng1.mi@linux.intel.com Signed-off-by: Jason Zeng <jason.zeng@intel.com>
commit 8c74e4e3e0596950554962229582260f1501d899 upstream. Replace magic numbers in attr_rdpmc with macros to improve readability and make their meanings clearer for users. Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://patch.msgid.link/20260114011750.350569-7-dapeng1.mi@linux.intel.com Signed-off-by: Jason Zeng <jason.zeng@intel.com>
commit 59af95e028d4114991b9bd96a39ad855b399cc07 upstream. Starting with Panther Cove, the rdpmc user disable feature is supported. This feature allows the perf system to disable user space rdpmc reads at the counter level. Currently, when a global counter is active, any user with rdpmc rights can read it, even if perf access permissions forbid it (e.g., disallow reading ring 0 counters). The rdpmc user disable feature mitigates this security concern. Details: - A new RDPMC_USR_DISABLE bit (bit 37) in each EVNTSELx MSR indicates that the GP counter cannot be read by RDPMC in ring 3. - New RDPMC_USR_DISABLE bits in IA32_FIXED_CTR_CTRL MSR (bits 33, 37, 41, 45, etc.) for fixed counters 0, 1, 2, 3, etc. - When calling rdpmc instruction for counter x, the following pseudo code demonstrates how the counter value is obtained: If (!CPL0 && RDPMC_USR_DISABLE[x] == 1) ? 0 : counter_value; - RDPMC_USR_DISABLE is enumerated by CPUID.0x23.0.EBX[2]. This patch extends the current global user space rdpmc control logic via the sysfs interface (/sys/devices/cpu/rdpmc) as follows: - rdpmc = 0: Global user space rdpmc and counter-level user space rdpmc for all counters are both disabled. - rdpmc = 1: Global user space rdpmc is enabled during the mmap-enabled time window, and counter-level user space rdpmc is enabled only for non-system-wide events. This prevents counter data leaks as count data is cleared during context switches. - rdpmc = 2: Global user space rdpmc and counter-level user space rdpmc for all counters are enabled unconditionally. The new rdpmc settings only affect newly activated perf events; currently active perf events remain unaffected. This simplifies and cleans up the code. The default value of rdpmc remains unchanged at 1. For more details about rdpmc user disable, please refer to chapter 15 "RDPMC USER DISABLE" in ISE documentation. Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://patch.msgid.link/20260114011750.350569-8-dapeng1.mi@linux.intel.com Signed-off-by: Jason Zeng <jason.zeng@intel.com>
commit 1d07bbd7ea36ea0b8dfa8068dbe67eb3a32d9590 upstream.
When running the command:
'perf record -e "{instructions,instructions:p}" -j any,counter sleep 1',
a "shift-out-of-bounds" warning is reported on CWF.
UBSAN: shift-out-of-bounds in /kbuild/src/consumer/arch/x86/events/intel/lbr.c:970:15
shift exponent 64 is too large for 64-bit type 'long long unsigned int'
......
intel_pmu_lbr_counters_reorder.isra.0.cold+0x2a/0xa7
intel_pmu_lbr_save_brstack+0xc0/0x4c0
setup_arch_pebs_sample_data+0x114b/0x2400
The warning occurs because the second "instructions:p" event, which
involves branch counters sampling, is incorrectly programmed to fixed
counter 0 instead of the general-purpose (GP) counters 0-3 that support
branch counters sampling. Currently only GP counters 0-3 support branch
counters sampling on CWF, any event involving branch counters sampling
should be programed on GP counters 0-3. Since the counter index of fixed
counter 0 is 32, it leads to the "src" value in below code is right
shifted 64 bits and trigger the "shift-out-of-bounds" warning.
cnt = (src >> (order[j] * LBR_INFO_BR_CNTR_BITS)) & LBR_INFO_BR_CNTR_MASK;
The root cause is the loss of the branch counters constraint for the
new event in the branch counters sampling event group. Since it isn't
yet part of the sibling list. This results in the second
"instructions:p" event being programmed on fixed counter 0 incorrectly
instead of the appropriate GP counters 0-3.
To address this, we apply the missing branch counters constraint for
the last event in the group. Additionally, we introduce a new function,
`intel_set_branch_counter_constr()`, to apply the branch counters
constraint and avoid code duplication.
Fixes: 3374491 ("perf/x86/intel: Support branch counters logging")
Reported-by: Xudong Hao <xudong.hao@intel.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260228053320.140406-2-dapeng1.mi@linux.intel.com
Cc: stable@vger.kernel.org
Signed-off-by: Jason Zeng <jason.zeng@intel.com>
commit e7fcc54524f04e42641de99028edd9c69dc19f8c upstream. When omr_source is 0x2, the omr_snoop (bit[6]) and omr_promoted (bit[7]) fields are combined to represent the snoop information. However, the omr_promoted field was not left-shifted by 1 bit, resulting in incorrect snoop information. Besides, the snoop information parsing is not accurate for some OMR sources, like the snoop information should be SNOOP_NONE for these memory access (omr_source >= 7) instead of SNOOP_HIT. Fix these issues. Closes: https://lore.kernel.org/all/CAP-5=fW4zLWFw1v38zCzB9-cseNSTTCtup=p2SDxZq7dPayVww@mail.gmail.com/ Fixes: d2bdcde9626c ("perf/x86/intel: Add support for PEBS memory auxiliary info field in DMR") Reported-by: Ian Rogers <irogers@google.com> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Reviewed-by: Ian Rogers <irogers@google.com> Link: https://patch.msgid.link/20260311075201.2951073-1-dapeng1.mi@linux.intel.com Signed-off-by: Jason Zeng <jason.zeng@intel.com>
commit dbde07f06226438cd2cf1179745fa1bec5d8914a upstream.
Auto counter reload may have a group of events with software events
present within it. The software event PMU isn't the x86_hybrid_pmu and
a container_of operation in intel_pmu_set_acr_caused_constr (via the
hybrid helper) could cause out of bound memory reads. Avoid this by
guarding the call to intel_pmu_set_acr_caused_constr with an
is_x86_event check.
Fixes: ec980e4facef ("perf/x86/intel: Support auto counter reload")
Signed-off-by: Ian Rogers <irogers@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Thomas Falcon <thomas.falcon@intel.com>
Link: https://patch.msgid.link/20260312194305.1834035-1-irogers@google.com
Signed-off-by: Jason Zeng <jason.zeng@intel.com>
commit b191aa32be2c960ca2391b40e862c389230cadef upstream. It's good enough to only check GP counters for PEBS constraints validation since constraints overlap can only happen on GP counters. Besides opportunistically refine the code style and use pr_warn() to replace pr_info() as the message itself is a warning message. Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://patch.msgid.link/20260228053320.140406-1-dapeng1.mi@linux.intel.com Signed-off-by: Jason Zeng <jason.zeng@intel.com>
commit 5ad732a56be46aabf158c16aa0c095291727aaef upstream.
Currently there are several issues on the user space ACR mask validation
and configuration.
- The validation for user space ACR mask (attr.config2) is incomplete,
e.g., the ACR mask could include the index which belongs to another
ACR events group, but it's not validated.
- An early return on an invalid ACR mask caused all subsequent ACR groups
to be skipped.
- The stale hardware ACR mask (hw.config1) is not cleared before setting
new hardware ACR mask.
The following changes address all of the above issues.
- Figure out the event index group of an ACR group. Any bits in the
user-space mask not present in the index group are now dropped.
- Instead of an early return on invalid bits, drop only the invalid
portions and continue iterating through all ACR events to ensure full
configuration.
- Explicitly clear the stale hardware ACR mask for each event prior to
writing the new configuration.
Besides, a non-leader event member of ACR group could be disabled in
theory. This could cause bit-shifting errors in the acr_mask of remaining
group members. But since ACR sampling requires all events to be active,
this should not be a big concern in real use case. Add a "FIXME" comment
to notice this risk.
Fixes: ec980e4facef ("perf/x86/intel: Support auto counter reload")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260430002558.712334-2-dapeng1.mi@linux.intel.com
Signed-off-by: Jason Zeng <jason.zeng@intel.com>
commit 8ba0b706a485b1e607594cf4210786d517ad1611 upstream.
Members of an ACR group are logically linked via a bitmask of their
hardware counter indices. If some members of the group are assigned new
hardware counters during rescheduling, even events that keep their
original counter index must be updated with a new mask.
Without this, an event will continue to use a stale acr_mask that
references the old indices of its group peers. Ensure all ACR events are
reprogrammed during the scheduling path to maintain consistency across
the group.
Fixes: ec980e4facef ("perf/x86/intel: Support auto counter reload")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260430002558.712334-3-dapeng1.mi@linux.intel.com
Signed-off-by: Jason Zeng <jason.zeng@intel.com>
commit 1271aeccc307066315b2d3b0d5af2510e27018b5 upstream.
On platforms with Auto Counter Reload (ACR) support, such as NVL, a
"NMI received for unknown reason 30" warning is observed when running
multiple events in a group with ACR enabled:
$ perf record -e '{instructions/period=20000,acr_mask=0x2/u,\
cycles/period=40000,acr_mask=0x3/u}' ./test
The warning occurs because the Performance Monitoring Interrupt (PMI)
is enabled for the self-reloaded event (the cycles event in this case).
According to the Intel SDM, the overflow bit
(IA32_PERF_GLOBAL_STATUS.PMCn_OVF) is never set for self-reloaded events.
Since the bit is not set, the perf NMI handler cannot identify the source
of the interrupt, leading to the "unknown reason" message.
Furthermore, enabling PMI for self-reloaded events is unnecessary and
can lead to extraneous records that pollute the user's requested data.
Disable the interrupt bit for all events configured with ACR self-reload.
Fixes: ec980e4facef ("perf/x86/intel: Support auto counter reload")
Reported-by: Andi Kleen <ak@linux.intel.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260430002558.712334-4-dapeng1.mi@linux.intel.com
Signed-off-by: Jason Zeng <jason.zeng@intel.com>
commit aa4384bc8f4360167f3c3d5322121fe892289ea2 upstream.
Panther cove µarch starts to support auto counter reload (ACR), but the
static_call intel_pmu_enable_acr_event() is not updated for the Panther
Cove µarch used by DMR. It leads to the auto counter reload is not
really enabled on DMR.
Update static_call intel_pmu_enable_acr_event() in intel_pmu_init_pnc().
Fixes: d345b6bb8860 ("perf/x86/intel: Add core PMU support for DMR")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260430002558.712334-5-dapeng1.mi@linux.intel.com
Signed-off-by: Jason Zeng <jason.zeng@intel.com>
commit 5c3cdc74af25fc7f64a3ed260b4f6eb8313a3b75 upstream. Both Auto Counter Reload (ACR) and Architectural PEBS use the PERF_CFG_C MSRs to configure event behavior. Currently, the driver maintains two independent variables acr_cfg_c and cfg_c_val to cache the values intended for these MSRs. Using separate variables to track a single hardware register state is error-prone and can lead to configuration conflicts. Consolidate the tracking into a single cfg_c_val variable to ensure a unified and consistent view of the PERF_CFG_C MSR state. Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://patch.msgid.link/20260430002558.712334-6-dapeng1.mi@linux.intel.com Signed-off-by: Jason Zeng <jason.zeng@intel.com>
commit ffbe4ab upstream. The current ref-cycles event is only available on the fixed counter 2. Starting from the GLC and GRT core, the architectural UnHalted Reference Cycles event (0x013c) which is available on general-purpose counters can collect the exact same events as the fixed counter 2. Update the mapping of ref-cycles to 0x013c. So the ref-cycles can be available on both fixed counter 2 and general-purpose counters. Signed-off-by: Kan Liang <kan.liang@linux.intel.com> Signed-off-by: Ingo Molnar <mingo@kernel.org> Link: https://lore.kernel.org/r/20230911144139.2354015-1-kan.liang@linux.intel.com Conflicts: arch/x86/events/intel/core.c [jz: resolve context conflict] Signed-off-by: Jason Zeng <jason.zeng@intel.com>
commit acc41cdcb091453f48381d1c0d2e76b63c9d985f upstream. Update perf hard-coded event constraints and cache_extra_regs[] for Icelake server according to the latest ICX perfmon events (v1.30). Since the value of cache extra registers differs with previous generations, introduce new snc_hw_cache_extra_regs[] to represent the value of extra registers on ICX. ICX perfmon events: https://github.com/intel/perfmon/blob/main/ICX/events/icelakex_core.json Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://patch.msgid.link/20260515061143.338553-2-dapeng1.mi@linux.intel.com Signed-off-by: Jason Zeng <jason.zeng@intel.com>
commit 30d82ddee085b4b0f9a1b5bded53acaca68d9a52 upstream. Update perf hard-coded event constraints and cache_extra_regs[] for Sapphire rapids according to the latest SPR perfmon events (v1.39). Emerald Rapids (EMR) and Granite Rapids (GNR) share exactly same event constraints and extra MSR values with SPR. No extra changes are needed for EMR and GNR. Please note the change could temporarily impact other platforms which share the hard coded data structures, but it would be fixed in subsequent patches soon. SPR perfmon events: https://github.com/intel/perfmon/blob/main/SPR/events/sapphirerapids_core.json Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://patch.msgid.link/20260515061143.338553-3-dapeng1.mi@linux.intel.com Signed-off-by: Jason Zeng <jason.zeng@intel.com>
commit 070bd45e1dba684bfa4e7fdaa8ef8efa99b4572e upstream. Add missed event constraint for 0x0200 event and add comments to show the event names in pnc_hw_cache_extra_regs[]. Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://patch.msgid.link/20260515061143.338553-4-dapeng1.mi@linux.intel.com Signed-off-by: Jason Zeng <jason.zeng@intel.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This is to support PMU core events for Intel new platform DMR.
Test
The following LKVS PMU test suite are conducted and PASS