cpu-stats

cpu-stats is a simple daemon monitoring the cpus of a linux system

Description

cpu-stats consists of a daemon monitoring the power consumption of the cpus and of amd gpus and the frequencies of the cores and a client program accessing these collected data.

The daemon requires access to the rapl information of the system to monitor the power consumption of the packages. The minimum power sampling rate is 1 time per second (1Hz) in order to avoid leaking information out of the kernel,

It is possible to pass through the collected data into lxc containers.

Getting Started

Dependencies

• the program works only under linux
• it has never been compiled with a compiler other than gcc

Build and test

• Execute fakeroot debian/rules binary to produce a debian package
• otherwise you may edit the Makefile in the root directory to change paths and compile it

Data pass through into lxc containers

Add the following lines to the configuration file of the container:

# lxc.mount entries for cpu-stats
lxc.mount.entry = none dev/shm tmpfs nodev,nosuid,noexec,mode=1777,create=dir 0 0
lxc.mount.entry=/dev/shm/proc_stats_load dev/shm/proc_stats_load none bind,ro,optional,create=file
lxc.mount.entry=/dev/shm/cpu_stats_p_pkg_000 dev/shm/cpu_stats_p_pkg_000 none bind,ro,optional,create=file
...
lxc.mount.entry=/dev/shm/cpu_stats_p_amdgpu_000 dev/shm/cpu_stats_p_amdgpu_000 none bind,ro,optional,create=file
...
lxc.mount.entry=/dev/shm/cpu_stats_f_cpu_000 dev/shm/cpu_stats_f_cpu_000 none bind,ro,optional,create=file
lxc.mount.entry=/dev/shm/cpu_stats_f_cpu_001 dev/shm/cpu_stats_f_cpu_001 none bind,ro,optional,create=file
lxc.mount.entry=/dev/shm/cpu_stats_f_cpu_002 dev/shm/cpu_stats_f_cpu_002 none bind,ro,optional,create=file
lxc.mount.entry=/dev/shm/cpu_stats_f_cpu_003 dev/shm/cpu_stats_f_cpu_003 none bind,ro,optional,create=file
lxc.mount.entry=/dev/shm/cpu_stats_f_cpu_004 dev/shm/cpu_stats_f_cpu_004 none bind,ro,optional,create=file
lxc.mount.entry=/dev/shm/cpu_stats_f_cpu_005 dev/shm/cpu_stats_f_cpu_005 none bind,ro,optional,create=file
lxc.mount.entry=/dev/shm/cpu_stats_f_cpu_006 dev/shm/cpu_stats_f_cpu_006 none bind,ro,optional,create=file
lxc.mount.entry=/dev/shm/cpu_stats_f_cpu_007 dev/shm/cpu_stats_f_cpu_007 none bind,ro,optional,create=file
lxc.mount.entry=/dev/shm/cpu_stats_f_cpu_008 dev/shm/cpu_stats_f_cpu_008 none bind,ro,optional,create=file

The number of /dev/shm/cpu_stats_f_cpu_XXX lines should match the number of visible cores in the container.

Install the cpu-stats package into/in the container.

the cpu-stats-daemon

The cpu-stats-daemon collects data from different sources

• load from /proc/stat
• power data from rapl packages
• power data from amdgpu-hwmon
• cpu frequencies from sysfs and stores the data into histograms in shared memory.

The sampling frequency should be low enough to not leak data from kernel to user space.

the cpu-stats program

The cpu-stats program reads the shared memory created by the daemon and prints out the informations:

========================================================================
load samples=3.2798300000000000000000e+05
ld/%       %   sum %  | ld/%       %   sum %  | ld/%       %   sum % 
 100    2.08  100.00  |   66    0.29   79.65  |   32    0.06   79.07
  99    0.41   97.92  |   65    0.13   79.36  |   31    0.03   79.01
  98    0.78   97.51  |   64    0.05   79.23  |   30    0.02   78.98
  97    0.94   96.73  |   63    0.02   79.18  |   29    0.01   78.96
  96    1.05   95.79  |   62    0.01   79.16  |   28    0.01   78.95
  95    1.09   94.74  |   61    0.00   79.15  |   27    0.00   78.94
  94    1.03   93.65  |   60    0.00   79.15  |   26    0.09   78.94
  93    0.98   92.62  |   59    0.00   79.15  |   25    0.00   78.85
  92    0.86   91.64  |   58    0.00   79.15  |   24    0.00   78.85
  91    0.85   90.78  |   57    0.00   79.15  |   23    0.00   78.85
  90    0.75   89.93  |   56    0.00   79.15  |   22    0.00   78.85
  89    0.69   89.18  |   55    0.00   79.15  |   21    0.00   78.85
  88    0.70   88.49  |   54    0.00   79.15  |   20    0.00   78.85
  87    0.74   87.79  |   53    0.00   79.15  |   19    0.00   78.85
  86    0.81   87.05  |   52    0.00   79.15  |   18    0.00   78.85
  85    0.81   86.24  |   51    0.02   79.15  |   17    0.00   78.85
  84    0.71   85.43  |   50    0.00   79.13  |   16    0.00   78.85
  83    0.54   84.72  |   49    0.00   79.13  |   15    0.00   78.85
  82    0.40   84.18  |   48    0.00   79.13  |   14    0.00   78.85
  81    0.30   83.78  |   47    0.01   79.13  |   13    0.01   78.85
  80    0.25   83.48  |   46    0.00   79.12  |   12    0.06   78.84
  79    0.23   83.23  |   45    0.00   79.12  |   11    0.09   78.78
  78    0.19   83.00  |   44    0.00   79.12  |   10    0.09   78.69
  77    0.22   82.81  |   43    0.00   79.12  |    9    0.07   78.60
  76    0.20   82.59  |   42    0.00   79.12  |    8    0.05   78.53
  75    0.18   82.39  |   41    0.00   79.12  |    7    0.04   78.48
  74    0.18   82.21  |   40    0.00   79.12  |    6    0.14   78.44
  73    0.16   82.03  |   39    0.00   79.12  |    5    0.39   78.30
  72    0.21   81.87  |   38    0.00   79.12  |    4    0.73   77.91
  71    0.27   81.66  |   37    0.00   79.12  |    3    1.72   77.18
  70    0.38   81.39  |   36    0.00   79.12  |    2    3.08   75.46
  69    0.47   81.01  |   35    0.00   79.12  |    1   72.38   72.38
  68    0.48   80.54  |   34    0.01   79.12
  67    0.41   80.06  |   33    0.04   79.11
average load: ~19%, last measured load 2.1%
load since boot: 21.1%, since start: 18.6%
========================================================================
rapl package 0, samples=3.2798300000000000000000e+05
Pwr/W       %   sum %  | Pwr/W       %   sum %  | Pwr/W       %   sum % 
 12.5    0.01  100.00  |   7.5   10.47   99.98  |   2.5   67.61   67.61
 10.0    0.01   99.99  |   5.0   21.90   89.51
average power: ~2.32 W, power over last interval: 0.76 W
used energy:   ~1.608893000000000e+06 Ws, ~4.470000000000000e-01 kWh
========================================================================
cpu 0, f_min=700, f_max=3400, samples=3.2798300000000000000000e+05
f/MHz       %   sum %  | f/MHz       %   sum %  | f/MHz       %   sum % 
 3400    0.01  100.00  |  2400    0.47   99.75  |  1400    2.14   94.15
 3200    0.01   99.99  |  2200    0.69   99.28  |  1200    2.99   92.01
 3000    0.05   99.98  |  2000    1.60   98.59  |  1000    3.98   89.02
 2800    0.06   99.93  |  1800    1.23   96.99  |   800   80.64   85.04
 2600    0.12   99.87  |  1600    1.61   95.76  |   600    4.40    4.40
average frequency: ~890 MHz, last measured frequency: ~700 MHz
========================================================================
cpu 1, f_min=700, f_max=3400, samples=3.2798300000000000000000e+05
f/MHz       %   sum %  | f/MHz       %   sum %  | f/MHz       %   sum % 
 3400    0.02  100.00  |  2400    0.53   99.64  |  1400    2.23   93.16
 3200    0.02   99.98  |  2200    0.75   99.11  |  1200    3.04   90.93
 3000    0.04   99.96  |  2000    1.72   98.36  |  1000    4.66   87.89
 2800    0.11   99.92  |  1800    1.48   96.64  |   800   75.93   83.23
 2600    0.17   99.81  |  1600    2.00   95.16  |   600    7.30    7.30
average frequency: ~898 MHz, last measured frequency: ~700 MHz
========================================================================
cpu 2, f_min=700, f_max=3400, samples=3.2798300000000000000000e+05
f/MHz       %   sum %  | f/MHz       %   sum %  | f/MHz       %   sum % 
 3400    0.03  100.00  |  2400    0.58   99.62  |  1400    2.28   93.26
 3200    0.02   99.97  |  2200    0.75   99.04  |  1200    3.15   90.98
 3000    0.05   99.95  |  2000    1.73   98.29  |  1000    4.70   87.83
 2800    0.11   99.90  |  1800    1.46   96.56  |   800   67.22   83.13
 2600    0.17   99.79  |  1600    1.84   95.10  |   600   15.91   15.91
average frequency: ~881 MHz, last measured frequency: ~700 MHz
========================================================================
cpu 3, f_min=700, f_max=3400, samples=3.2798300000000000000000e+05
f/MHz       %   sum %  | f/MHz       %   sum %  | f/MHz       %   sum % 
 3400    0.03  100.00  |  2400    0.54   99.65  |  1400    2.25   93.41
 3200    0.01   99.97  |  2200    0.76   99.11  |  1200    3.04   91.16
 3000    0.04   99.96  |  2000    1.72   98.35  |  1000    4.20   88.12
 2800    0.11   99.92  |  1800    1.44   96.63  |   800   78.55   83.92
 2600    0.16   99.81  |  1600    1.78   95.19  |   600    5.37    5.37
average frequency: ~899 MHz, last measured frequency: ~700 MHz

the kabini-power-limit-daemon

The kabini-power-limit daemon tries to implement something similiar to a turbo mode for the poor on an athlon 5350 kabini processors using estimation of read from amdctl for power consumption in the processor. It works by monitoring the load of the system via /proc/stat and writing /sys/devices/system/cpu/cpuX/cpufreq/scaling_max_freq to a load dependend value.

The table used for the athlon 5350 is shown below:

    // the values below are taken from amdctl -m -g
    { 0, "2050000", { 4.00f, 8.00f, 12.00f, 16.00f }},
    { 1, "1850000", { 3.60f, 7.20f, 10.80f, 14.40f }},
    { 2, "1650000", { 2.70f, 5.40f,  8.10f, 10.80f }},
    { 3, "1400000", { 2.00f, 4.00f,  6.00f,  8.00f }},
    { 4, "1200000", { 1.34f, 2.68f,  4.02f,  5.36f }},
    { 5, "1000000", { 1.02f, 2.04f,  3.06f,  4.08f }},
    { 6,  "800000", { 0.75f, 1.50f,  2.25f,  3.00f }}

Measured entries for the P0-State using the difference between idle and full load on 1, 2, 3 or 4 cores respectively are:

    { 0, "2050000", { 5.7f, 8.3f, 10.7f, 13.5f }}

The selection of the table above yields with a power limit of 6 Watts to the following limits for the ondemand governor:

kabini-power-limit-daemon[953]: using a limit of 2050000kHz for 0 < load <= 0.25
kabini-power-limit-daemon[953]: using a limit of 1650000kHz for 0.25 < load <= 0.5
kabini-power-limit-daemon[953]: using a limit of 1400000kHz for 0.5 < load <= 0.75
kabini-power-limit-daemon[953]: using a limit of 1200000kHz for 0.75 < load <= 1

Sending the signal USR1 to the running daemon triggers a write of the current state into system log:

kabini-power-limit-daemon[953]: 12440 loads checked, 199 frequency changes performed

License

This project is licensed under the GPL v2.0 License.