Skip to content

Commit 46a558f

Browse files
authored
Checker: System Resource Monitor (#400)
- New ZEL_ENABLE_SYSTEM_RESOURCE_TRACKER_CHECKER which enables for users to track the system resource usage with relation to Level Zero apis. - Can log to a level zero logger file or to a .csv for plotting using the included plot_resource_tracker.py - Currently limited to Linux support. Signed-off-by: Neil R. Spruit <neil.r.spruit@intel.com>
1 parent fb5118b commit 46a558f

8 files changed

Lines changed: 2752 additions & 0 deletions

File tree

scripts/plot_resource_tracker.py

Lines changed: 362 additions & 0 deletions
Large diffs are not rendered by default.

source/layers/validation/README.md

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ By default, no validation modes will be enabled. The individual validation modes
2222
- `ZEL_ENABLE_BASIC_LEAK_CHECKER`
2323
- `ZE_ENABLE_THREADING_VALIDATION` (Not yet Implemented)
2424
- `ZEL_ENABLE_CERTIFICATION_CHECKER`
25+
- `ZEL_ENABLE_SYSTEM_RESOURCE_TRACKER_CHECKER`
2526

2627
## Validation Modes
2728

@@ -89,6 +90,40 @@ Validates:
8990
When this mode is enabled, the certification checker validates API usage against the version supported by the driver or an explicitly specified version.
9091
If an API is used that was introduced in a version higher than the supported version, the checker will return `ZE_RESULT_ERROR_UNSUPPORTED_VERSION`.
9192

93+
### `ZEL_ENABLE_SYSTEM_RESOURCE_TRACKER_CHECKER` (Linux Only)
94+
95+
The System Resource Tracker monitors both Level Zero API resources and system resources in real-time. It tracks:
96+
97+
- **L0 Resources**: Contexts, command queues, modules, kernels, event pools, command lists, events, fences, images, samplers, and memory allocations
98+
- **System Metrics**: Virtual memory (VmSize, VmRSS, VmData, VmPeak), thread count, file descriptors
99+
- **Deltas**: Resource changes for each API call
100+
- **Cumulative Totals**: Running summaries of all resource types
101+
102+
The tracker can log to the Level Zero debug log and optionally export data to CSV for graphing and analysis:
103+
104+
```bash
105+
export ZE_ENABLE_VALIDATION_LAYER=1
106+
export ZEL_ENABLE_SYSTEM_RESOURCE_TRACKER_CHECKER=1
107+
export ZEL_SYSTEM_RESOURCE_TRACKER_CSV=tracker_output.csv # Optional: enable CSV export
108+
export ZEL_ENABLE_LOADER_LOGGING=1
109+
export ZEL_LOADER_LOGGING_LEVEL=debug
110+
```
111+
112+
**CSV Output Features:**
113+
- Per-process unique filenames (PID appended automatically)
114+
- 22 columns of metrics including timestamps, system resources, L0 resource counts, and deltas
115+
- Atomic line writes for thread safety
116+
- Companion Python plotting script (`scripts/plot_resource_tracker.py`) for visualization
117+
118+
**Use Cases:**
119+
- Performance analysis and memory leak detection
120+
- Resource lifecycle tracking and optimization
121+
- Debugging and benchmarking
122+
- CI/CD integration for automated resource monitoring
123+
124+
**Platform Support:** This checker is Linux-only and uses `/proc/self/status` for system metrics. It is automatically excluded from Windows and macOS builds.
125+
126+
See [System Resource Tracker documentation](checkers/system_resource_tracker/system_resource_tracker.md) for detailed usage and CSV format.
92127

93128
## Testing
94129

source/layers/validation/checkers/CMakeLists.txt

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,8 @@ add_subdirectory(certification)
33
add_subdirectory(events_checker)
44
add_subdirectory(parameter_validation)
55
add_subdirectory(template)
6+
7+
# System resource tracker is Linux-only (uses /proc/self/status)
8+
if(UNIX AND NOT APPLE)
9+
add_subdirectory(system_resource_tracker)
10+
endif()
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# System resource tracker is Linux-only (uses /proc/self/status)
2+
if(UNIX AND NOT APPLE)
3+
target_sources(${TARGET_NAME}
4+
PRIVATE
5+
${CMAKE_CURRENT_LIST_DIR}/zel_system_resource_tracker_checker.h
6+
${CMAKE_CURRENT_LIST_DIR}/zel_system_resource_tracker_checker.cpp
7+
)
8+
endif()
468 KB
Loading

0 commit comments

Comments
 (0)