Skip to content

[FEA] Create script(s) to validate on-prem environment for readiness for Spark RAPIDS #239

@mattahrens

Description

@mattahrens

Is your feature request related to a problem? Please describe.
I would like to have a tool that would validate a Spark environment for its readiness for acceleration with GPU for the Spark RAPIDS plugin.

Describe the solution you'd like
The solution should include the following:

  • Environment analyzer to collect information about the nodes and setup -- including number of nodes, number of cores per node, amount of memory on each node, amount of disk storage on each node, disk bandwidth, network bandwidth, and GPU type and memory (if applicable)
  • Wrapper script for the NDS power run to generate data, run the benchmark and generate an event log and driver log
  • Report results from environment analyzer and NDS power run along with event log and driver log
  • The solution can allow for a configurable data size to be used along with data type (e.g. SF3K, parquet).

Describe alternatives you've considered
N/A

Additional context
N/A

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions