diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..80fe6f0 --- /dev/null +++ b/.gitignore @@ -0,0 +1,5 @@ +.idea +.vscode +*.swp +*.pyc +__pycache__ diff --git a/README.md b/README.md new file mode 100644 index 0000000..35ff81d --- /dev/null +++ b/README.md @@ -0,0 +1,194 @@ +# HIP Testsuite + +## Description + +HIP Testsuite is a suite to test HIP on different platforms. At high level, testsuite helps in doing below activities +- Download required projects for tests (as mentioned in cfg.py) +- Build all/specific tests for specified platform (as provided through command line) +- Run all/specific tests +- Consolidate the reports and logs +Testsuite predominantly uses python scripts that can run with python 3.x + +## Download and execute HIP-Testsuite + +### Preconditions + +- Python 3.6 or above is installed on system +- Appropriate Rocm or Nvcc drivers are installed on system +- Git (version 2.17.1 or more) should be installed +- CMake (version 3.4 or more) should be installed + +### Steps +• Clone the TestSuite repository +``` + +$ cd hip-testsuite +``` + +• One time dependency installation + +``` +$ pip3 install -r requirements.txt +``` + +• Required packages for some tests on Ubuntu +``` +$ sudo apt install autoconf autogen automake autotools-dev bzip2 g++ gcc \ +gfortran libtool libtool-bin make patch pkg-config zlib1g-dev +``` + +• Verify "cfg.py" contents to make sure that desired values are present +To provide input configurations like log location, platform, repos etc. "cfg.py" is used. Please check 6.3.1 for further details. + +• Use “run.py” to run the test +To start executing the tests "run.py" is used. User inputs like test filters need to be provided via command line input to "run.py". Platform name (e.g. nvidia/amd) can also be provided via the command line input. +Some of the command line for run.py are briefly mentioned here. For detailed description of these, please refer to 6.3.2 and 6.3.3. +-h: To show all options +``` +$ python3 run.py -h +``` +-lstq: To list all tests quickly +``` +$ python3 run.py -lstq +``` +-lst: To list all the test (Note: platform should be specified if running for other than AMD platform) +``` +$ python3 run.py -lst +``` +Run all the tests on amd platform +``` +$ python3 run.py +``` +Select the test with -t to run specific +on amd platform +``` +$ python3 run.py -t +``` +on Nvidia platform +``` +$ python3 run.py -t --platform nvidia +``` +Some examples to execute a category of tests +``` +$ python3 run.py -t conformance --platform amd +$ python3 run.py -t samples --platform amd +$ python3 run.py -t examples --platform amd +``` + +## Details for working with HIP-Testsuite + +### Testsuite configuration + +Test suite considers contents of “cfg.py” for configurable items. There are two types of information in cfg.py. +- Default value for some parameters that can be over-ridden through command line while running test suite +- Information like test repository/branch info etc. + +| Configuration | Explanation | +| --------------- | ------------- | +| log_location | We can provide the location, in python string, where log files should be dumped. | +| | If none is provided, then logs are dumped under "report" folder in the current working directory, which is "hip-testsuite/". | +| user_password | If any password needs to be set during the execution of tests, then it can be | +| | provided using this parameter. Currently set to None. | +| HIP_PLATFORM | Platform value (amd/nvidia) can also be passed to the test using this parameter. | +| repos | This is a Python dictionary structure to provide information on all the repos required for tests. | +| | - "repo_url" should contain the GIT URL of the repository to clone. | +| | - "branch" should contain the branch name | +| | - "commit_id" should contain the GIT Commit ID to test. | +| |• If no branch is to be specified, then keep branch = None. | +| |• If no commit_id is to specified, then keep commit_id = None. | + + +### Command line options for run.py + +#### Help + +To get help on "run.py" execute +``` +$ python3 run.py -h +``` +or +``` +$ python3 run.py --help +``` +#### Test selection options + +"-lstq" or "--list_tests_quick": List all tests quickly. Note that this option generates only a partial list of test cases. For tests which are time consuming to generate only category:* will be displayed for them. To display the full list use "-lst" or "--list_tests". + +"-lst" or "--list_tests": Gets the list of all test cases. + +Note: "-lst/--list_tests" will take some time to display the results as build happens in the background to fetch the list of tests for some ctest based modules like dtest. + +Note: For platforms other than "amd" be sure to provide --platform option. For example: +``` +$ python3 run.py -lst --platform nvidia +``` + +"-t": With this option we provide testcase filter. For example, by providing a filter name like "samples" (shown in section 2) we can execute all tests cases under HIP-Samples. + +Likewise, to execute all performance test cases, use "performance" as filter. + +To run all performance tests under sample, use filter "samples:performance". + +When "-t" option is not provided, all tests in testsuite are executed. + +"--platform": With this option we provide the platform type ("amd"/"nvidia") where the testcases are run. If "--platform" is not provided, "amd" is taken as default. + +### Overview of filters for run.py + +All tests in the hip-testsuite are broadly classified into the following categories - "samples", "examples" and "conformance". Under "samples" and "examples" there are further 3 sub-categories - "performance", "stress", and "mini-app". HIP directed tests fall under "conformance" category while the rest of the tests use subcategories - "performance", "stress" and "mini-app". + +Below are the list of filters that can be applied currently after "-t": + +| Filter | Test cases executed | +| ------ | ------------------- | +| No filter options provided | All tests are executed by default. | +| “samples” | Executes all test cases under HIP-Samples | +| "samples:performance" | Executes all performance test cases under HIP-Samples | +| "samples:mini-app" | Executes all mini-app test cases under HIP-Samples | +| "examples" | Executes all test cases under HIP-Examples | +| "examples:performance" | Executes all performance test cases under HIP-Examples | +| "examples:stress" | Executes all stress test cases under HIP-Examples | +| "examples:mini-app" | Executes all mini-app test cases under HIP-Examples | +| "performance" | Executes all performance test cases in testsuite | +| "mini-app" | Executes all mini-app test cases in testsuite | +| "stress" | Executes all stress test cases in testsuite | +| "conformance" | Executes all HIP dtests | +| "testname" for e.g "BitExtract" ("testname" is typically the class name) | Executes only the specific test cases with names with “testname” | + +Executing a group of tests with similar name Use wildcards like ".*testname.*". See example 2 below + +Executing Multiple tests To execute multiple tests, specify the tests. The test names must be separated with a space. Check example 3 below. + +Examples: + +1. runs test cases "bitextract" and "directed_tests.devicelib.hip_bitextract" +``` +$ python3 run.py -t BitExtract +``` +2. We can use wildcards to filter test cases to execute. + +To execute all test cases containing "memset" name +``` +$ python3 run.py -t ".*memset.*" +``` +To execute only the "memset" test cases under "directed_tests.runtimeapi.memory" +``` +$ python3 run.py -t directed_tests.runtimeapi.memory.*memset.* +``` +3. Specifying multiple tests +``` +$ python3 run.py -t samples examples --platform amd +``` +4. When no "-t" option is specified all tests,except directed tests, are executed by default. +``` +$ python3 run.py --platform nvidia +``` +Use "-lst" or "--list_tests" options to get the list of all test cases. + +### Testsuite Report + +Reports are generated under the folder mentioned in parameter "log_location" in cfg.py. The report for each run is timestamped. For example, "report/2021_07_12_23_32_04/bitextract". At the end of each run, the summary report is displayed. This summary report provides the list of test cases with result, the metric and system information. The same is available under "report/" folder as report.log. The same report also will be available in JSON format as report.json + +## Adding new tests to the testsuite +Please refer to "examples" folder for example tests + diff --git a/cfg.py b/cfg.py new file mode 100644 index 0000000..7b37b51 --- /dev/null +++ b/cfg.py @@ -0,0 +1,143 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +version = "1.0.0" + +user_password = None +log_location = None + +# None/amd/nvidia +HIP_PLATFORM = None + +# None/0/1/2/3 +Optimization_Level = None + +# None/0/1/2/4 +HIPCC_VERBOSE = None + +# None/0/1 +CONFORMANCE_VERBOSE = None + +# None/cuda directory +CUDA_PATH = None + +# None/rocm directory +ROCM_PATH = None + +# None/offload target +build_for_gfx_target = None +build_for_cuda_target = "compute_70" + +# -I. +# None/List of paths +includes_path = None + +# -l +# None/List of -l +link_libs = None + +# L. +# None/list of -L +link_libs_path = None + +# None/(test_name/test_suite/regex OR list of test_names/test_suites/regex) +# e.g.1 run_tests = ["bitextract", "matrixtranspose"] +# e.g.2 run_tests = "hip_samples" +# e.g.3 run_tests = ".*samples.*" + +# If ts: test suite, tc: test case +# e.g.4 run_tests = [[ts1,tc1], [ts1,tc2], [tc3], [ts2,tc4], [ts3]] +# e.g.4 run_tests = [ts1:tc1, ts1:tc2, tc3, ts2:tc4, ts3] +run_tests = None + + +branch = None +repos = { + "hip_examples": { + "repo_url": "https://github.com/ROCm-Developer-Tools/HIP-Examples", + "branch": None, + "commit_id": "8a3b04c9b10bae344c7483a63c13034869da184b" + }, + "hip": { + "repo_url": "https://github.com/ROCm-Developer-Tools/HIP", + "branch": None, + "commit_id": "96ee9d1397f02ac4b4badd9243994728f6a89fe5" + }, + "mixbench": { + "repo_url": "https://github.com/ekondis/mixbench.git", + "branch": None, + "commit_id": "e1d6c00bd86d7d904b658213370ddb780a116d1f" + }, + "gpu_stream": { + "repo_url": "https://github.com/UoB-HPC/GPU-STREAM.git", + "branch": None, + "commit_id": "6fe81e19556ac26761a1c7247ae29fa88fb4e0ab" + }, + "mgbench": { + "repo_url": "https://github.com/tbennun/mgbench.git", + "branch": None, + "commit_id": "6f12d3848020af8f718074a30c68e6f0b232bfb3" + }, + "cuda_grep": { + "repo_url": "https://github.com/bkase/CUDA-grep.git", + "branch": None, + "commit_id": "fa6630eb0c3e782620ec3eaf989873dadc9a036b" + }, + "cuda_memtest": { + "repo_url": "https://github.com/ComputationalRadiationPhysics/cuda_memtest.git", + "branch": None, + "commit_id": "0cd3a996ce82682fcf50fa6f433b6f1f2ce1353d" + }, + "quicksilver": { + "repo_url": "https://github.com/LLNL/Quicksilver.git", + "branch": "AMD-HIP", + "commit_id": "3eddfc36003de27ee404499923b18e04efef8dad" + }, + "gridtools": { + "repo_url": "https://github.com/GridTools/gridtools.git", + "branch": "v1.1.3", + "commit_id": "d33fa6fecee0a7bd9e080212c1038f0dbd31fe97" + }, + "gtbench": { + "repo_url": "https://github.com/GridTools/gtbench.git", + "branch": None, + "commit_id": "42687cce5085e0e175cb02fefe87fbb3d952fd5c" + }, + "kokkos": { + "repo_url": "https://github.com/kokkos/kokkos.git", + "branch": None, + "commit_id": "c28a8b03288b185f846ddfb1b7c08213e12e2634" + }, + "mfem": { + "repo_url": "https://github.com/mfem/mfem.git ./mfem", + "branch": "amd", + "commit_id": "1ded8554ea470e2018284a881594b888b938ed0b" + }, + "Laghos": { + "repo_url": "https://github.com/CEED/Laghos.git", + "branch": None, + "commit_id": "a7f6123d42847f6bdbdb614f5af876541f49cd16" + }, + "cp2k": { + "repo_url": "git@github.com:cp2k/cp2k.git", + "branch": "master", + "commit_id": "a11bcb3b8565e52344423df66485879857c7b754" + } +} diff --git a/cleanPrj.sh b/cleanPrj.sh new file mode 100755 index 0000000..59f48a8 --- /dev/null +++ b/cleanPrj.sh @@ -0,0 +1,22 @@ +#!/bin/sh + +# This shell script removes all generated files/folders and cloned repositories +echo "Cleaning hip-testsuite project.." +find . | grep -E "(__pycache__|\.pyc|\.pyo$)" | xargs rm -rf +rm -Rf report +rm -Rf src/hiptestsuite/applications/cuda_grep/CUDA-grep +rm -Rf src/hiptestsuite/applications/cuda_memtest/cuda_memtest +rm -Rf src/hiptestsuite/applications/hip_examples/GPU-STREAM +rm -Rf src/hiptestsuite/applications/hip_examples/HIP-Examples +rm -Rf src/hiptestsuite/applications/hip_examples/mixbench +rm -Rf src/hiptestsuite/applications/mgbench/mgbench +rm -Rf src/hiptestsuite/applications/hpc_apps/gridtools/boost_1_72_0 +rm -Rf src/hiptestsuite/applications/hpc_apps/gridtools/GridTools +rm -Rf src/hiptestsuite/applications/hpc_apps/gridtools/gridtools +rm -Rf src/hiptestsuite/applications/hpc_apps/kokkos/kokkos +rm -Rf src/hiptestsuite/applications/hpc_apps/laghos/hypre* +rm -Rf src/hiptestsuite/applications/hpc_apps/laghos/Laghos +rm -Rf src/hiptestsuite/applications/hpc_apps/laghos/metis* +rm -Rf src/hiptestsuite/applications/hpc_apps/laghos/mfem +rm -Rf src/hiptestsuite/applications/hpc_apps/quicksilver/Quicksilver +rm -Rf src/hiptestsuite/conformance/HIP diff --git a/examples/cfg.py b/examples/cfg.py new file mode 100644 index 0000000..e8e0e08 --- /dev/null +++ b/examples/cfg.py @@ -0,0 +1,102 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +version = "1.0.0" + +user_password = "AH64_uh1" +log_location = None + +# None/amd/nvidia +HIP_PLATFORM = None + +# None/0/1/2/3 +Optimization_Level = None + +# None/0/1/2/4 +HIPCC_VERBOSE = None + +# None/cuda directory +CUDA_PATH = None + +# None/rocm directory +ROCM_PATH = None + +# None/offload target +build_for_target = None + +# -I. +# None/List of paths +includes_path = None + +# -l +# None/List of -l +link_libs = None + +# L. +# None/list of -L +link_libs_path = None + +# User Input for example 4.py +user_value = 5 + +# None/(test_name/test_suite/regex OR list of test_names/test_suites/regex) +# e.g.1 run_tests = ["bitextract", "matrixtranspose"] +# e.g.2 run_tests = "hip_samples" +# e.g.3 run_tests = ".*samples.*" + +# If ts: test suite, tc: test case +# e.g.4 run_tests = [[ts1,tc1], [ts1,tc2], [tc3], [ts2,tc4], [ts3]] +# e.g.4 run_tests = [ts1:tc1, ts1:tc2, tc3, ts2:tc4, ts3] +run_tests = None + + +branch = "rocm-4.2.x" +repos = { + "hip_examples": { + "repo_url": "https://github.com/ROCm-Developer-Tools/HIP-Examples", + "branch": branch, + # "commit_id": "" + }, + "hip": { + "repo_url": "https://github.com/ROCm-Developer-Tools/HIP", + "branch": branch, + # "commit_id": "" + }, + "mixbench": { + "repo_url": "https://github.com/ekondis/mixbench.git", + "branch": None, + # "commit_id": "" + }, + "gpu_stream": { + "repo_url": "https://github.com/UoB-HPC/GPU-STREAM.git", + "branch": None, + # "commit_id": "" + }, + "rocclr": { + "repo_url": "https://github.com/ROCm-Developer-Tools/ROCclr.git", + "branch": branch, + # "commit_id": "" + }, + "opencl": { + "repo_url": "https://github.com/RadeonOpenCompute/ROCm-OpenCL-Runtime.git", + "branch": branch, + # "commit_id": "" + } +} diff --git a/examples/examples/0.py b/examples/examples/0.py new file mode 100644 index 0000000..4f29896 --- /dev/null +++ b/examples/examples/0.py @@ -0,0 +1,39 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from pathlib import Path + +from hiptestsuite.TesterRepository import Tester +from hiptestsuite.Test import TestData, TestResult + + +class Test0(Tester): + """ + Simple test case, + Which tests whether rocm is installed + """ + def __init__(self): + Tester.__init__(self) + + def test(self, test_data: TestData): + if Path("/opt/rocm").exists(): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL diff --git a/examples/examples/1.py b/examples/examples/1.py new file mode 100644 index 0000000..ae5ef7e --- /dev/null +++ b/examples/examples/1.py @@ -0,0 +1,55 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.TesterRepository import Tester +from hiptestsuite.Test import TestData, TestResult, Test +from hiptestsuite.test_classifier import TestClassifier + +from typing import List, Union + + +class C1(TestClassifier): + def __init__(self): + TestClassifier.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + TestClassifier.add_matched_with_names(self, {"c1": matched_with_names}) + + +class Test1(Tester): + """ + This assigns classifier to test, so it can be invoked with c1, c1:test1, test1 + """ + def __init__(self): + Tester.__init__(self) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + c1 = C1() + c1.add_matched_with_names() + test.classifiers = [c1] + test.tester = self + + return [test] + + def test(self, test_data: TestData): + print("called for {test_name}".format(test_name=test_data.test.test_name)) + test_data.test_result = TestResult.PASS diff --git a/examples/examples/2.py b/examples/examples/2.py new file mode 100644 index 0000000..095fc66 --- /dev/null +++ b/examples/examples/2.py @@ -0,0 +1,71 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.TesterRepository import Tester +from hiptestsuite.Test import TestData, TestResult, Test +from hiptestsuite.test_classifier import TestClassifier + +from typing import List, Union + + +class C1(TestClassifier): + def __init__(self): + TestClassifier.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + TestClassifier.add_matched_with_names(self, {"c1": matched_with_names}) + + +class C2(C1): + def __init__(self): + C1.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + C1.add_matched_with_names(self, {"c2": matched_with_names}) + + +class Test2(Tester): + """ + This assigns classifier to test, so it can be invoked with c1, c1:test2_1, test2_1 like 1.py. + In addition to that, it adds test2_2, and assigns classifier c2, which is subset of c1 + """ + def __init__(self): + Tester.__init__(self) + + def getTests(self) -> List[Test]: + test2_1 = Test() + test2_1.test_name = "test2_1" + c1 = C1() + c1.add_matched_with_names() + test2_1.classifiers = [c1] + test2_1.tester = self + + test2_2 = Test() + test2_2.test_name = "test2_2" + c2 = C2() + c2.add_matched_with_names() + test2_2.classifiers = [c2] + test2_2.tester = self + + return [test2_1, test2_2] + + def test(self, test_data: TestData): + print("called for {test_name}".format(test_name=test_data.test.test_name)) + test_data.test_result = TestResult.PASS diff --git a/examples/examples/3.py b/examples/examples/3.py new file mode 100644 index 0000000..851f429 --- /dev/null +++ b/examples/examples/3.py @@ -0,0 +1,37 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.TesterRepository import Tester +from hiptestsuite.Test import HIPTestData, TestResult, Test +from hiptestsuite.test_classifier import TestClassifier + +from typing import List, Union + + +class PlatformTest(Tester): + """ + This is example for accessing hip platform to test with + """ + def __init__(self): + Tester.__init__(self) + + def test(self, test_data: HIPTestData): + print(test_data.HIP_PLATFORM) + test_data.test_result = TestResult.PASS diff --git a/examples/examples/4.py b/examples/examples/4.py new file mode 100644 index 0000000..bc9c587 --- /dev/null +++ b/examples/examples/4.py @@ -0,0 +1,52 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.TesterRepository import Tester +from hiptestsuite.Test import TestResult, TestData, LogLocation, UserAccess + + +class MyTestData(TestData, LogLocation, UserAccess): + """ + Custom Test Input Data + """ + def __init__(self): + self.user_value = None + UserAccess.__init__(self) + LogLocation.__init__(self) + TestData.__init__(self) + + def loadConfig(self): + UserAccess.loadConfig(self) + self.user_value = self.config.user_value + + +class CustomTestData(Tester): + """ + This is example for accessing creating custom test data, here of type MyTestData + that can be constructed using user input values + """ + def __init__(self): + Tester.__init__(self) + + def test(self, test_data: MyTestData): + if test_data.user_value == 5: + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL diff --git a/examples/examples/__init__.py b/examples/examples/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/examples/run.py b/examples/run.py new file mode 100644 index 0000000..1ff802c --- /dev/null +++ b/examples/run.py @@ -0,0 +1,78 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import sys +import os +import argparse + +if __name__ == "__main__": + sys.path.append(os.path.join(os.path.dirname(__file__), "..", "src")) + + +from hiptestsuite.TestersExecutor import TestersExecutor +from hiptestsuite.TesterRepository import TesterRepository +from hiptestsuite.list_tests import list_tests +import cfg +import examples + + +def parse_args(tester_repository): + parser = argparse.ArgumentParser() + parser.add_argument('--platform', help="On which hip_platform to test? amd/nvidia/ default:amd") + parser.add_argument('-t', '--tests', nargs='+', + # required=True, + # choices=TESTS_CHOICES, + metavar='', help="Test name/Regex/Category/Category:*/List of those separated by space") + parser.add_argument('-lst', '--list_tests', default=False, action='store_true', help="List all tests") + parser.add_argument('-lstq', '--list_tests_quick', default=False, action='store_true', help="List all tests quickly, Warning: This may not list some tests which are time consuming to generate, and only category:* will be displayed for them, use -lst for listing all tests") + + args = parser.parse_args() + + if args.platform: + cfg.HIP_PLATFORM = args.platform + + if args.list_tests: + list_tests(quick=False, cfg=cfg, tester_repository=tester_repository) + return False + + if args.list_tests_quick: + list_tests(quick=True, cfg=cfg, tester_repository=tester_repository) + return False + + if args.tests: + cfg.run_tests = args.tests + + return True + + +def main(): + tester_repository = TesterRepository() + tester_repository.clearTesterFrom() + tester_repository.addTesterFrom(pkgs=[examples]) + tester_repository.addAllTesters() + tester_executor: TestersExecutor = TestersExecutor() + tester_executor.config = cfg + + if parse_args(tester_repository=tester_repository): + tester_executor.executeTests(tester_repository=tester_repository) + + +if __name__ == "__main__": + main() diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000..e8d889f --- /dev/null +++ b/requirements.txt @@ -0,0 +1 @@ +prettytable==2.1.0 \ No newline at end of file diff --git a/run.py b/run.py new file mode 100644 index 0000000..9edd23e --- /dev/null +++ b/run.py @@ -0,0 +1,74 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import sys +import os +import argparse + +if __name__ == "__main__": + sys.path.append(os.path.join(os.path.dirname(__file__), "src")) + + +from hiptestsuite.TestersExecutor import TestersExecutor +from hiptestsuite.list_tests import list_tests +import cfg + + +def parse_args(): + parser = argparse.ArgumentParser() + parser.add_argument('--platform', help="On which hip_platform to test? amd/nvidia/ default:amd") + parser.add_argument('-t', '--tests', nargs='+', + # required=True, + # choices=TESTS_CHOICES, + metavar='', help="Test name/Regex/Category/Category:*/List of those separated by space") + parser.add_argument('-lst', '--list_tests', default=False, action='store_true', help="List all tests") + parser.add_argument('-lstq', '--list_tests_quick', default=False, action='store_true', help="List all tests quickly, Warning: This may not list some tests which are time consuming to generate, and only category:* will be displayed for them, use -lst for listing all tests") + + args = parser.parse_args() + + if args.platform: + cfg.HIP_PLATFORM = args.platform + + if args.list_tests: + list_tests(quick=False, cfg=cfg) + return False + + if args.list_tests_quick: + list_tests(quick=True, cfg=cfg) + return False + + if args.tests: + cfg.run_tests = args.tests + + return True + + +def main(): + # Exclude tests from thirdparty folder + exclude_module_paths = ["hiptestsuite/thirdparty"] + + if parse_args(): + tester_executor: TestersExecutor = TestersExecutor() + tester_executor.config = cfg + tester_executor.executeTests(exclude_module_paths=exclude_module_paths) + + +if __name__ == "__main__": + main() diff --git a/src/hiptestsuite/AMD.py b/src/hiptestsuite/AMD.py new file mode 100644 index 0000000..32cf8c0 --- /dev/null +++ b/src/hiptestsuite/AMD.py @@ -0,0 +1,23 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +class AMDObject: + def __init__(self): + pass diff --git a/src/hiptestsuite/Test.py b/src/hiptestsuite/Test.py new file mode 100644 index 0000000..a580665 --- /dev/null +++ b/src/hiptestsuite/Test.py @@ -0,0 +1,244 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from typing import List, Union, Set, Dict +from hiptestsuite.AMD import AMDObject +from hiptestsuite.targets import Target +from enum import Enum, auto +from hiptestsuite.test_classifier import TestClassifier +from hiptestsuite.config_processor import ConfigProcessor + + +class Test(AMDObject): + def __init__(self): + # Cycle Import + from hiptestsuite.TesterRepository import Tester + + AMDObject.__init__(self) + self.test_name: Union[None, str] = None + self.also_matched_with_test_names: Union[None, List[str]] = None + self.classifiers: Union[None, List[TestClassifier]] = None + self.tester: Union[None, Tester] = None + self.applicable_for_target: Union[None, Set[Target]] = None + + +class TestResult(Enum): + PASS = auto() + FAIL = auto() + SKIP = auto() + ERROR = auto() + + +class UserAccess(ConfigProcessor): + def __init__(self): + ConfigProcessor.__init__(self) + self.user_password: Union[None, str] = None + + def loadConfig(self): + self.user_password = self.config.user_password + + +class LogLocation(AMDObject): + def __init__(self): + AMDObject.__init__(self) + self.log_location: Union[None, str] = None + + +class TestData(AMDObject): + def __init__(self): + AMDObject.__init__(self) + self.test: Union[None, Test] = None + self.test_result: Union[None, TestResult] = None + + +class ConformanceTestData(AMDObject): + def __init__(self): + AMDObject.__init__(self) + self.CONFORMANCE_VERBOSE = None + + def loadConfig(self): + if self.config.CONFORMANCE_VERBOSE == 0: + self.CONFORMANCE_VERBOSE = 0 + elif self.config.CONFORMANCE_VERBOSE == 1: + self.CONFORMANCE_VERBOSE = 1 + else: + self.CONFORMANCE_VERBOSE = 0 + + +class CompileData(AMDObject): + def __init__(self): + AMDObject.__init__(self) + self.build_for_gfx_target: Union[None, Set[Target]] = None + self.build_for_cuda_target: Union[None, Set[Target]] = None + + +class GitData(AMDObject): + def __init__(self): + AMDObject.__init__(self) + self.repo_url: Union[None, str] = None + self.branch: Union[None, str] = None + self.commit_id: Union[None, str] = None + + +class AllGitData(ConfigProcessor): + def __init__(self): + ConfigProcessor.__init__(self) + self.repos: Union[None, Dict[str, GitData]] = None + + def loadConfig(self): + repos = dict() + for repo_key, repo_detail in self.config.repos.items(): + git_data = GitData() + git_data.repo_url = repo_detail["repo_url"] + if "branch" in repo_detail: + git_data.branch = repo_detail["branch"] + if "commit_id" in repo_detail: + git_data.commit_id = repo_detail["commit_id"] + repos[repo_key] = git_data + self.repos = repos + + +class HIPCCVerbose(Enum): + ZERO = auto() + ONE = auto() + TWO = auto() + FOUR = auto() + +class Optimization_Level(Enum): + ZERO = auto() + ONE = auto() + TWO = auto() + THREE = auto() + + +class HIP_PLATFORM(Enum): + nvidia = auto() + amd = auto() + + +class HIPCCCompileData(CompileData, ConfigProcessor): + def __init__(self): + CompileData.__init__(self) + ConfigProcessor.__init__(self) + + self.Optimization_Level: Union[None, Optimization_Level] = None + + self.HIPCC_VERBOSE: Union[None, HIPCCVerbose] = None + self.HIP_PLATFORM: Union[None, HIP_PLATFORM] = None + + self.CUDA_PATH: Union[None, str] = None + self.ROCM_PATH: Union[None, str] = None + + # if self.build_for_gfx_target, then --offload_arch= + + # -I ./ + self.includes_path: Union[None, List[str]] = None + # -lm + self.link_libs: Union[None, List[str]] = None + # -L. + self.link_libs_path: Union[None, List[str]] = None + + def loadConfig(self): + if self.config.HIP_PLATFORM == "amd": + print("Platform selected: amd") + self.HIP_PLATFORM = HIP_PLATFORM.amd + elif self.config.HIP_PLATFORM == "nvidia": + print("Platform selected: nvidia") + self.HIP_PLATFORM = HIP_PLATFORM.nvidia + elif self.config.HIP_PLATFORM is None: + print("Platform selected: amd") + self.HIP_PLATFORM = HIP_PLATFORM.amd + else: + print("Warning: Platform " + self.config.HIP_PLATFORM + " is not supported, defaulting to AMD") + self.HIP_PLATFORM = HIP_PLATFORM.amd + + if self.config.Optimization_Level == 0: + self.Optimization_Level = Optimization_Level.ZERO + elif self.config.Optimization_Level == 1: + self.Optimization_Level = Optimization_Level.ONE + elif self.config.Optimization_Level == 2: + self.Optimization_Level = Optimization_Level.TWO + elif self.config.Optimization_Level == 3: + self.Optimization_Level = Optimization_Level.THREE + else: + self.Optimization_Level = None + + if self.config.HIPCC_VERBOSE == 0: + self.HIPCC_VERBOSE = HIPCCVerbose.ZERO + elif self.config.HIPCC_VERBOSE == 1: + self.HIPCC_VERBOSE = HIPCCVerbose.ONE + elif self.config.HIPCC_VERBOSE == 2: + self.HIPCC_VERBOSE = HIPCCVerbose.TWO + elif self.config.HIPCC_VERBOSE == 4: + self.HIPCC_VERBOSE = HIPCCVerbose.FOUR + else: + self.HIPCC_VERBOSE = None + + self.ROCM_PATH = self.config.ROCM_PATH + self.CUDA_PATH = self.config.CUDA_PATH + + self.build_for_gfx_target = self.config.build_for_gfx_target + self.build_for_cuda_target = self.config.build_for_cuda_target + self.includes_path = self.config.includes_path + self.link_libs = self.config.link_libs + self.link_libs_path = self.config.link_libs_path + + # ToDo offload_target + + +class HIPTestData(TestData, HIPCCCompileData, AllGitData, LogLocation, UserAccess, ConformanceTestData): + def __init__(self): + UserAccess.__init__(self) + LogLocation.__init__(self) + TestData.__init__(self) + ConformanceTestData.__init__(self) + HIPCCCompileData.__init__(self) + AllGitData.__init__(self) + + def loadConfig(self): + UserAccess.loadConfig(self) + ConformanceTestData.loadConfig(self) + AllGitData.loadConfig(self) + HIPCCCompileData.loadConfig(self) + + +class Quick(AMDObject): + def __init__(self): + AMDObject.__init__(self) + self.quick: bool = False + + +class GetTestsData(Quick): + def __init__(self): + Quick.__init__(self) + + +class HIPBuildData(GetTestsData, HIPCCCompileData, UserAccess, AllGitData, LogLocation): + def __init__(self): + GetTestsData.__init__(self) + HIPCCCompileData.__init__(self) + UserAccess.__init__(self) + AllGitData.__init__(self) + LogLocation.__init__(self) + + def loadConfig(self): + UserAccess.loadConfig(self) + AllGitData.loadConfig(self) + HIPCCCompileData.loadConfig(self) diff --git a/src/hiptestsuite/TesterRepository.py b/src/hiptestsuite/TesterRepository.py new file mode 100644 index 0000000..5f2248e --- /dev/null +++ b/src/hiptestsuite/TesterRepository.py @@ -0,0 +1,148 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.Test import Test, TestData, GetTestsData, LogLocation, Quick +from hiptestsuite.test_classifier import TestClassifier +from hiptestsuite.AMD import AMDObject +from hiptestsuite.config_processor import ConfigProcessor +from hiptestsuite.match_fun_args_call import match_fun_args_call +import hiptestsuite + +from typing import List, Union +import pkgutil +import traceback +import typing + + +class Tester(AMDObject): + def __init__(self): + AMDObject.__init__(self) + + def getTests(self, get_tests_data: GetTestsData) -> List[Test]: + test = Test() + test.tester = self + test.test_name = self.__class__.__name__ + return [test] + + def test(self, test_data: TestData): + pass + + def get_test_classifiers(self) -> Union[None, List[TestClassifier]]: + return None + + def clean(self): + pass + + +class TesterRepository(AMDObject): + def __init__(self): + AMDObject.__init__(self) + self.getTestersFrom = [hiptestsuite] + self.testers: Union[None, List[Tester]] = None + + def getTesters(self) -> List[Tester]: + return self.testers + + def addTesterFrom(self, pkgs: List): + self.getTestersFrom.extend(pkgs) + + def clearTesterFrom(self): + self.getTestersFrom.clear() + + def addAllTesters(self): + if self.testers is None: + self.testers = list() + tester_children = get_cls_children_from_pkgs(cls=Tester, pkgs=self.getTestersFrom) + for tester_child in tester_children: + self.testers.append(tester_child()) + + def addTester(self, tester: Tester): + if self.testers is None: + self.testers = list() + self.testers.append(tester) + + +class GetTests(ConfigProcessor): + def __init__(self, tester_repository: TesterRepository): + ConfigProcessor.__init__(self) + self.tester_repository = tester_repository + + def get_tests(self, log_location=None, quick=None): + config = self.config + testers: List[Tester] = self.tester_repository.getTesters() + tests = list() + + for tester in testers: + get_tests_t = typing.get_type_hints(tester.getTests) + get_tests_data_t = None + if get_tests_t: + if "get_tests_data" in get_tests_t: + get_tests_data_t = get_tests_t["get_tests_data"] + + if get_tests_data_t is not None: + get_tests_data = get_tests_data_t() + else: + get_tests_data = None + + if isinstance(get_tests_data, LogLocation): + get_tests_data: LogLocation + get_tests_data.log_location = log_location + + if isinstance(get_tests_data, ConfigProcessor): + get_tests_data.config = config + get_tests_data.loadConfig() + + if isinstance(get_tests_data, Quick): + get_tests_data: Quick + get_tests_data.quick = quick + + try: + tests_of_tester = match_fun_args_call(fun=tester.getTests, args={"get_tests_data": get_tests_data}) + tests.extend(tests_of_tester) + except Exception as err: + print("{tester} failed to generate tests".format(tester=tester.__class__.__name__)) + traceback.print_exc() + continue + return tests + + +def get_cls_children_from_pkgs(cls, pkgs: List) -> set: + import_all_scripts_from(pkgs=pkgs) + cls_children = get_cls_children(cls=cls) + return cls_children + + +def get_cls_children(cls) -> set: + cls_children = set() + + for subclass in cls.__subclasses__(): + cls_children.add(subclass) + cls_children.union(get_cls_children(subclass)) + + return cls_children + + +def import_all_scripts_from(pkgs: List): + for pkg in pkgs: + for importer, modname, ispkg in pkgutil.iter_modules(pkg.__path__): + if ispkg: + import_all_scripts_from([importer.find_module(modname).load_module(modname)]) + else: + importer.find_module(modname).load_module(modname) diff --git a/src/hiptestsuite/TestersExecutor.py b/src/hiptestsuite/TestersExecutor.py new file mode 100644 index 0000000..8bfaf69 --- /dev/null +++ b/src/hiptestsuite/TestersExecutor.py @@ -0,0 +1,336 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.TesterRepository import TesterRepository, Tester, Test +from hiptestsuite.test_selector import TestSelector +from hiptestsuite.config_processor import ConfigProcessor +from hiptestsuite.Test import TestResult + +import os +import traceback +import logging +import subprocess +import sys +import typing +from typing import Union, List, Dict +import json +import re +import time +import datetime + + +class TestersExecutor(ConfigProcessor): + def __init__(self): + ConfigProcessor.__init__(self) + + def executeTests(self, tester_repository: TesterRepository=None, exclude_module_paths=None): + start_datetime = datetime.datetime.now() + config = self.config + log_location = config.log_location + if log_location is None: + log_location = os.getcwd() + os.makedirs(log_location, exist_ok=True) + root_log_dir = "report" + root_log_location = os.path.join(log_location, root_log_dir) + os.makedirs(root_log_location, exist_ok=True) + timestamped_log_location = os.path.join(root_log_location, start_datetime.strftime("%Y_%m_%d_%H_%M_%S")) + os.makedirs(timestamped_log_location, exist_ok=True) + relative_timestamped_log_location = os.path.join(root_log_dir, start_datetime.strftime("%Y_%m_%d_%H_%M_%S")) + logger = logging.getLogger(__name__) + logger.setLevel(logging.DEBUG) + formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') + ch = logging.StreamHandler() + ch.setLevel(logging.DEBUG) + ch.setFormatter(formatter) + fh = logging.FileHandler(filename=os.path.join(timestamped_log_location, "report.log"), mode='w+') + fh.setLevel(logging.DEBUG) + fh.setFormatter(formatter) + logger.addHandler(ch) + logger.addHandler(fh) + + selected_test_filter = config.run_tests + if selected_test_filter: + logger.info("Selected Test Filter: {selected_test_filter}".format(selected_test_filter=" ".join(selected_test_filter))) + logger.info("Execution Logs: {log_location}".format(log_location=log_location)) + + if tester_repository is None: + tester_repository: TesterRepository = TesterRepository() + tester_repository.addAllTesters() + + test_selector: TestSelector = TestSelector(tester_repository=tester_repository) + test_selector.config = config + tests: List[Test] = test_selector.select_tests(log_location=timestamped_log_location, exclude_module_paths=exclude_module_paths) + tests: List[Test] = sorted(tests, key=lambda x: x.test_name) + tests_status = dict() + tests_logs = dict() + tests_relative_logs = dict() + + for test in tests: + print("Started Test: {test_name}".format(test_name=test.test_name.lower())) + + test_t = typing.get_type_hints(test.tester.test) + test_data_t = None + if test_t: + test_data_t = test_t["test_data"] + + test_data = test_data_t() + + if isinstance(test_data, ConfigProcessor): + test_data: ConfigProcessor + test_data.config = config + test_data.loadConfig() + test_data.test = test + test_data.log_location = os.path.join(timestamped_log_location, test.test_name.lower() + ".log.d") + os.makedirs(test_data.log_location, exist_ok=True) + + try: + test.tester.test(test_data=test_data) + except Exception as error: + test_data.test_result = TestResult.ERROR + traceback.print_exc() + + tests_status[test] = test_data.test_result + tests_logs[test] = test_data.log_location + tests_relative_logs[test] = os.path.join(relative_timestamped_log_location, test.test_name.lower() + ".log.d") + print("Completed Test: {test_name} with result {result}".format(test_name=test.test_name.lower(), result=test_data.test_result.name)) + + for test in tests: + try: + test.tester.clean() + except Exception as error: + traceback.print_exc() + + end_datetime = datetime.datetime.now() + + # ### Reporting + try: + opt_rocm_version: Union[None, str] = get_opt_rocm_version() + except Exception as error: + opt_rocm_version = None + try: + os_name: Union[None, str] = get_os_name() + except Exception as error: + os_name = None + try: + os_version: Union[None, str] = get_os_version() + except Exception as error: + os_version = None + try: + rocm_agents: Union[None, List[str]] = get_rocm_agents() + except Exception as error: + rocm_agents = None + try: + cuda_gpus: Union[None, List[str]] = get_cuda_gpus() + except Exception as error: + cuda_gpus = None + try: + cuda_rt_version: str = get_cuda_rt_version() + except Exception as error: + cuda_rt_version = None + + passed_tests = get_passed_tests(tests_status=tests_status) + failed_tests = get_failed_tests(tests_status=tests_status) + errored_tests = get_errored_tests(tests_status=tests_status) + skipped_tests = get_skipped_tests(tests_status=tests_status) + + logger.info("Start Time: {start_datetime}".format(start_datetime=start_datetime.strftime("%Y/%m/%d %H:%M:%S"))) + logger.info("End Time: {end_datetime}".format(end_datetime=end_datetime.strftime("%Y/%m/%d %H:%M:%S"))) + # ### prettytable + field_names = ["Test Name", "Result", "Log"] + system_info_field_names = ["Component", "Information"] + test_cnt_field_names = ["TOTAL", "PASS", "FAIL", "ERROR", "SKIP"] + try: + from prettytable import PrettyTable + if tests_status: + summary_table = PrettyTable() + summary_table.field_names = field_names + test_cnt_table = PrettyTable() + test_cnt_table.field_names = test_cnt_field_names + for test, test_status in passed_tests.items(): + summary_table.add_row([test.test_name.lower(), test_status.name, tests_relative_logs[test]]) + for test, test_status in failed_tests.items(): + summary_table.add_row([test.test_name.lower(), test_status.name, tests_relative_logs[test]]) + for test, test_status in errored_tests.items(): + summary_table.add_row([test.test_name.lower(), test_status.name, tests_relative_logs[test]]) + for test, test_status in skipped_tests.items(): + summary_table.add_row([test.test_name.lower(), test_status.name, tests_relative_logs[test]]) + + logger.info('\n' + summary_table.get_string(title="Summary")) + + test_cnt_table.add_row([str(len(tests_status)), str(len(passed_tests)), str(len(failed_tests)), str(len(errored_tests)), str(len(skipped_tests))]) + logger.info('\n' + test_cnt_table.get_string(title="Metrics")) + + system_info_table = PrettyTable() + system_info_table.field_names = system_info_field_names + system_info_table.add_row(["OS", (os_name if os_name else "Can't get OS name!") + " " + (os_version if os_version else "Can't get OS version")]) + system_info_table.add_row(["ROCm Agents", ", ".join(rocm_agents) if rocm_agents else "Can't get ROCm agents"]) + system_info_table.add_row(["/opt/rocm version", opt_rocm_version if opt_rocm_version else "Can't get /opt/rocm version"]) + system_info_table.add_row(["CUDA RT Version", cuda_rt_version if cuda_rt_version else "Can't get CUDA RT Version"]) + system_info_table.add_row(["CUDA GPUs", ", ".join(cuda_gpus) if cuda_gpus else "Can't get CUDA GPUs"]) + logger.info('\n' + system_info_table.get_string(title="System Information")) + except Exception as error: + logger.warning("For better UI experience, please pip3 install -r requirements.txt") + + if tests_status: + logger.info("********Summary********") + logger.info(" | ".join(field_names)) + for test, test_status in passed_tests.items(): + logger.info(test.test_name.lower() + " | " + test_status.name + " | " + tests_relative_logs[test]) + for test, test_status in failed_tests.items(): + logger.info(test.test_name.lower() + " | " + test_status.name + " | " + tests_relative_logs[test]) + for test, test_status in errored_tests.items(): + logger.info(test.test_name.lower() + " | " + test_status.name + " | " + tests_relative_logs[test]) + for test, test_status in skipped_tests.items(): + logger.info(test.test_name.lower() + " | " + test_status.name + " | " + tests_relative_logs[test]) + + logger.info("********Metrics********") + logger.info(" | ".join(test_cnt_field_names)) + logger.info(str(len(tests_status)) + " | " + str(len(passed_tests)) + " | " + str(len(failed_tests)) + " | " + str(len(errored_tests)) + " | " + str(len(skipped_tests))) + + logger.info("********System Information********") + logger.info(" | ".join(system_info_field_names)) + logger.info("OS" + " | " + (os_name if os_name else "Can't get OS name!") + " " + (os_version if os_version else "Can't get OS version")) + logger.info("ROCm Agents" + " | " + ", ".join(rocm_agents) if rocm_agents else "Can't get ROCm agents") + logger.info("/opt/rocm version" + " | " + opt_rocm_version if opt_rocm_version else "Can't get /opt/rocm version") + logger.info("CUDA RT Version" + " | " + cuda_rt_version if cuda_rt_version else "Can't get CUDA RT Version") + logger.info("CUDA GPUs" + " | " + ", ".join(cuda_gpus) if cuda_gpus else "Can't get CUDA GPUs") + + logger.info("Note, All log locations are relative to {log_location}".format(log_location=log_location)) + + # ### json + json_root = dict() + tests_root = json_root["tests"] = dict() + for test, test_status in tests_status.items(): + test_root = tests_root[test.test_name.lower()] = dict() + test_root["status"] = test_status.name + test_root["log_location"] = tests_logs[test] + + json_root["num_total"] = len(tests_status) + json_root["num_passed"] = len(passed_tests) + json_root["num_failed"] = len(failed_tests) + json_root["num_errored"] = len(errored_tests) + json_root["num_skipped"] = len(skipped_tests) + json_root["opt_rocm_version"] = opt_rocm_version + json_root["os_name"] = os_name + json_root["os_version"] = os_version + json_root["rocm_agents"] = rocm_agents + json_root["cuda_gpus"] = cuda_gpus + json_root["cuda_rt_version"] = cuda_rt_version + json_root["start_datetime"] = start_datetime.strftime("%Y_%m_%d_%H_%M_%S") + json_root["end_datetime"] = end_datetime.strftime("%Y_%m_%d_%H_%M_%S") + json_root["selected_test_filter"] = selected_test_filter + + with open(os.path.join(timestamped_log_location, 'report.json'), 'w+', encoding='utf-8') as f: + json.dump(json_root, f, ensure_ascii=False, indent=4) + + print("") + print("Test Complete: Log file directory is " + relative_timestamped_log_location) + + + +def get_os_name() -> str: + cmd = "awk -F= '/^NAME=/{print $2}' /etc/os-release" + o, e = subprocess.Popen([cmd], shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=True).communicate() + o = o.decode('utf-8') + os_version = o.strip('\n') + os_version = os_version.replace('"', '') + return os_version + + +def get_os_version() -> str: + cmd = "awk -F= '/^VERSION_ID=/{print $2}' /etc/os-release" + o, e = subprocess.Popen([cmd], shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=True).communicate() + o = o.decode('utf-8') + os_version = o.strip('\n') + os_version = os_version.replace('"', '') + return os_version + + +class NoROCmAgentsFound(Exception): + def __init__(self): + Exception.__init__(self) + + +def get_rocm_agents() -> List[str]: + gpus = list() + cmd = "/opt/rocm/bin/rocminfo" + o, e = subprocess.Popen([cmd], shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=True).communicate() + o = o.decode('utf-8') + for l in o.splitlines(): + gpus_match = re.findall("^ Name:\s+(.+)", l) + if gpus_match: + gpus.append(gpus_match[0].strip()) + if not gpus: + raise NoROCmAgentsFound() + return gpus + + +def get_opt_rocm_version() -> str: + cmd = "cat /opt/rocm/.info/version" + o, e = subprocess.Popen([cmd], shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=True).communicate() + o = o.decode('utf-8') + opt_rocm_version = o.strip('\n') + return opt_rocm_version + + +def get_cuda_rt_version() -> str: + cuda_rt_version = None + with open('/usr/local/cuda/version.json', "r") as f: + data = json.load(f) + try: + cuda_rt_version = data['cuda_cudart']['version'] + except: + pass + return cuda_rt_version + + +def get_cuda_gpus() -> List[str]: + cmd = "nvidia-smi --query-gpu=name --format=csv,noheader" + o, e = subprocess.Popen([cmd], shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=True).communicate() + o = o.decode('utf-8') + if o: + cuda_gpus = o.strip('\n').split('\n') + else: + cuda_gpus = None + return cuda_gpus + + +def get_passed_tests(tests_status: Dict[Test, TestResult]) -> Dict[Test, TestResult]: + return get_status_filtered_tests(tests_status=tests_status, status=TestResult.PASS) + + +def get_failed_tests(tests_status: Dict[Test, TestResult]) -> Dict[Test, TestResult]: + return get_status_filtered_tests(tests_status=tests_status, status=TestResult.FAIL) + + +def get_errored_tests(tests_status: Dict[Test, TestResult]) -> Dict[Test, TestResult]: + return get_status_filtered_tests(tests_status=tests_status, status=TestResult.ERROR) + + +def get_skipped_tests(tests_status: Dict[Test, TestResult]) -> Dict[Test, TestResult]: + return get_status_filtered_tests(tests_status=tests_status, status=TestResult.SKIP) + + +def get_status_filtered_tests(tests_status: Dict[Test, TestResult], status: TestResult) -> Dict[Test, TestResult]: + filtered_tests = dict() + for test, test_status in tests_status.items(): + if test_status == status: + filtered_tests[test] = test_status + return filtered_tests diff --git a/src/hiptestsuite/__init__.py b/src/hiptestsuite/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/hiptestsuite/applications/__init__.py b/src/hiptestsuite/applications/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/hiptestsuite/applications/application_test_classifier.py b/src/hiptestsuite/applications/application_test_classifier.py new file mode 100644 index 0000000..757d30c --- /dev/null +++ b/src/hiptestsuite/applications/application_test_classifier.py @@ -0,0 +1,31 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.test_classifier import TestClassifier + +from typing import Union + + +class APPLICATIONS(TestClassifier): + def __init__(self): + TestClassifier.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + TestClassifier.add_matched_with_names(self, {"applications": matched_with_names}) diff --git a/src/hiptestsuite/applications/cuda_grep/__init__.py b/src/hiptestsuite/applications/cuda_grep/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/hiptestsuite/applications/cuda_grep/cuda_grep.py b/src/hiptestsuite/applications/cuda_grep/cuda_grep.py new file mode 100644 index 0000000..2b19bbc --- /dev/null +++ b/src/hiptestsuite/applications/cuda_grep/cuda_grep.py @@ -0,0 +1,151 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.TesterRepository import Tester, Test, TestData +from hiptestsuite.Test import HIPTestData, TestResult, HIP_PLATFORM +from typing import Union, List +from hiptestsuite.test_classifier import TestClassifier +from hiptestsuite.applications.cuda_grep.cuda_grep_build_amd import BuildRunAmd +from hiptestsuite.applications.cuda_grep.cuda_grep_build_nvidia import BuildRunNvidia +from hiptestsuite.common.hip_get_packages import HipPackages +from hiptestsuite.common.hip_shell import execshellcmd + +import os +import re +# Common class to clone, set up, build and run test +class PrepareTest(): + def __init__(self, cwd, binary): + self.cwdAbs = cwd + self.binary = binary + self.app_path = os.path.join(self.cwdAbs,\ + "src/hiptestsuite/applications/cuda_grep/") + self.app_root = os.path.join(self.app_path, "CUDA-grep/") + self.thistestpath = self.app_root + self.prepareobj = None + self.apprepo = "" # Default + self.appbranch = "" + self.appcommitId = "" + + def set_cudagrep_repoinfo(self, test_data: HIPTestData): + validrepconfig = True + if test_data.repos["cuda_grep"].repo_url != None: + self.apprepo = test_data.repos["cuda_grep"].repo_url + else: + validrepconfig &= False + if test_data.repos["cuda_grep"].branch != None: + self.appbranch = test_data.repos["cuda_grep"].branch + if test_data.repos["cuda_grep"].commit_id != None: + self.appcommitId = test_data.repos["cuda_grep"].commit_id + return validrepconfig + + def downloadtest(self, logFile, test_data: HIPTestData): + ret = HipPackages().pull_repo(logFile, self.apprepo,\ + self.appbranch, self.appcommitId, "cudagrep") + return ret + + def buildtest(self, logFile, platform): + isBinaryPresent = True + if platform == HIP_PLATFORM.nvidia: + self.prepareobj = BuildRunNvidia(self.thistestpath, logFile, self.binary) + elif platform == HIP_PLATFORM.amd: + self.prepareobj = BuildRunAmd(self.thistestpath, logFile, self.binary) + else: + print("Invalid Platform") + return False + buildstatus = self.prepareobj.buildtest() + if buildstatus == False: + return False + # Check if test binary is created + if not os.path.isfile(\ + os.path.join(self.thistestpath, self.binary)): + isBinaryPresent &= False + return isBinaryPresent + + def clean(self): + if self.prepareobj != None: + self.prepareobj.clean() + + def runtest(self): + if self.prepareobj != None: + self.prepareobj.runtest() + + def parse_result(self): + if self.prepareobj != None: + return self.prepareobj.parse_result() + return False + +class CUDA_GREP(TestClassifier): + def __init__(self): + TestClassifier.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + TestClassifier.add_matched_with_names(self, {"cuda_grep": matched_with_names}) + +class MINI_APP(CUDA_GREP): + def __init__(self): + CUDA_GREP.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + CUDA_GREP.add_matched_with_names(self, {"mini-app": matched_with_names}) + + +# Test CUDA-grep +class cudagrep(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd, "nfa") + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + mini_app = MINI_APP() + mini_app.add_matched_with_names() + test.classifiers = [mini_app] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== CUDA-grep Test ===============") + # Set repo info + isrepocfgvalid = self.set_cudagrep_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/CUDA-grep.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest() + # Parse the test result + if True == self.parse_result(): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL diff --git a/src/hiptestsuite/applications/cuda_grep/cuda_grep_build_amd.py b/src/hiptestsuite/applications/cuda_grep/cuda_grep_build_amd.py new file mode 100644 index 0000000..654c594 --- /dev/null +++ b/src/hiptestsuite/applications/cuda_grep/cuda_grep_build_amd.py @@ -0,0 +1,70 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os +import tempfile +from hiptestsuite.common.hip_shell import * +from hiptestsuite.applications.cuda_grep.cuda_grep_parser_common import CudaGrepParser + +class BuildRunAmd(): + def __init__(self, thistestpath, logFile, binary): + self.thistestpath = thistestpath + self.logFile = logFile + self.runpath = os.path.join(self.thistestpath, "testbench") + self.runlog = os.path.join(self.thistestpath, "testbench/RESULTS") + self.binary = binary + + def buildtest(self): + # In this function put the build steps for test cases + # which differ across platforms (amd/nvidia/intel) + if not os.path.exists("/opt/rocm"): + print("ROCm not installed. Exiting!") + return False + print("Building cuda_grep..") + cmdcd = "cd " + self.thistestpath + ";" + cmd_hipify = "" + cmd_modify = "" + if not os.path.isfile(os.path.join(self.thistestpath, "hipified")): + cmd_hipify = "find . -type f \( -iname \*.h -o -iname \*.cpp -o -iname \*.cu -o -iname \*.cuh \) | sed 's|^./||'\ + | xargs -t -I % sh -c '/opt/rocm/bin/hipify-perl % > hip_%; rm %; mv hip_% %;';" + cmd_modify = "sed -i 's/#include //g' putil.cu; touch hipified;" + cmd_build = "/opt/rocm/bin/hipcc -O3 -m64 pnfa.cu putil.cu nfa.cpp nfautil.cpp regex.cpp -o " + self.binary + ";" + cmdexc = cmdcd + cmd_hipify + cmd_modify + cmd_build + execshellcmd(cmdexc, self.logFile, None) + return True + + def runtest(self): + print("Running cuda_grep..") + cmdexc = "cd " + self.runpath + ";" + "./runtests.sh;" + envtoset = os.environ.copy() + runlogdump = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, runlogdump, envtoset) + runlogdump.close() + + def clean(self): + print("Cleaning cuda_grep..") + cmdcd = "cd " + self.thistestpath + ";" + cmdrm = "rm -f " + self.binary + ";" + cmdrmres = "rm -f " + self.runlog + ";" + cmdexc = cmdcd + cmdrm + cmdrmres + execshellcmd(cmdexc, None, None) + + def parse_result(self): + return CudaGrepParser(self.runlog).parse() diff --git a/src/hiptestsuite/applications/cuda_grep/cuda_grep_build_nvidia.py b/src/hiptestsuite/applications/cuda_grep/cuda_grep_build_nvidia.py new file mode 100644 index 0000000..d1211da --- /dev/null +++ b/src/hiptestsuite/applications/cuda_grep/cuda_grep_build_nvidia.py @@ -0,0 +1,79 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os +import tempfile +from hiptestsuite.common.hip_shell import * +from hiptestsuite.applications.cuda_grep.cuda_grep_parser_common import CudaGrepParser + +class BuildRunNvidia(): + def __init__(self, thistestpath, logFile, binary): + self.thistestpath = thistestpath + self.logFile = logFile + self.runpath = os.path.join(self.thistestpath, "testbench") + self.runlog = os.path.join(self.thistestpath, "testbench/RESULTS") + self.binary = binary + + def getenvironmentvariables(self): + envtoset = os.environ.copy() + envtoset["HIP_PLATFORM"] = "nvidia" + envtoset["HIP_COMPILER"] = "nvcc" + envtoset["HIP_RUNTIME"] = "cuda" + return envtoset + + def buildtest(self): + # In this function put the build steps for test cases + # which differ across platforms (amd/nvidia/intel) + if not os.path.exists("/opt/rocm"): + print("ROCm not installed. Exiting!") + return False + print("Building cuda_grep..") + env = self.getenvironmentvariables() + cmdcd = "cd " + self.thistestpath + ";" + cmd_hipify = "" + cmd_modify = "" + if not os.path.isfile(os.path.join(self.thistestpath, "hipified")): + cmd_hipify = "find . -type f \( -iname \*.h -o -iname \*.cpp -o -iname \*.cu -o -iname \*.cuh \) | sed 's|^./||'\ + | xargs -t -I % sh -c '/opt/rocm/bin/hipify-perl % > hip_%; rm %; mv hip_% %;';" + cmd_modify = "sed -i 's/#include //g' putil.cu; touch hipified;" + cmd_build = "/opt/rocm/bin/hipcc -O3 -m64 pnfa.cu putil.cu nfa.cpp nfautil.cpp regex.cpp -o " + self.binary + ";" + cmdexc = cmdcd + cmd_hipify + cmd_modify + cmd_build + execshellcmd(cmdexc, self.logFile, env) + return True + + def runtest(self): + print("Running cuda_grep..") + env = self.getenvironmentvariables() + cmdexc = "cd " + self.runpath + ";" + "./runtests.sh;" + runlogdump = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, runlogdump, env) + runlogdump.close() + + + def clean(self): + print("Cleaning cuda_grep..") + cmdcd = "cd " + self.thistestpath + ";" + cmdrm = "rm -f " + self.binary + ";" + cmdrmres = "rm -f " + self.runlog + ";" + cmdexc = cmdcd + cmdrm + cmdrmres + execshellcmd(cmdexc, None, None) + + def parse_result(self): + return CudaGrepParser(self.runlog).parse() diff --git a/src/hiptestsuite/applications/cuda_grep/cuda_grep_parser_common.py b/src/hiptestsuite/applications/cuda_grep/cuda_grep_parser_common.py new file mode 100644 index 0000000..ca183f8 --- /dev/null +++ b/src/hiptestsuite/applications/cuda_grep/cuda_grep_parser_common.py @@ -0,0 +1,35 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.common.hip_shell import execshellcmd + +class CudaGrepParser(): + def __init__(self, results): + self.results = results + + def parse(self): + passed = False + count = 0 + with open(self.results, 'r') as testlog: + resulttxt = testlog.read() + count = resulttxt.count("All tests passed") + if count > 0: + passed = True + return passed diff --git a/src/hiptestsuite/applications/cuda_memtest/__init__.py b/src/hiptestsuite/applications/cuda_memtest/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/hiptestsuite/applications/cuda_memtest/cuda_memtest.cu b/src/hiptestsuite/applications/cuda_memtest/cuda_memtest.cu new file mode 100644 index 0000000..576a368 --- /dev/null +++ b/src/hiptestsuite/applications/cuda_memtest/cuda_memtest.cu @@ -0,0 +1,568 @@ +/* + * Illinois Open Source License + * + * University of Illinois/NCSA + * Open Source License + * + * Copyright � 2009, University of Illinois. All rights reserved. + * + * Developed by: + * + * Innovative Systems Lab + * National Center for Supercomputing Applications + * http://www.ncsa.uiuc.edu/AboutUs/Directorates/ISL.html + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal with + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, subject + * to the following conditions: + * + * * Redistributions of source code must retain the above copyright notice, this list + * of conditions and the following disclaimers. + * + * * Redistributions in binary form must reproduce the above copyright notice, this list + * of conditions and the following disclaimers in the documentation and/or other materials + * provided with the distribution. + * + * * Neither the names of the Innovative Systems Lab, the National Center for Supercomputing + * Applications, nor the names of its contributors may be used to endorse or promote products + * derived from this Software without specific prior written permission. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR + * PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE CONTRIBUTORS OR COPYRIGHT HOLDERS BE + * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT + * OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER + * DEALINGS WITH THE SOFTWARE. + */ + +#include "misc.h" +#include +#include +#include +#include +#include +#include +#include + +#define MAX_NUM_GPUS 128 +bool useMappedMemory; +void* mappedHostPtr; +char hostname[64]; +unsigned int verbose =0; +unsigned int interactive =0; +extern cuda_memtest_t cuda_memtests[11]; +unsigned int max_num_blocks = 0; +unsigned int exit_on_error = 0; +unsigned int monitor_temp = 0; +unsigned int monitor_interval = 5; +unsigned int email_notification = 0; +unsigned int global_pattern = 0; +unsigned long global_pattern_long = 0; +char emails[128]; +unsigned int report_interval = 1800; //senconds +unsigned int num_iterations = 1000; +unsigned int num_passes = 0; +unsigned int healthy_threads = 0; +unsigned int disable_serial_number = 0; +__thread unsigned int gpu_idx; +char driver_info[MAX_STR_LEN]; + +pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER; +pthread_mutex_t atomic_mutex = PTHREAD_MUTEX_INITIALIZER; + +void run_tests(char*, unsigned int); +extern void update_temperature(void); +extern void allocate_small_mem(void); +volatile int active_update_temperature; + +typedef struct arg_s{ + unsigned int device; +}arg_t; + +void +display_device_info(struct hipDeviceProp_t* prop) +{ +#if !defined(NVML_DEVICE_SERIAL_BUFFER_SIZE) + char devSerialNum[] = "unknown (no NVML found)"; +#else + char devSerialNum[NVML_DEVICE_SERIAL_BUFFER_SIZE]; + get_serial_number( gpu_idx, devSerialNum ); +#endif + + PRINTF("Device name=%s, global memory size=%zu, serial=%s\n", prop->name, prop->totalGlobalMem, devSerialNum); + return; +} + + +void +atomic_inc(unsigned int* value) +{ + pthread_mutex_lock(&atomic_mutex); + (*value)= (*value) + 1; + pthread_mutex_unlock(&atomic_mutex); +} + +unsigned int +atomic_read(unsigned int* value) +{ + unsigned int ret; + + pthread_mutex_lock(&atomic_mutex); + ret = *value; + pthread_mutex_unlock(&atomic_mutex); + + return ret; +} + +void* +thread_func(void* _arg) +{ + + arg_t* arg = (arg_t*)_arg; + unsigned int device = arg->device; + gpu_idx = device; + + + + hipDeviceProp_t prop; + hipGetDeviceProperties(&prop, device); CUERR; + + display_device_info(&prop); + + unsigned long totmem = prop.totalGlobalMem; + + PRINTF("major=%d, minor=%d\n", prop.major, prop.minor); + + //need to leave a little headroom or later calls will fail + unsigned int tot_num_blocks = totmem/BLOCKSIZE -16; + if (max_num_blocks != 0){ + tot_num_blocks = MIN(max_num_blocks+16, tot_num_blocks); + } + + + hipSetDevice(device); + hipDeviceSynchronize(); + CUERR; + + PRINTF("Attached to device %d successfully.\n", device); + + size_t free, total; + hipMemGetInfo(&free, &total); + + allocate_small_mem(); + + char* ptr = NULL; + + tot_num_blocks = MIN(tot_num_blocks, free/BLOCKSIZE - 16); + do{ + tot_num_blocks -= 16 ; //magic number 16 MB + DEBUG_PRINTF("Trying to allocate %d MB\n", tot_num_blocks); + if (tot_num_blocks <= 0){ + FPRINTF("ERROR: cannot allocate any memory from GPU\n"); + exit(ERR_GENERAL); + } + if(useMappedMemory) + { + //create cuda mapped memory + hipHostAlloc((void**)&mappedHostPtr,tot_num_blocks* BLOCKSIZE,hipHostMallocMapped); + hipHostGetDevicePointer((void **)&ptr,mappedHostPtr,0); + } + else + { + hipMalloc((void**)&ptr, tot_num_blocks* BLOCKSIZE); + } + }while(hipGetLastError() != hipSuccess); + + PRINTF("Allocated %d MB\n", tot_num_blocks); + + atomic_inc(&healthy_threads); + run_tests(ptr, tot_num_blocks); + + return NULL; + +} + + +void* +temp_monitor_thread_func(void*) +{ + do{ + update_temperature(); + sleep(monitor_interval); + }while(active_update_temperature); + return NULL; +} + + +void list_tests_info(void) +{ + size_t i; + for (i = 0;i < DIM(cuda_memtests); i++){ + printf("%s %s\n", cuda_memtests[i].desc, cuda_memtests[i].enabled?"":" ==disabled by default=="); + } + return; +} + + +void +usage(char** argv) +{ + + char example_usage[] = + "run on default setting: ./cuda_memtest\n" + "run on stress test only: ./cuda_memtest --stress\n"; + + printf("Usage:%s [options]\n", argv[0]); + printf("options:\n"); + printf("--mappedMem run all checks with cuda mapped memory instead of native device memory\n"); + printf("--silent Do not print out progress message (default)\n"); + printf("--device Designate one device for test\n"); + printf("--interactive Progress info will be printed in the same line\n"); + printf("--disable_all Disable all tests\n"); + printf("--enable_test Enable the test \n"); + printf("--disable_test Disable the test \n"); + printf("--max_num_blocks Set the maximum of blocks of memory to test\n"); + printf(" 1 block = 1 MB in here\n"); + printf("--exit_on_error When finding error, print error message and exit\n"); + printf("--monitor_temp Monitoring temperature, the temperature will be updated every seconds\n"); + printf(" This feature is experimental\n"); + printf("--emails Setting email notification\n"); + printf("--report_interval Setting the interval in seconds between email notifications(default 1800)\n"); + printf("--pattern Manually set test pattern for test4/test8/test10\n"); + printf("--list_tests List all test descriptions\n"); + printf("--num_iterations Set the number of iterations (only effective on test0 and test10)\n"); + printf("--num_passes Set the number of test passes (this affects all tests)\n"); + printf("--verbose Setting verbose level\n"); + printf(" 0 -- Print out test start and end message only (default)\n"); + printf(" 1 -- Print out pattern messages in test\n"); + printf(" 2 -- Print out progress messages\n"); + printf("--stress Stress test. Equivalent to --disable_all --enable_test 10 --exit_on_error\n"); + printf("--help Print this message\n"); + printf("\nExample usage:\n\n"); + printf("%s\n", example_usage); + + exit(ERR_GENERAL); +} + + +int +main(int argc, char** argv) +{ + int i; + useMappedMemory=false; + mappedHostPtr=NULL; + + if (argc >=2 ){ + if( strcmp(argv[1], "--help")== 0){ + usage(argv); + } + } + + if(gethostname(hostname, 64) !=0){ + fprintf(stderr, "ERROR: gethostname() returns error\n"); + exit(ERR_GENERAL); + } + + for(i=0;i < 64; i++){ + if (hostname[i] == '.'){ + hostname[i] = 0; + break; + } + } + + PRINTF("Running cuda memtest, version %s\n", VERSION); + int device = -1; + int num_gpus; + hipGetDeviceCount(&num_gpus);CUERR; + + if (num_gpus == 0){ + fprintf(stderr,"ERROR: no GPUs found\n"); + exit(ERR_GENERAL); + } + + + for (i =1;i < argc; i++){ + + if( strcmp(argv[i], "--help")== 0){ + usage(argv); + } + + if( strcmp(argv[i], "--mappedMem")== 0){ + useMappedMemory=true; + continue; + } + + if( strcmp(argv[i], "--verbose") == 0){ + if (i+1 >= argc){ + usage(argv); + } + verbose = atoi(argv[i+1]); + i++; + continue; + + } + if (strcmp(argv[i], "--silent") == 0){ + verbose = 0; + continue; + } + if (strcmp(argv[i], "--interactive") == 0){ + interactive = 1; + continue; + } + if (strcmp(argv[i], "--noninteractive") == 0){ + interactive = 0; + continue; + } + if (strcmp(argv[i], "--enable_test") == 0){ + if (i+1 >= argc){ + usage(argv); + } + size_t idx = atoi(argv[i+1]); + if (idx >= DIM(cuda_memtests)){ + fprintf(stderr, "Error: invalid test id\n"); + usage(argv); + } + + cuda_memtests[idx].enabled = 1; + + i++; + continue; + } + if (strcmp(argv[i], "--disable_test") == 0){ + if (i+1 >= argc){ + usage(argv); + } + size_t idx = atoi(argv[i+1]); + if (idx >= DIM(cuda_memtests)){ + fprintf(stderr, "Error: invalid test id\n"); + usage(argv); + } + + cuda_memtests[idx].enabled = 0; + i++; + continue; + } + if (strcmp(argv[i], "--disable_all") == 0){ + size_t k; + for (k=0;k < DIM(cuda_memtests);k++){ + cuda_memtests[k].enabled = 0; + } + continue; + } + + if (strcmp(argv[i], "--device") == 0){ + if (i+1 >= argc){ + usage(argv); + } + device = atoi(argv[i+1]); + i++; + num_gpus = 1; + continue; + } + + if (strcmp(argv[i], "--max_num_blocks") == 0){ + if (i+1 >= argc){ + usage(argv); + } + max_num_blocks = atoi(argv[i+1]); + i++; + continue; + } + + if (strcmp(argv[i], "--exit_on_error") == 0){ + exit_on_error = 1; + continue; + } + + if (strcmp(argv[i], "--monitor_temp") == 0){ + if( ENABLE_NVML != 1 ) + { + printf("ERROR: --monitor_temp requires NVML (not found)\n\n"); + usage(argv); + } + monitor_temp =1; + if (i+1 >= argc){ + usage(argv); + } + monitor_interval = atoi(argv[i+1]); + i++; + continue; + } + if (strcmp(argv[i], "--pattern") == 0){ + if (i+1 >= argc){ + usage(argv); + } + sscanf(argv[i+1], "0x%lx", &global_pattern_long); + if (global_pattern_long ==0){ + printf("ERROR: global test pattern cannot be zero\n"); + usage(argv); + } + printf("Using global test pattern: 0x%lx\n", global_pattern_long); + global_pattern = (unsigned long)global_pattern_long; + i++; + continue; + } + if (strcmp(argv[i], "--emails") == 0){ + email_notification =1; + + struct stat statbuf; + if (stat(MAILFILE, &statbuf)!=0){ + fprintf(stderr, "ERROR: stating mail unitility(%s) failed\n", MAILFILE); + usage(argv); + } + + if( !(S_IXOTH & statbuf.st_mode)){ + fprintf(stderr, "ERROR: no permission on exeution on the mail utility\n"); + usage(argv); + } + + + if (i+1 >= argc){ + usage(argv); + } + if ( strlen( argv[i+1]) > sizeof(emails)){ + fprintf(stderr, "ERROR: email string too long\n"); + usage(argv); + } + strcpy(emails, argv[i+1]); + i++; + continue; + } + if (strcmp(argv[i], "--report_interval") == 0){ + + if (i+1 >= argc){ + usage(argv); + } + report_interval = atoi(argv[i+1]); + i++; + continue; + } + + if (strcmp(argv[i], "--num_iterations") == 0){ + + if (i+1 >= argc){ + usage(argv); + } + num_iterations = atoi(argv[i+1]); + if (num_iterations <= 0){ + printf("ERROR: invalid number of iterations\n"); + usage(argv); + } + i++; + continue; + } + + if (strcmp(argv[i], "--num_passes") == 0){ + + if (i+1 >= argc){ + usage(argv); + } + num_passes = atoi(argv[i+1]); + if (num_passes <= 0){ + printf("ERROR: invalid number of passes\n"); + usage(argv); + } + i++; + continue; + } + + if (strcmp(argv[i], "--disable_serial_number") == 0){ + printf("DEPRECATED: --disable_serial_number is ignored\n"); + continue; + } + + if (strcmp(argv[i], "--stress") == 0){ + //equal to "--disable_all --enable_test 10 --exit_on_error" + size_t k; + for (k=0;k < DIM(cuda_memtests);k++){ + cuda_memtests[k].enabled = 0; + } + cuda_memtests[10].enabled = 1; + exit_on_error = 1; + continue; + } + + if (strcmp(argv[i], "--list_tests") == 0){ + list_tests_info(); + return 0; + } + fprintf(stderr, "ERROR: Invalid option:%s\n", argv[i]); + usage(argv); + } + + get_driver_info(driver_info, MAX_STR_LEN); + + PRINTF("num_gpus=%d\n", num_gpus); + if(num_gpus > MAX_NUM_GPUS){ + fprintf(stderr, "Error: max number of GPUs (%d) exceeded: %d\n", MAX_NUM_GPUS, num_gpus); + exit(ERR_GENERAL); + } + pthread_t temp_pid; + if (monitor_temp){ + active_update_temperature = 1; + if (pthread_create(&temp_pid, NULL, temp_monitor_thread_func, NULL) != 0){ + printf("ERROR: creating thread for temperature monitoring failed\n"); + exit(ERR_GENERAL); + } + } + +#if (ENABLE_NVML==1) + NVML_CHECK(nvmlInit()); +#endif + + arg_t args[MAX_NUM_GPUS]; + pthread_t pid[MAX_NUM_GPUS]; + + if (device != -1){ //device set, only 1 GPU + args[0].device = device; + pthread_create(&pid[0], NULL, thread_func, (void*)&args[0]); + }else{//multiple GPUs + for (i=0;i < num_gpus;i++){ + args[i].device = i; + pthread_create(&pid[i], NULL, thread_func, (void*)&args[i]); + } + + } + + struct timeval t0, t1; + int ht=0; + double wait_time = 500; + gettimeofday(&t0, NULL); + + while(1){ + ht = atomic_read(&healthy_threads); + if (ht == num_gpus){ + break; + } + + gettimeofday(&t1, NULL); + double passed_time = TDIFF(t1, t0); + if (passed_time >= wait_time){ + break; + } + sleep(1); + } + + if (ht < num_gpus){ + printf("ERROR: Some GPU threads are not progressing (healthy_threads=%d, num_gpus=%d)\n", ht, num_gpus); + fflush(stdout); fflush(stderr); + for(i=0;i < num_gpus;i++){ + pthread_kill(pid[i], SIGTERM); + } + exit(ERR_BAD_STATE); + } + + active_update_temperature = 0; + + for(i=0;i < num_gpus;i++){ + pthread_join(pid[i], NULL); + } + + printf("main thread: Program exits\n"); + + return 0; +} + diff --git a/src/hiptestsuite/applications/cuda_memtest/cuda_memtest.py b/src/hiptestsuite/applications/cuda_memtest/cuda_memtest.py new file mode 100644 index 0000000..1b5a556 --- /dev/null +++ b/src/hiptestsuite/applications/cuda_memtest/cuda_memtest.py @@ -0,0 +1,602 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.TesterRepository import Tester, Test, TestData +from hiptestsuite.Test import HIPTestData, TestResult, HIP_PLATFORM +from typing import Union, List +from hiptestsuite.test_classifier import TestClassifier +from hiptestsuite.applications.cuda_memtest.cuda_memtest_build_amd import BuildRunAmd +from hiptestsuite.applications.cuda_memtest.cuda_memtest_build_nvidia import BuildRunNvidia +from hiptestsuite.common.hip_get_packages import HipPackages +from hiptestsuite.common.hip_shell import execshellcmd + +import os +import re +# Common class to clone, set up, build and run test +class PrepareTest(): + def __init__(self, cwd, binary): + self.cwdAbs = cwd + self.binary = binary + self.app_path = os.path.join(self.cwdAbs,\ + "src/hiptestsuite/applications/cuda_memtest/") + self.app_root = os.path.join(self.app_path, "cuda_memtest/") + self.thistestpath = self.app_root + self.prepareobj = None + self.apprepo = "" # Default + self.appbranch = "" + self.appcommitId = "" + + def set_cudamemtest_repoinfo(self, test_data: HIPTestData): + validrepconfig = True + if test_data.repos["cuda_memtest"].repo_url != None: + self.apprepo = test_data.repos["cuda_memtest"].repo_url + else: + validrepconfig &= False + if test_data.repos["cuda_memtest"].branch != None: + self.appbranch = test_data.repos["cuda_memtest"].branch + if test_data.repos["cuda_memtest"].commit_id != None: + self.appcommitId = test_data.repos["cuda_memtest"].commit_id + return validrepconfig + + def downloadtest(self, logFile, test_data: HIPTestData): + ret = HipPackages().pull_repo(logFile, self.apprepo,\ + self.appbranch, self.appcommitId, "cudamemtest") + return ret + + def buildtest(self, logFile, platform): + isBinaryPresent = True + if platform == HIP_PLATFORM.nvidia: + self.prepareobj = BuildRunNvidia(self.thistestpath, logFile, self.binary) + elif platform == HIP_PLATFORM.amd: + self.prepareobj = BuildRunAmd(self.thistestpath, logFile, self.binary) + else: + print("Invalid Platform") + return False + buildstatus = self.prepareobj.buildtest() + if buildstatus == False: + return False + # Check if test binary is created + if not os.path.isfile(\ + os.path.join(self.thistestpath, self.binary)): + isBinaryPresent &= False + return isBinaryPresent + + def clean(self): + if self.prepareobj != None: + self.prepareobj.clean() + + def runtest(self, testnum): + if self.prepareobj != None: + self.prepareobj.runtest(testnum) + + def parse_result(self): + if self.prepareobj != None: + return self.prepareobj.parse_result() + return False + +class CUDA_MEMTEST(TestClassifier): + def __init__(self): + TestClassifier.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + TestClassifier.add_matched_with_names(self, {"cuda_memtest": matched_with_names}) + +class STRESS(CUDA_MEMTEST): + def __init__(self): + CUDA_MEMTEST.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + CUDA_MEMTEST.add_matched_with_names(self, {"stress": matched_with_names}) + + +# Test0 cuda_memtest +class cudamemtest0(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd, "cuda_memtest") + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + classifier = STRESS() + classifier.add_matched_with_names() + test.classifiers = [classifier] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== cudamemtest0 ===============") + # Set repo info + isrepocfgvalid = self.set_cudamemtest_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/cudamemtest0.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(0) + # Parse the test result + if True == self.parse_result(): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + +# Test1 cuda_memtest +class cudamemtest1(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd, "cuda_memtest") + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + classifier = STRESS() + classifier.add_matched_with_names() + test.classifiers = [classifier] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== cudamemtest1 ===============") + # Set repo info + isrepocfgvalid = self.set_cudamemtest_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/cudamemtest1.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(1) + # Parse the test result + if True == self.parse_result(): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + +# Test2 cuda_memtest +class cudamemtest2(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd, "cuda_memtest") + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + classifier = STRESS() + classifier.add_matched_with_names() + test.classifiers = [classifier] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== cudamemtest2 ===============") + # Set repo info + isrepocfgvalid = self.set_cudamemtest_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/cudamemtest2.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(2) + # Parse the test result + if True == self.parse_result(): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + +# Test3 cuda_memtest +class cudamemtest3(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd, "cuda_memtest") + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + classifier = STRESS() + classifier.add_matched_with_names() + test.classifiers = [classifier] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== cudamemtest3 ===============") + # Set repo info + isrepocfgvalid = self.set_cudamemtest_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/cudamemtest3.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(3) + # Parse the test result + if True == self.parse_result(): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + +# Test4 cuda_memtest +class cudamemtest4(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd, "cuda_memtest") + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + classifier = STRESS() + classifier.add_matched_with_names() + test.classifiers = [classifier] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== cudamemtest4 ===============") + # Set repo info + isrepocfgvalid = self.set_cudamemtest_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/cudamemtest4.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(4) + # Parse the test result + if True == self.parse_result(): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test5 cuda_memtest +class cudamemtest5(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd, "cuda_memtest") + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + classifier = STRESS() + classifier.add_matched_with_names() + test.classifiers = [classifier] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== cudamemtest5 ===============") + # Set repo info + isrepocfgvalid = self.set_cudamemtest_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/cudamemtest5.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(5) + # Parse the test result + if True == self.parse_result(): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test6 cuda_memtest +class cudamemtest6(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd, "cuda_memtest") + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + classifier = STRESS() + classifier.add_matched_with_names() + test.classifiers = [classifier] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== cudamemtest6 ===============") + if False: + # Set repo info + isrepocfgvalid = self.set_cudamemtest_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/cudamemtest6.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(6) + # Parse the test result + if True == self.parse_result(): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + else: + test_data.test_result = TestResult.SKIP + +# Test7 cuda_memtest +class cudamemtest7(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd, "cuda_memtest") + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + classifier = STRESS() + classifier.add_matched_with_names() + test.classifiers = [classifier] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== cudamemtest7 ===============") + # Set repo info + isrepocfgvalid = self.set_cudamemtest_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/cudamemtest7.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(7) + # Parse the test result + if True == self.parse_result(): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test8 cuda_memtest +class cudamemtest8(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd, "cuda_memtest") + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + classifier = STRESS() + classifier.add_matched_with_names() + test.classifiers = [classifier] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== cudamemtest8 ===============") + # Set repo info + isrepocfgvalid = self.set_cudamemtest_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/cudamemtest8.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(8) + # Parse the test result + if True == self.parse_result(): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test9 cuda_memtest +class cudamemtest9(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd, "cuda_memtest") + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + classifier = STRESS() + classifier.add_matched_with_names() + test.classifiers = [classifier] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== cudamemtest9 ===============") + if False: + # Set repo info + isrepocfgvalid = self.set_cudamemtest_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/cudamemtest9.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(9) + # Parse the test result + if True == self.parse_result(): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + else: + test_data.test_result = TestResult.SKIP + +# Test10 cuda_memtest +class cudamemtest10(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd, "cuda_memtest") + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + classifier = STRESS() + classifier.add_matched_with_names() + test.classifiers = [classifier] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== cudamemtest10 ===============") + # Set repo info + isrepocfgvalid = self.set_cudamemtest_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/cudamemtest10.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(10) + # Parse the test result + if True == self.parse_result(): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + diff --git a/src/hiptestsuite/applications/cuda_memtest/cuda_memtest_build_amd.py b/src/hiptestsuite/applications/cuda_memtest/cuda_memtest_build_amd.py new file mode 100644 index 0000000..f3c5aa9 --- /dev/null +++ b/src/hiptestsuite/applications/cuda_memtest/cuda_memtest_build_amd.py @@ -0,0 +1,70 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os +import tempfile +from hiptestsuite.common.hip_shell import * +from hiptestsuite.applications.cuda_memtest.cuda_memtest_parser_common import CudaMemtestParser + +class BuildRunAmd(): + def __init__(self, thistestpath, logFile, binary): + self.thistestpath = thistestpath + self.logFile = logFile + self.runlog = None + self.binary = binary + + def buildtest(self): + if not os.path.exists("/opt/rocm"): + print("ROCm not installed. Exiting!") + return False + # In this function put the build steps for test cases + # which differ across platforms (amd/nvidia/intel) + print("Building cuda_memtest..") + cmdcd = "cd " + self.thistestpath + ";" + cmd_hipify = "" + cmd_modify = "" + if not os.path.isfile(os.path.join(self.thistestpath, "hipified")): + cmd_hipify = "ls cuda_memtest.* misc.* tests.cu | xargs -t -I % sh -c '/opt/rocm/bin/hipify-perl % > hip_%; rm %; mv hip_% %;';" + cmd_modify = "cp ../cuda_memtest.cu .;" + cmd_build = "/opt/rocm/bin/hipcc -DENABLE_NVML=0 cuda_memtest.cu misc.cpp tests.cu -o " + self.binary + ";" + cmdexc = cmdcd + cmd_hipify + cmd_modify + cmd_build + execshellcmd(cmdexc, self.logFile, None) + return True + + def runtest(self, testnum): + print("Running cuda_memtest " + str(testnum) + "..") + cmdcd = "cd " + self.thistestpath + ";" + cmdrun = "./" + self.binary + " --disable_all --enable_test " + str(testnum) + " --num_passes 1" + cmdexc = cmdcd + cmdrun + envtoset = os.environ.copy() + self.runlog = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, self.runlog, envtoset) + + def clean(self): + print("Cleaning cuda_memtest..") + cmdcd = "cd " + self.thistestpath + ";" + cmdrm = "rm -f " + self.binary + ";" + cmdexc = cmdcd + cmdrm + if self.runlog != None: + self.runlog.close() + execshellcmd(cmdexc, None, None) + + def parse_result(self): + return CudaMemtestParser(self.runlog).parse() diff --git a/src/hiptestsuite/applications/cuda_memtest/cuda_memtest_build_nvidia.py b/src/hiptestsuite/applications/cuda_memtest/cuda_memtest_build_nvidia.py new file mode 100644 index 0000000..b21f521 --- /dev/null +++ b/src/hiptestsuite/applications/cuda_memtest/cuda_memtest_build_nvidia.py @@ -0,0 +1,78 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os +import tempfile +from hiptestsuite.common.hip_shell import * +from hiptestsuite.applications.cuda_memtest.cuda_memtest_parser_common import CudaMemtestParser + +class BuildRunNvidia(): + def __init__(self, thistestpath, logFile, binary): + self.thistestpath = thistestpath + self.logFile = logFile + self.runlog = None + self.binary = binary + + def getenvironmentvariables(self): + envtoset = os.environ.copy() + envtoset["HIP_PLATFORM"] = "nvidia" + envtoset["HIP_COMPILER"] = "nvcc" + envtoset["HIP_RUNTIME"] = "cuda" + return envtoset + + def buildtest(self): + if not os.path.exists("/opt/rocm"): + print("ROCm not installed. Exiting!") + return False + # In this function put the build steps for test cases + # which differ across platforms (amd/nvidia/intel) + print("Building cuda_memtest..") + env = self.getenvironmentvariables() + cmdcd = "cd " + self.thistestpath + ";" + cmd_hipify = "" + cmd_modify = "" + if not os.path.isfile(os.path.join(self.thistestpath, "hipified")): + cmd_hipify = "ls cuda_memtest.* misc.* tests.cu | xargs -t -I % sh -c '/opt/rocm/bin/hipify-perl % > hip_%; rm %; mv hip_% %;';" + cmd_modify = "cp ../cuda_memtest.cu .;" + cmd_build = "/opt/rocm/bin/hipcc -DENABLE_NVML=0 cuda_memtest.cu misc.cpp tests.cu -o " + self.binary + ";" + cmdexc = cmdcd + cmd_hipify + cmd_modify + cmd_build + execshellcmd(cmdexc, self.logFile, env) + return True + + def runtest(self, testnum): + print("Running cuda_memtest " + str(testnum) + "..") + env = self.getenvironmentvariables() + cmdcd = "cd " + self.thistestpath + ";" + cmdrun = "./" + self.binary + " --disable_all --enable_test " + str(testnum) + " --num_passes 1" + cmdexc = cmdcd + cmdrun + self.runlog = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, self.runlog, env) + + def clean(self): + print("Cleaning cuda_memtest..") + cmdcd = "cd " + self.thistestpath + ";" + cmdrm = "rm -f " + self.binary + ";" + cmdexc = cmdcd + cmdrm + if self.runlog != None: + self.runlog.close() + execshellcmd(cmdexc, None, None) + + def parse_result(self): + return CudaMemtestParser(self.runlog).parse() diff --git a/src/hiptestsuite/applications/cuda_memtest/cuda_memtest_parser_common.py b/src/hiptestsuite/applications/cuda_memtest/cuda_memtest_parser_common.py new file mode 100644 index 0000000..4cf1454 --- /dev/null +++ b/src/hiptestsuite/applications/cuda_memtest/cuda_memtest_parser_common.py @@ -0,0 +1,35 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import re + +class CudaMemtestParser(): + def __init__(self, results): + results.seek(0) + logbytes = results.read() + self.results = logbytes + + def parse(self): + passed = False + statement1lst = re.findall("Attached to device \d+ successfully\.", self.results) + statement2lst = re.findall("Test\d+ finished in \d+\.\d+ seconds", self.results) + if len(statement1lst) == len(statement2lst): + passed = True + return passed diff --git a/src/hiptestsuite/applications/hip_examples/__init__.py b/src/hiptestsuite/applications/hip_examples/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/hiptestsuite/applications/hip_examples/hip_examples.py b/src/hiptestsuite/applications/hip_examples/hip_examples.py new file mode 100644 index 0000000..38ff571 --- /dev/null +++ b/src/hiptestsuite/applications/hip_examples/hip_examples.py @@ -0,0 +1,2070 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.TesterRepository import Tester, Test, TestData +from hiptestsuite.Test import HIPTestData, TestResult, HIP_PLATFORM +from typing import Union, List +from hiptestsuite.test_classifier import TestClassifier +from hiptestsuite.applications.hip_examples.hip_examples_build_amd import BuildRunAmd +from hiptestsuite.applications.hip_examples.hip_examples_build_nvidia import BuildRunNvidia +from hiptestsuite.common.hip_get_packages import HipPackages +from hiptestsuite.common.hip_shell import * + +import os + +# Common class to clone, set up, build and run test +class PrepareTest(): + def __init__(self, path, cwd): + self.cwdAbs = cwd + self.appPath = os.path.join(self.cwdAbs,\ + "src/hiptestsuite/applications/hip_examples/") + self.examplepath = os.path.join(self.appPath, "HIP-Examples/") + self.thistestpath = os.path.join(self.examplepath, path) + self.prepareobj = None + self.apprepo = "" # Default + self.appbranch = "" + self.appcommitId = "" + self.gpustrm_repo = "" + self.gpustrm_branch = "" + self.gpustrm_commitId = "" + self.mixbn_repo = "" + self.mixbn_branch = "" + self.mixbn_commitId = "" + + def set_hipex_repoinfo(self, test_data: HIPTestData): + validrepconfig = True + if test_data.repos["hip_examples"].repo_url != None: + self.apprepo = test_data.repos["hip_examples"].repo_url + else: + validrepconfig &= False + if test_data.repos["hip_examples"].branch != None: + self.appbranch = test_data.repos["hip_examples"].branch + if test_data.repos["hip_examples"].commit_id != None: + self.appcommitId = test_data.repos["hip_examples"].commit_id + + return validrepconfig + + def set_mixben_repoinfo(self, test_data: HIPTestData): + validrepconfig = True + if test_data.repos["mixbench"].repo_url != None: + self.mixbn_repo = test_data.repos["mixbench"].repo_url + else: + validrepconfig &= False + if test_data.repos["mixbench"].branch != None: + self.mixbn_branch = test_data.repos["mixbench"].branch + if test_data.repos["mixbench"].commit_id != None: + self.mixbn_commitId = test_data.repos["mixbench"].commit_id + + return validrepconfig + + def set_gpustr_repoinfo(self, test_data: HIPTestData): + validrepconfig = True + if test_data.repos["gpu_stream"].repo_url != None: + self.gpustrm_repo = test_data.repos["gpu_stream"].repo_url + else: + validrepconfig &= False + if test_data.repos["gpu_stream"].branch != None: + self.gpustrm_branch = test_data.repos["gpu_stream"].branch + if test_data.repos["gpu_stream"].commit_id != None: + self.gpustrm_commitId = test_data.repos["gpu_stream"].commit_id + + return validrepconfig + + def download_hipexample(self, logFile): + ret = HipPackages().pull_repo(logFile, self.apprepo,\ + self.appbranch, self.appcommitId, "hip_examples") + return ret + + def download_gpustream(self, logFile): + ret = HipPackages().pull_repo(logFile, self.gpustrm_repo,\ + self.gpustrm_branch, self.gpustrm_commitId, "gpu-stream") + return ret + + def download_mixbench(self, logFile): + ret = HipPackages().pull_repo(logFile, self.mixbn_repo,\ + self.mixbn_branch, self.mixbn_commitId, "mixbench") + return ret + + def buildtest(self, logFile, platform, testid): + if platform == HIP_PLATFORM.nvidia: + self.prepareobj = BuildRunNvidia(self.thistestpath) + elif platform == HIP_PLATFORM.amd: + self.prepareobj = BuildRunAmd(self.thistestpath) + else: + print("Invalid Platform") + return False + return self.prepareobj.buildtest(logFile, testid) + + def clean(self, testid): + if self.prepareobj != None: + self.prepareobj.clean(testid) + + def runtest(self, logFile, testid): + status = False + if self.prepareobj != None: + status = self.prepareobj.runtest(logFile, testid) + return status + + +class EXAMPLES(TestClassifier): + def __init__(self): + TestClassifier.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + TestClassifier.add_matched_with_names(self, {"examples": matched_with_names}) + + +class MINIAPP(EXAMPLES): + def __init__(self): + EXAMPLES.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + EXAMPLES.add_matched_with_names(self, {"mini-app": matched_with_names}) + + +class STRESS(EXAMPLES): + def __init__(self): + EXAMPLES.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + EXAMPLES.add_matched_with_names(self, {"stress": matched_with_names}) + + +class PERFORMANCE(EXAMPLES): + def __init__(self): + EXAMPLES.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + EXAMPLES.add_matched_with_names(self, {"performance": matched_with_names}) + + +# Test vectorAdd/ +class VectorAdd(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "vectorAdd/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = MINIAPP() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "vectorAdd") + + def test(self, test_data: HIPTestData): + print("=============== vectorAdd test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "vectorAdd" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test gpu-burn/ +class GpuBurn(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "gpu-burn/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = STRESS() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "gpu-burn") + + def test(self, test_data: HIPTestData): + print("=============== gpu-burn test ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "gpu-burn" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test strided-access/ +class StridedAccess(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "strided-access/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = MINIAPP() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "strided-access") + + def test(self, test_data: HIPTestData): + print("=============== strided-access test ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "strided-access" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test rtm8/ +class Rtm8(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "rtm8/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "rtm8") + + def test(self, test_data: HIPTestData): + print("=============== rtm8 test ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "rtm8" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + +# Test reduction/ +class Reduction(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "reduction/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "reduction") + + def test(self, test_data: HIPTestData): + print("=============== reduction test ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "reduction" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test mini-nbody/ +class Mini_nbody(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "mini-nbody/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "mini-nbody") + + def test(self, test_data: HIPTestData): + print("=============== mini-nbody test ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "mini-nbody" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test add4/ +class Add4(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "add4/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "add4") + + def test(self, test_data: HIPTestData): + print("=============== add4 test ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "add4" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test cuda-stream/ +class Cuda_stream(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "cuda-stream/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "cuda-stream") + + def test(self, test_data: HIPTestData): + print("=============== cuda-stream test ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "cuda-stream" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test rodinia_3.0/hip/bfs/ +class Bfs(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "rodinia_3.0/hip/bfs/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "rodinia_3.bfs") + + def test(self, test_data: HIPTestData): + print("=============== rodinia_3.0/hip/bfs test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "rodinia_3.bfs" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test rodinia_3.0/hip/cfd/ +class Cfd(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "rodinia_3.0/hip/cfd/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "rodinia_3.cfd") + + def test(self, test_data: HIPTestData): + print("=============== rodinia_3.0/hip/cfd test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "rodinia_3.cfd" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test rodinia_3.0/hip/dwt2d/ +# This test is skipped for nvidia +class Dwt2d(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "rodinia_3.0/hip/dwt2d/", self.cwd) + self.platform = None + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + if self.platform != HIP_PLATFORM.nvidia: + PrepareTest.clean(self, "rodinia_3.dwt2d") + + def test(self, test_data: HIPTestData): + print("=============== rodinia_3.0/hip/dwt2d test start ===============") + self.platform = test_data.HIP_PLATFORM + if test_data.HIP_PLATFORM == HIP_PLATFORM.nvidia: + test_data.test_result = TestResult.SKIP + else: + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "rodinia_3.dwt2d" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test rodinia_3.0/hip/gaussian/ +class Gaussian(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "rodinia_3.0/hip/gaussian/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "rodinia_3.gaussian") + + def test(self, test_data: HIPTestData): + print("=============== rodinia_3.0/hip/gaussian test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "rodinia_3.gaussian" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test rodinia_3.0/hip/heartwall/ +class Heartwall(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "rodinia_3.0/hip/heartwall/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "rodinia_3.heartwall") + + def test(self, test_data: HIPTestData): + print("=============== rodinia_3.0/hip/heartwall test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "rodinia_3.heartwall" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test rodinia_3.0/hip/hotspot/ +class Hotspot(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "rodinia_3.0/hip/hotspot/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "rodinia_3.hotspot") + + def test(self, test_data: HIPTestData): + print("=============== rodinia_3.0/hip/hotspot test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "rodinia_3.hotspot" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test rodinia_3.0/hip/hybridsort/ +class Hybridsort(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "rodinia_3.0/hip/hybridsort/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "rodinia_3.hybridsort") + + def test(self, test_data: HIPTestData): + print("=============== rodinia_3.0/hip/hybridsort test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "rodinia_3.hybridsort" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test rodinia_3.0/hip/kmeans/ +class Kmeans(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "rodinia_3.0/hip/kmeans/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "rodinia_3.kmeans") + + def test(self, test_data: HIPTestData): + print("=============== rodinia_3.0/hip/kmeans test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "rodinia_3.kmeans" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test rodinia_3.0/hip/lavaMD/ +class LavaMD(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "rodinia_3.0/hip/lavaMD/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "rodinia_3.lavaMD") + + def test(self, test_data: HIPTestData): + print("=============== rodinia_3.0/hip/lavaMD test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "rodinia_3.lavaMD" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test rodinia_3.0/hip/lud/ +class Lud(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "rodinia_3.0/hip/lud/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "rodinia_3.lud") + + def test(self, test_data: HIPTestData): + print("=============== rodinia_3.0/hip/lud test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "rodinia_3.lud" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test rodinia_3.0/hip/myocyte/ +class Myocyte(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "rodinia_3.0/hip/myocyte/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "rodinia_3.myocyte") + + def test(self, test_data: HIPTestData): + print("=============== rodinia_3.0/hip/myocyte test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "rodinia_3.myocyte" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test rodinia_3.0/hip/nn/ +class Nn(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "rodinia_3.0/hip/nn/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "rodinia_3.nn") + + def test(self, test_data: HIPTestData): + print("=============== rodinia_3.0/hip/nn test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "rodinia_3.nn" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test rodinia_3.0/hip/nw/ +class Nw(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "rodinia_3.0/hip/nw/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "rodinia_3.nw") + + def test(self, test_data: HIPTestData): + print("=============== rodinia_3.0/hip/nw test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "rodinia_3.nw" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test rodinia_3.0/hip/particlefilter/ +# This test is skipped for amd. +class Particlefilter(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "rodinia_3.0/hip/particlefilter/", self.cwd) + self.platform = None + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + if self.platform != HIP_PLATFORM.amd: + PrepareTest.clean(self, "rodinia_3.particlefilter") + + def test(self, test_data: HIPTestData): + print("=============== rodinia_3.0/hip/particlefilter test start ===============") + self.platform = test_data.HIP_PLATFORM + if test_data.HIP_PLATFORM == HIP_PLATFORM.amd: + test_data.test_result = TestResult.SKIP + else: + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "rodinia_3.particlefilter" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test rodinia_3.0/hip/pathfinder/ +class pathfinder(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "rodinia_3.0/hip/pathfinder/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "rodinia_3.pathfinder") + + def test(self, test_data: HIPTestData): + print("=============== rodinia_3.0/hip/pathfinder test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "rodinia_3.pathfinder" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test rodinia_3.0/hip/srad/ +class Srad(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "rodinia_3.0/hip/srad/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "rodinia_3.srad") + + def test(self, test_data: HIPTestData): + print("=============== rodinia_3.0/hip/srad test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "rodinia_3.srad" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test rodinia_3.0/hip/streamcluster/ +class Streamcluster(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "rodinia_3.0/hip/streamcluster/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "rodinia_3.streamcluster") + + def test(self, test_data: HIPTestData): + print("=============== rodinia_3.0/hip/streamcluster test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "rodinia_3.streamcluster" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test rodinia_3.0/hip/backprop/ +# This test is skipped for amd +class Backprop(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "rodinia_3.0/hip/backprop/", self.cwd) + self.platform = None + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + if self.platform != HIP_PLATFORM.amd: + PrepareTest.clean(self, "rodinia_3.backprop") + + def test(self, test_data: HIPTestData): + print("=============== rodinia_3.0/hip/backprop test start ===============") + self.platform = test_data.HIP_PLATFORM + if test_data.HIP_PLATFORM == HIP_PLATFORM.amd: + test_data.test_result = TestResult.SKIP + else: + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "rodinia_3.backprop" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test HIP-Examples-Applications/BinomialOption/ +class BinomialOption(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "HIP-Examples-Applications/BinomialOption/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = MINIAPP() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "HIP-Examples-Applications.BinomialOption") + + def test(self, test_data: HIPTestData): + print("=============== HIP-Examples-Applications/BinomialOption test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "HIP-Examples-Applications.BinomialOption" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test HIP-Examples-Applications/BitonicSort/ +class BitonicSort(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "HIP-Examples-Applications/BitonicSort/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = MINIAPP() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "HIP-Examples-Applications.BitonicSort") + + def test(self, test_data: HIPTestData): + print("=============== HIP-Examples-Applications/BitonicSort test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "HIP-Examples-Applications.BitonicSort" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test HIP-Examples-Applications/dct/ +class Dct(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "HIP-Examples-Applications/dct/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = MINIAPP() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "HIP-Examples-Applications.dct") + + def test(self, test_data: HIPTestData): + print("=============== HIP-Examples-Applications/dct test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "HIP-Examples-Applications.dct" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test HIP-Examples-Applications/dwtHaar1D/ +class DwtHaar1D(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "HIP-Examples-Applications/dwtHaar1D/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = MINIAPP() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "HIP-Examples-Applications.dwtHaar1D") + + def test(self, test_data: HIPTestData): + print("=============== HIP-Examples-Applications/dwtHaar1D test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "HIP-Examples-Applications.dwtHaar1D" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test HIP-Examples-Applications/FastWalshTransform/ +class FastWalshTransform(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "HIP-Examples-Applications/FastWalshTransform/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = MINIAPP() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "HIP-Examples-Applications.FastWalshTransform") + + def test(self, test_data: HIPTestData): + print("=============== HIP-Examples-Applications/FastWalshTransform test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "HIP-Examples-Applications.FastWalshTransform" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test HIP-Examples-Applications/FloydWarshall/ +class FloydWarshall(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "HIP-Examples-Applications/FloydWarshall/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = MINIAPP() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "HIP-Examples-Applications.FloydWarshall") + + def test(self, test_data: HIPTestData): + print("=============== HIP-Examples-Applications/FloydWarshall test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "HIP-Examples-Applications.FloydWarshall" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test HIP-Examples-Applications/HelloWorld/ +class HelloWorld(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "HIP-Examples-Applications/HelloWorld/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = MINIAPP() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "HIP-Examples-Applications.HelloWorld") + + def test(self, test_data: HIPTestData): + print("=============== HIP-Examples-Applications/HelloWorld test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "HIP-Examples-Applications.HelloWorld" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test HIP-Examples-Applications/Histogram/ +class Histogram(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "HIP-Examples-Applications/Histogram/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = MINIAPP() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "HIP-Examples-Applications.Histogram") + + def test(self, test_data: HIPTestData): + print("=============== HIP-Examples-Applications/Histogram test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "HIP-Examples-Applications.Histogram" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test HIP-Examples-Applications/MatrixMultiplication/ +class MatrixMultiplication(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "HIP-Examples-Applications/MatrixMultiplication/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = MINIAPP() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "HIP-Examples-Applications.MatrixMultiplication") + + def test(self, test_data: HIPTestData): + print("=============== HIP-Examples-Applications/MatrixMultiplication test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "HIP-Examples-Applications.MatrixMultiplication" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test HIP-Examples-Applications/PrefixSum/ +class PrefixSum(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "HIP-Examples-Applications/PrefixSum/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = MINIAPP() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "HIP-Examples-Applications.PrefixSum") + + def test(self, test_data: HIPTestData): + print("=============== HIP-Examples-Applications/PrefixSum test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "HIP-Examples-Applications.PrefixSum" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test HIP-Examples-Applications/RecursiveGaussian/ +class RecursiveGaussian(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "HIP-Examples-Applications/RecursiveGaussian/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = MINIAPP() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "HIP-Examples-Applications.RecursiveGaussian") + + def test(self, test_data: HIPTestData): + print("=============== HIP-Examples-Applications/RecursiveGaussian test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "HIP-Examples-Applications.RecursiveGaussian" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test HIP-Examples-Applications/SimpleConvolution/ +class SimpleConvolution(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "HIP-Examples-Applications/SimpleConvolution/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = MINIAPP() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "HIP-Examples-Applications.SimpleConvolution") + + def test(self, test_data: HIPTestData): + print("=============== HIP-Examples-Applications/SimpleConvolution test start ===============") + # Set repo info + isrepocfgvalid = self.set_hipex_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "HIP-Examples-Applications.SimpleConvolution" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_hipexample(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test GPU-STREAM Double +class GpuStreamDouble(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "../GPU-STREAM/", self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self, "GPU-STREAM-DOUBLE") + + def test(self, test_data: HIPTestData): + print("=============== GPU-STREAM Double start ===============") + # Set repo info + isrepocfgvalid = self.set_gpustr_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "GPU-STREAM-DOUBLE" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_gpustream(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test mixbench-hip-alt +# This test is skipped for nvidia +class MixBenchAlt(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "../mixbench/mixbench-hip/", self.cwd) + self.platform = None + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + if self.platform != HIP_PLATFORM.nvidia: + PrepareTest.clean(self, "mixbench-hip-alt") + + def test(self, test_data: HIPTestData): + print("=============== mixbench-hip-alt start ===============") + self.platform = test_data.HIP_PLATFORM + if test_data.HIP_PLATFORM == HIP_PLATFORM.nvidia: + test_data.test_result = TestResult.SKIP + else: + # Set repo info + isrepocfgvalid = self.set_mixben_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "mixbench-hip-alt" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_mixbench(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test mixbench-hip-ro +# This test is skipped for nvidia +class MixBenchRO(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "../mixbench/mixbench-hip/", self.cwd) + self.platform = None + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = PERFORMANCE() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + if self.platform != HIP_PLATFORM.nvidia: + PrepareTest.clean(self, "mixbench-hip-ro") + + def test(self, test_data: HIPTestData): + print("=============== GPU-STREAM Float start ===============") + self.platform = test_data.HIP_PLATFORM + if test_data.HIP_PLATFORM == HIP_PLATFORM.nvidia: + test_data.test_result = TestResult.SKIP + else: + # Set repo info + isrepocfgvalid = self.set_mixben_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + testid = "mixbench-hip-ro" + with open(resultLogDir + "/" + testid + ".log", 'w+') as testLogger: + res = self.download_mixbench(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, testid) + if not res: + test_data.test_result = TestResult.FAIL + return + + if True == self.runtest(testLogger, testid): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + diff --git a/src/hiptestsuite/applications/hip_examples/hip_examples_build_amd.py b/src/hiptestsuite/applications/hip_examples/hip_examples_build_amd.py new file mode 100644 index 0000000..149fd05 --- /dev/null +++ b/src/hiptestsuite/applications/hip_examples/hip_examples_build_amd.py @@ -0,0 +1,50 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os +from hiptestsuite.applications.hip_examples.hip_examples_parser import Hip_examples_parser +from hiptestsuite.applications.hip_examples.hip_examples_build_common import BuildRunCommon +from hiptestsuite.common.hip_shell import * + +class BuildRunAmd(BuildRunCommon): + def __init__(self, path): + BuildRunCommon.__init__(self, path) + + def buildtest(self, logFile, testid): + # In this function put the build steps for test cases + # which differ across platforms (amd/nvidia/intel) else + # invoke BuildRunCommon.buildtest + if not os.path.exists("/opt/rocm"): + print("ROCm not installed. Exiting!") + return False + ret = BuildRunCommon.buildtest(self, logFile, testid) + return ret + + def runtest(self, logFile, testid): + # In this function put the execution steps for test cases + # which differ across platforms (amd/nvidia/intel) else + # invoke BuildRunCommon.buildtest + envtoset = os.environ.copy() + ret = BuildRunCommon.runtest(self, logFile, testid, envtoset) + return ret + + # Clean all generated binaries + def clean(self, testid): + BuildRunCommon.clean(self, testid) diff --git a/src/hiptestsuite/applications/hip_examples/hip_examples_build_common.py b/src/hiptestsuite/applications/hip_examples/hip_examples_build_common.py new file mode 100644 index 0000000..becc790 --- /dev/null +++ b/src/hiptestsuite/applications/hip_examples/hip_examples_build_common.py @@ -0,0 +1,343 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os +import tempfile + +from hiptestsuite.applications.hip_examples.hip_examples_parser import Hip_examples_parser +from hiptestsuite.common.hip_shell import * + +class BuildRunCommon(): + ''' + In this class insert the build and execution steps for test cases + which are identical across different platforms (amd/nvidia/intel). + ''' + def __init__(self, path): + self.thistestpath = path + self.runlogdump = tempfile.TemporaryFile("w+") + self.genbinaryname = None + self.binarydic = {"vectorAdd":["vectoradd_hip.exe"],\ + "gpu-burn":["/build/gpuburn-hip"],\ + "strided-access":["strided-access"],\ + "rtm8":["rtm8_hip"],\ + "reduction":["reduction"],\ + "mini-nbody":["/hip/nbody-block", "/hip/nbody-orig", "/hip/nbody-soa"],\ + "add4":["gpu-stream-hip"],\ + "cuda-stream":["stream"],\ + "openmp-helloworld":["openmp_helloworld.exe"],\ + "rodinia_3.bfs":["bfs"],\ + "rodinia_3.cfd":["euler3d"],\ + "rodinia_3.dwt2d":["dwt2d"],\ + "rodinia_3.gaussian":["gaussian"],\ + "rodinia_3.heartwall":["heartwall"],\ + "rodinia_3.hotspot":["hotspot"],\ + "rodinia_3.hybridsort":["hybridsort"],\ + "rodinia_3.kmeans":["kmeans"],\ + "rodinia_3.lavaMD":["lavaMD"],\ + "rodinia_3.lud":["lud"],\ + "rodinia_3.myocyte":["myocyte"],\ + "rodinia_3.nn":["nn"],\ + "rodinia_3.nw":["nw"],\ + "rodinia_3.particlefilter":["particlefilter_naive","particlefilter_float"],\ + "rodinia_3.pathfinder":["pathfinder"],\ + "rodinia_3.srad":["/srad_v1/srad"],\ + "rodinia_3.streamcluster":["streamcluster"],\ + "rodinia_3.b+tree":["b+tree"],\ + "rodinia_3.backprop":["backprop"],\ + "HIP-Examples-Applications.BinomialOption":["BinomialOption"],\ + "HIP-Examples-Applications.BitonicSort":["BitonicSort"],\ + "HIP-Examples-Applications.dct":["dct"],\ + "HIP-Examples-Applications.dwtHaar1D":["dwtHaar1D"],\ + "HIP-Examples-Applications.FastWalshTransform":["FastWalshTransform"],\ + "HIP-Examples-Applications.FloydWarshall":["FloydWarshall"],\ + "HIP-Examples-Applications.HelloWorld":["HelloWorld"],\ + "HIP-Examples-Applications.Histogram":["Histogram"],\ + "HIP-Examples-Applications.MatrixMultiplication":["MatrixMultiplication"],\ + "HIP-Examples-Applications.PrefixSum":["PrefixSum"],\ + "HIP-Examples-Applications.RecursiveGaussian":["RecursiveGaussian"],\ + "HIP-Examples-Applications.SimpleConvolution":["SimpleConvolution"],\ + "GPU-STREAM-DOUBLE":["hip-stream"],\ + "GPU-STREAM-FLOAT":["hip-stream"],\ + "mixbench-hip-alt":["mixbench-hip-alt", "mixbench-hip-ro"],\ + "mixbench-hip-ro":["mixbench-hip-alt", "mixbench-hip-ro"] + } + + def buildtest(self, logFile, testid, env = None): + # Prepare the shell command to execute + cmdcd = "cd "+self.thistestpath+";" + cmd_build = "" + if testid == "vectorAdd" or\ + testid == "gpu-burn" or\ + testid == "strided-access" or\ + testid == "cuda-stream" or\ + testid == "rodinia_3.bfs" or\ + testid == "rodinia_3.cfd" or\ + testid == "rodinia_3.dwt2d" or\ + testid == "rodinia_3.gaussian" or\ + testid == "rodinia_3.heartwall" or\ + testid == "rodinia_3.hotspot" or\ + testid == "rodinia_3.hybridsort" or\ + testid == "rodinia_3.kmeans" or\ + testid == "rodinia_3.lavaMD" or\ + testid == "rodinia_3.lud" or\ + testid == "rodinia_3.myocyte" or\ + testid == "rodinia_3.nn" or\ + testid == "rodinia_3.nw" or\ + testid == "rodinia_3.particlefilter" or\ + testid == "rodinia_3.pathfinder" or\ + testid == "rodinia_3.srad" or\ + testid == "rodinia_3.streamcluster" or\ + testid == "rodinia_3.b+tree" or\ + testid == "rodinia_3.backprop" or\ + testid == "HIP-Examples-Applications.BinomialOption" or\ + testid == "HIP-Examples-Applications.BitonicSort" or\ + testid == "HIP-Examples-Applications.dct" or\ + testid == "HIP-Examples-Applications.dwtHaar1D" or\ + testid == "HIP-Examples-Applications.FastWalshTransform" or\ + testid == "HIP-Examples-Applications.FloydWarshall" or\ + testid == "HIP-Examples-Applications.HelloWorld" or\ + testid == "HIP-Examples-Applications.Histogram" or\ + testid == "HIP-Examples-Applications.MatrixMultiplication" or\ + testid == "HIP-Examples-Applications.PrefixSum" or\ + testid == "HIP-Examples-Applications.RecursiveGaussian" or\ + testid == "HIP-Examples-Applications.SimpleConvolution": + cmd_build = "make clean;make;" + elif testid == "rtm8": + cmd_build = "./build_hip.sh;" + elif testid == "reduction": + cmd_build = "make clean;make;" + elif testid == "mini-nbody": + cmd_build = "cd hip;\ + bash ./HIP-nbody-block.sh;\ + bash ./HIP-nbody-orig.sh;\ + bash ./HIP-nbody-soa.sh;" + elif testid == "add4": + cmd_build = "make clean;./buildit.sh;" + elif testid == "openmp-helloworld": + cmd_build = "cd openmp-helloworld;make clean;make;" + elif testid == "GPU-STREAM-DOUBLE" or testid == "GPU-STREAM-FLOAT": + cmd_build = "make -f HIP.make clean;make -f HIP.make;" + elif testid == "mixbench-hip-alt" or testid == "mixbench-hip-ro": + cmd_build = "make clean;rm -Rf Makefile;cmake .;make;" + + cmdexc = cmdcd + cmd_build + # Execute the command on shell + execshellcmd_largedump(cmdexc, logFile, self.runlogdump, env) + # Check if the test binary/ies is/are generated + for binary in self.binarydic[testid]: + if not os.path.isfile(self.thistestpath + binary): + return False + + return True + + def runtest(self, logFile, testid, env = None): + res = True + if testid == "vectorAdd": + # Test already executed during make + # Run Parser + self.runlogdump.seek(0) + logbytes = self.runlogdump.read() + ret = Hip_examples_parser().vectorAdd(logbytes) + if ret == "Failed": + res &= False + self.runlogdump.close() + elif testid == "gpu-burn": + cmdexc = "cd " + self.thistestpath + ";" + "." +\ + self.binarydic[testid][0] + " -t 5" + execshellcmd_largedump(cmdexc, logFile, self.runlogdump, env) + self.runlogdump.seek(0) + logbytes = self.runlogdump.read() + ret = Hip_examples_parser().gpu_burn(logbytes) + if ret == "Failed": + res &= False + self.runlogdump.close() + elif testid == "strided-access": + cmdexc = "cd " + self.thistestpath + ";" + "./" +\ + self.binarydic[testid][0] + execshellcmd_largedump(cmdexc, logFile, self.runlogdump, env) + self.runlogdump.seek(0) + logbytes = self.runlogdump.read() + ret = Hip_examples_parser().strided_access(logbytes) + if ret == "Failed": + res &= False + self.runlogdump.close() + elif testid == "rtm8": + cmdexc = "cd " + self.thistestpath + ";" + "./" +\ + self.binarydic[testid][0] + execshellcmd_largedump(cmdexc, logFile, self.runlogdump, env) + self.runlogdump.seek(0) + logbytes = self.runlogdump.read() + ret = Hip_examples_parser().rtm8(logbytes) + if ret == "Failed": + res &= False + self.runlogdump.close() + elif testid == "reduction": + cmdexc = "cd " + self.thistestpath + ";" +\ + "bash ./run.sh" + execshellcmd_largedump(cmdexc, logFile, self.runlogdump, env) + self.runlogdump.seek(0) + logbytes = self.runlogdump.read() + ret = Hip_examples_parser().reduction(logbytes) + if ret == "Failed": + res &= False + self.runlogdump.close() + elif testid == "mini-nbody": + # Test already executed during make + # Run Parser + self.runlogdump.seek(0) + logbytes = self.runlogdump.read() + ret = Hip_examples_parser().mini_nbody(logbytes) + if ret == "Failed": + res &= False + self.runlogdump.close() + elif testid == "add4": + cmdexc = "cd " + self.thistestpath + ";" +\ + "./runhip.sh" + execshellcmd_largedump(cmdexc, logFile, self.runlogdump, env) + self.runlogdump.seek(0) + logbytes = self.runlogdump.read() + ret = Hip_examples_parser().add4(logbytes) + if ret == "Failed": + res &= False + self.runlogdump.close() + elif testid == "cuda-stream": + cmdexc = "cd " + self.thistestpath + ";" + "./" +\ + self.binarydic[testid][0] + execshellcmd_largedump(cmdexc, logFile, self.runlogdump, env) + self.runlogdump.seek(0) + logbytes = self.runlogdump.read() + ret = Hip_examples_parser().cuda_stream(logbytes) + if ret == "Failed": + res &= False + self.runlogdump.close() + elif testid == "openmp-helloworld": + # Test already executed during make + # Run Parser + self.runlogdump.seek(0) + logbytes = self.runlogdump.read() + ret = Hip_examples_parser().openmp_helloworld(logbytes) + if ret == "Failed": + res &= False + self.runlogdump.close() + elif testid == "rodinia_3.bfs" or testid == "rodinia_3.cfd" or \ + testid == "rodinia_3.dwt2d" or testid == "rodinia_3.particlefilter": + cmdexc = "cd " + self.thistestpath + ";" + "make test;" + execshellcmd_largedump(cmdexc, logFile, self.runlogdump, env) + self.runlogdump.seek(0) + logbytes = self.runlogdump.read() + ret = Hip_examples_parser().rodina3(logbytes, "PASSED", 2) + if ret == "Failed": + res &= False + self.runlogdump.close() + elif testid == "rodinia_3.gaussian" or testid == "rodinia_3.lavaMD": + cmdexc = "cd " + self.thistestpath + ";" + "make test;" + execshellcmd_largedump(cmdexc, logFile, self.runlogdump, env) + self.runlogdump.seek(0) + logbytes = self.runlogdump.read() + ret = Hip_examples_parser().rodina3(logbytes, "PASSED", 5) + if ret == "Failed": + res &= False + self.runlogdump.close() + elif testid == "rodinia_3.heartwall" or testid == "rodinia_3.hotspot"\ + or testid == "rodinia_3.hybridsort" or testid == "rodinia_3.lud"\ + or testid == "rodinia_3.myocyte" or testid == "rodinia_3.nn"\ + or testid == "rodinia_3.nn" or testid == "rodinia_3.pathfinder"\ + or testid == "rodinia_3.srad" or testid == "rodinia_3.streamcluster"\ + or testid == "rodinia_3.b+tree" or testid == "rodinia_3.backprop": + cmdexc = "cd " + self.thistestpath + ";" + "make test;" + execshellcmd_largedump(cmdexc, logFile, self.runlogdump, env) + self.runlogdump.seek(0) + logbytes = self.runlogdump.read() + ret = Hip_examples_parser().rodina3(logbytes, "PASSED", 1) + if ret == "Failed": + res &= False + self.runlogdump.close() + elif testid == "rodinia_3.kmeans": + cmdexc = "cd " + self.thistestpath + ";" + "make test;" + execshellcmd_largedump(cmdexc, logFile, self.runlogdump, env) + self.runlogdump.seek(0) + logbytes = self.runlogdump.read() + ret = Hip_examples_parser().rodina3(logbytes, "PASSED", 4) + if ret == "Failed": + res &= False + self.runlogdump.close() + elif testid == "HIP-Examples-Applications.BinomialOption" or\ + testid == "HIP-Examples-Applications.BitonicSort" or\ + testid == "HIP-Examples-Applications.dct" or\ + testid == "HIP-Examples-Applications.dwtHaar1D" or\ + testid == "HIP-Examples-Applications.FastWalshTransform" or\ + testid == "HIP-Examples-Applications.FloydWarshall" or\ + testid == "HIP-Examples-Applications.HelloWorld" or\ + testid == "HIP-Examples-Applications.Histogram" or\ + testid == "HIP-Examples-Applications.MatrixMultiplication" or\ + testid == "HIP-Examples-Applications.PrefixSum" or\ + testid == "HIP-Examples-Applications.RecursiveGaussian" or\ + testid == "HIP-Examples-Applications.SimpleConvolution": + # Test already executed during make + # Run Parser + ret = Hip_examples_parser().hip_examples_applications(self.runlogdump) + if ret == "Failed": + res &= False + elif testid == "GPU-STREAM-DOUBLE": + cmdexc = "cd " + self.thistestpath + ";" +\ + "./hip-stream" + execshellcmd_largedump(cmdexc, logFile, self.runlogdump, env) + self.runlogdump.seek(0) + logbytes = self.runlogdump.read() + ret = Hip_examples_parser().gpu_stream(logbytes) + if ret == "Failed": + res &= False + self.runlogdump.close() + elif testid == "GPU-STREAM-FLOAT": + cmdexc = "cd " + self.thistestpath + ";" +\ + "./hip-stream --float" + execshellcmd_largedump(cmdexc, logFile, self.runlogdump, env) + self.runlogdump.seek(0) + logbytes = self.runlogdump.read() + ret = Hip_examples_parser().gpu_stream(logbytes) + if ret == "Failed": + res &= False + self.runlogdump.close() + elif testid == "mixbench-hip-alt": + cmdexc = "cd " + self.thistestpath + ";" +\ + "./mixbench-hip-alt" + execshellcmd_largedump(cmdexc, logFile, self.runlogdump, env) + self.runlogdump.seek(0) + logbytes = self.runlogdump.read() + ret = Hip_examples_parser().mix_bench(logbytes) + if ret == "Failed": + res &= False + self.runlogdump.close() + elif testid == "mixbench-hip-ro": + cmdexc = "cd " + self.thistestpath + ";" +\ + "./mixbench-hip-ro" + execshellcmd_largedump(cmdexc, logFile, self.runlogdump, env) + self.runlogdump.seek(0) + logbytes = self.runlogdump.read() + ret = Hip_examples_parser().mix_bench(logbytes) + if ret == "Failed": + res &= False + self.runlogdump.close() + return res + + def clean(self, testid): + for binary in self.binarydic[testid]: + if os.path.exists(self.thistestpath + binary): + os.remove(self.thistestpath + binary) diff --git a/src/hiptestsuite/applications/hip_examples/hip_examples_build_nvidia.py b/src/hiptestsuite/applications/hip_examples/hip_examples_build_nvidia.py new file mode 100644 index 0000000..a1f85c1 --- /dev/null +++ b/src/hiptestsuite/applications/hip_examples/hip_examples_build_nvidia.py @@ -0,0 +1,61 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os + +from hiptestsuite.applications.hip_examples.hip_examples_parser import Hip_examples_parser +from hiptestsuite.applications.hip_examples.hip_examples_build_common import BuildRunCommon +from hiptestsuite.common.hip_shell import * + +class BuildRunNvidia(BuildRunCommon): + def __init__(self, path): + self.hippath = os.path.join(os.getcwd(),"src/hiptestsuite/conformance/HIP/") + BuildRunCommon.__init__(self, path) + + def getenvironmentvariables(self): + envtoset = os.environ.copy() + envtoset["HIP_PLATFORM"] = "nvidia" + envtoset["HIP_COMPILER"] = "nvcc" + envtoset["HIP_RUNTIME"] = "cuda" + envtoset["HIP_PATH"] = "/opt/rocm/hip" + return envtoset + + def buildtest(self, logFile, testid): + env = self.getenvironmentvariables() + # In this function put the build steps for test cases + # which differ across platforms (amd/nvidia/intel) else + # invoke BuildRunCommon.buildtest + if not os.path.exists("/opt/rocm"): + print("ROCm not installed. Exiting!") + return False + ret = BuildRunCommon.buildtest(self, logFile, testid, env) + return ret + + def runtest(self, logFile, testid): + env = self.getenvironmentvariables() + # In this function put the execution steps for test cases + # which differ across platforms (amd/nvidia/intel) else + # invoke BuildRunCommon.buildtest + ret = BuildRunCommon.runtest(self, logFile, testid, env) + return ret + + # Clean all generated binaries + def clean(self, testid): + BuildRunCommon.clean(self, testid) diff --git a/src/hiptestsuite/applications/hip_examples/hip_examples_parser.py b/src/hiptestsuite/applications/hip_examples/hip_examples_parser.py new file mode 100644 index 0000000..bfe8e04 --- /dev/null +++ b/src/hiptestsuite/applications/hip_examples/hip_examples_parser.py @@ -0,0 +1,267 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import re + +class Hip_examples_parser: + def __init__(self): + pass + + def rodina3(self, text, pass_string, num): + if num == text.count(pass_string): + return "Passed" + return "Failed" + + def openmp_helloworld(self, text): + status = 'Failed' + text = text.splitlines() + for line in text: + if 'PASSED!' in line: + status = 'Passed' + return status + + def vectorAdd(self, text): + status = 'Failed' + text = text.splitlines() + for line in text: + if 'PASSED!' in line: + status = 'Passed' + return status + + def reduction(self, text): + status = 'Failed' + count = 0 + + text = text.splitlines() + try: + + for line in text: + if 'result is CORRECT' in line: + count = count + 1 + if count >= 8: + status = 'Passed' + except: + pass + return status + + def rtm8(self, text): + status = 'Failed' + count = 0 + text = text.splitlines() + try: + for line in text: + if "memory" in line: + count = count + 1 + if "pts" in line: + count = count + 1 + if "Tflops" in line: + count = count + 1 + if "dt" in line: + count = count + 1 + if "pt_rate" in line: + count = count + 1 + if "flop_rate" in line: + count = count + 1 + if "speedup" in line: + count = count + 1 + if count >= 7: + status = 'Passed' + except: + pass + return status + + def add4(self, text): + status = 'Failed' + copy_count = 0 + mul_count = 0 + add_count = 0 + triad_count = 0 + geomean_count = 0 + copy_re = re.compile('Copy\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d+') + mul_re = re.compile('Mul\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d+') + add_re = re.compile('Add4\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d+') + triad_re = re.compile('Triad\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d+') + geomean_re = re.compile('GEOMEAN\s+\d+\.\d+') + lines = text.splitlines() + try: + + for line in lines: + copy = re.search(copy_re, line) + mul = re.search(mul_re, line) + add = re.search(add_re, line) + triad = re.search(triad_re, line) + geomean = re.search(geomean_re, line) + if copy: + copy_count = copy_count + 1 + if mul: + mul_count = mul_count + 1 + if add: + add_count = add_count + 1 + if triad: + triad_count = triad_count + 1 + if geomean: + geomean_count = geomean_count + 1 + if copy_count >= 4 and mul_count >= 4 and add_count >= 4 and triad_count >= 4 and geomean_count >= 4: + status = 'Passed' + except: + pass + return status + + def gpu_burn(self, text): + status = 'Failed' + copy_count = 0 + mul_count = 0 + add_count = 0 + triad_count = 0 + geomean_count = 0 + lines = text.splitlines() + gpu_re = re.compile('Total no. of GPUs found:\s+(\d)') + gpus = 0 + for line in lines: + copy = re.search(gpu_re, line) + if copy: + gpus = int(copy.group(1)) + if 'Init Burn Thread for device' in line: + copy_count = copy_count + 1 + if 'Burn Thread using device' in line: + mul_count = mul_count + 1 + if 'Temps:' in line: + add_count = add_count + 1 + if 'Stopping burn thread on device' in line: + triad_count = triad_count + 1 + if copy_count >= gpus and mul_count >= gpus and add_count >= 5 and triad_count >= gpus and gpus >= 1: + status = 'Passed' + return status + + def cuda_stream(self, text): + status = 'Failed' + copy_count = 0 + mul_count = 0 + add_count = 0 + triad_count = 0 + lines = text.splitlines() + copy_re = re.compile(r'Copy:\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d+') + mul_re = re.compile(r'Scale:\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d+') + add_re = re.compile(r'Add:\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d+') + triad_re = re.compile(r'Triad:\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d+') + try: + for line in lines: + copy = re.search(copy_re, line) + mul = re.search(mul_re, line) + add = re.search(add_re, line) + triad = re.search(triad_re, line) + + if copy: + copy_count = copy_count + 1 + if mul: + mul_count = mul_count + 1 + if add: + add_count = add_count + 1 + if triad: + triad_count = triad_count + 1 + if copy_count >= 1 and mul_count >= 1 and add_count >= 1 and triad_count >= 1: + status = 'Passed' + except: + + print('exception') + pass + return status + + def mini_nbody(self, text): + hip_minibody_count = 0 + status = 'Failed' + count = 0 + lines = text.splitlines() + try: + for line in lines: + search = re.search('\d+\,\s+\d+\.\d+', line) + if search: + hip_minibody_count = hip_minibody_count + 1 + + if hip_minibody_count >= 26: + status = 'Passed' + except: + pass + return status + + def strided_access(self, text): + status = 'Failed' + count = 0 + try: + lines = text.splitlines() + for line in lines: + search = re.search('\d+\s+\d+\.\d+\s+\d+\.\d+', line) + if search: + count = count + 1 + if count >= 30: + status = 'Passed' + except: + pass + return status + + def gpu_stream(self, text): + testsuccess = True + test_status = 'Failed' + count = 0 + search = re.search('Copy\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d', text) + if search: + count = count + 1 + search = re.search('Mul\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d', text) + if search: + count = count + 1 + search = re.search('Add\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d', text) + if search: + count = count + 1 + search = re.search('Triad\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d', text) + if search: + count = count + 1 + search = re.search('Dot\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d+\s+\d+\.\d', text) + if search: + count = count + 1 + + search = re.search("Validation failed", text); + if search: + testsuccess &= False + + if (count >= 5) and (testsuccess != False): + test_status = 'Passed' + else: + test_status = 'Failed' + + return test_status + + def mix_bench(self, text): + test_status = 'Failed' + count = len(re.findall(' +\d+, *\d+\.\d+| +\d+, +inf', text)) + if (count >= 33): + test_status = 'Passed' + return test_status + + def hip_examples_applications(self, logfile): + pass_pattern1 = 'fault' + pass_pattern2 = 'Aborted' + pass_pattern3 = 'Error' + pass_pattern4 = 'failed' + logfile.seek(0) + for line in logfile: + if (pass_pattern1 in line) or (pass_pattern2 in line)\ + or (pass_pattern3 in line) or (pass_pattern4 in line): + return "Failed" + return "Passed" diff --git a/src/hiptestsuite/applications/hip_samples/Samples_Patch_4.2.x b/src/hiptestsuite/applications/hip_samples/Samples_Patch_4.2.x new file mode 100644 index 0000000..37efd52 --- /dev/null +++ b/src/hiptestsuite/applications/hip_samples/Samples_Patch_4.2.x @@ -0,0 +1,1514 @@ +diff -ruN HIP/samples/0_Intro/bit_extract/Makefile samples/0_Intro/bit_extract/Makefile +--- HIP/samples/0_Intro/bit_extract/Makefile 2021-07-09 10:33:24.804268020 +0530 ++++ samples/0_Intro/bit_extract/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -20,17 +20,23 @@ + + #Dependencies : [MYHIP]/bin must be in user's path. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) + HIP_PATH=../../.. ++ endif + endif +-HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++ + HIPCC=$(HIP_PATH)/bin/hipcc ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) + + # Show how to use PLATFORM to specify different options for each compiler: +-ifeq (${HIP_PLATFORM}, nvcc) +- HIPCC_FLAGS = -gencode=arch=compute_20,code=sm_20 +-endif ++# By default, it will be built for current target ++# ifeq (${HIP_PLATFORM}, nvidia) ++# HIPCC_FLAGS = -gencode=arch=compute_20,code=sm_20 ++# endif + + EXE=bit_extract + +diff -ruN HIP/samples/0_Intro/module_api/defaultDriver.cpp samples/0_Intro/module_api/defaultDriver.cpp +--- HIP/samples/0_Intro/module_api/defaultDriver.cpp 2021-07-09 10:33:24.804268020 +0530 ++++ samples/0_Intro/module_api/defaultDriver.cpp 2021-07-09 10:28:19.000000000 +0530 +@@ -44,10 +44,11 @@ + + hipInit(0); + hipDevice_t device; ++#ifdef __HIP_PLATFORM_NVIDIA__ + hipCtx_t context; + hipDeviceGet(&device, 0); + hipCtxCreate(&context, 0, device); +- ++#endif + hipMalloc((void**)&Ad, SIZE); + hipMalloc((void**)&Bd, SIZE); + +@@ -78,10 +79,12 @@ + std::cout << "FAILED!\n"; + }; + +- hipFree(Ad); +- hipFree(Bd); ++ hipFree((void *)Ad); ++ hipFree((void *)Bd); + delete[] A; + delete[] B; ++#ifdef __HIP_PLATFORM_NVIDIA__ + hipCtxDestroy(context); ++#endif + return 0; + } +diff -ruN HIP/samples/0_Intro/module_api/launchKernelHcc.cpp samples/0_Intro/module_api/launchKernelHcc.cpp +--- HIP/samples/0_Intro/module_api/launchKernelHcc.cpp 2021-07-09 10:33:24.804268020 +0530 ++++ samples/0_Intro/module_api/launchKernelHcc.cpp 2021-07-09 10:28:19.000000000 +0530 +@@ -55,10 +55,11 @@ + + hipInit(0); + hipDevice_t device; ++#ifdef __HIP_PLATFORM_NVIDIA__ + hipCtx_t context; + hipDeviceGet(&device, 0); + hipCtxCreate(&context, 0, device); +- ++#endif + hipMalloc((void**)&Ad, SIZE); + hipMalloc((void**)&Bd, SIZE); + +@@ -109,6 +110,8 @@ + hipFree(Bd); + delete[] A; + delete[] B; ++#ifdef __HIP_PLATFORM_NVIDIA__ + hipCtxDestroy(context); ++#endif + return 0; + } +diff -ruN HIP/samples/0_Intro/module_api/Makefile samples/0_Intro/module_api/Makefile +--- HIP/samples/0_Intro/module_api/Makefile 2021-07-09 10:33:24.804268020 +0530 ++++ samples/0_Intro/module_api/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,14 +18,24 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif ++ + HIPCC=$(HIP_PATH)/bin/hipcc +-HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) + ++all: ++ifeq (nvidia, $(HIP_PLATFORM)) ++all: vcpy_kernel.code runKernel.hip.out defaultDriver.hip.out ++else + all: vcpy_kernel.code runKernel.hip.out launchKernelHcc.hip.out defaultDriver.hip.out ++endif + + runKernel.hip.out: runKernel.cpp + $(HIPCC) $(HIPCC_FLAGS) $< -o $@ +diff -ruN HIP/samples/0_Intro/module_api/runKernel.cpp samples/0_Intro/module_api/runKernel.cpp +--- HIP/samples/0_Intro/module_api/runKernel.cpp 2021-07-09 10:33:24.804268020 +0530 ++++ samples/0_Intro/module_api/runKernel.cpp 2021-07-09 10:28:19.000000000 +0530 +@@ -25,7 +25,7 @@ + #include + #include + #include +-#include ++//#include + + #define LEN 64 + #define SIZE LEN << 2 +@@ -52,10 +52,11 @@ + + hipInit(0); + hipDevice_t device; ++#ifdef __HIP_PLATFORM_NVIDIA__ + hipCtx_t context; + hipDeviceGet(&device, 0); + hipCtxCreate(&context, 0, device); +- ++#endif + hipMalloc((void**)&Ad, SIZE); + hipMalloc((void**)&Bd, SIZE); + +@@ -97,10 +98,12 @@ + std::cout << "FAILED!\n"; + }; + +- hipFree(Ad); +- hipFree(Bd); ++ hipFree((void *)Ad); ++ hipFree((void *)Bd); + delete[] A; + delete[] B; ++#ifdef __HIP_PLATFORM_NVIDIA__ + hipCtxDestroy(context); ++#endif + return 0; + } +diff -ruN HIP/samples/0_Intro/module_api_global/Makefile samples/0_Intro/module_api_global/Makefile +--- HIP/samples/0_Intro/module_api_global/Makefile 2021-07-09 10:33:24.804268020 +0530 ++++ samples/0_Intro/module_api_global/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,12 +18,17 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif ++ + HIPCC=$(HIP_PATH)/bin/hipcc +-HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) + + all: vcpy_kernel.code runKernel.hip.out + +diff -ruN HIP/samples/0_Intro/module_api_global/runKernel.cpp samples/0_Intro/module_api_global/runKernel.cpp +--- HIP/samples/0_Intro/module_api_global/runKernel.cpp 2021-07-09 10:33:24.804268020 +0530 ++++ samples/0_Intro/module_api_global/runKernel.cpp 2021-07-09 10:28:19.000000000 +0530 +@@ -25,7 +25,7 @@ + #include + #include + #include +-#include ++//#include + + #define LEN 64 + #define SIZE LEN * sizeof(float) +@@ -54,10 +54,11 @@ + + hipInit(0); + hipDevice_t device; ++#ifdef __HIP_PLATFORM_NVIDIA__ + hipCtx_t context; + hipDeviceGet(&device, 0); + hipCtxCreate(&context, 0, device); +- ++#endif + hipMalloc((void**)&Ad, SIZE); + hipMalloc((void**)&Bd, SIZE); + +@@ -69,7 +70,7 @@ + float myDeviceGlobal_h = 42.0; + float* deviceGlobal; + size_t deviceGlobalSize; +- HIP_CHECK(hipModuleGetGlobal((void**)&deviceGlobal, &deviceGlobalSize, Module, "myDeviceGlobal")); ++ HIP_CHECK(hipModuleGetGlobal((hipDeviceptr_t *)&deviceGlobal, &deviceGlobalSize, Module, "myDeviceGlobal")); + HIP_CHECK(hipMemcpyHtoD(hipDeviceptr_t(deviceGlobal), &myDeviceGlobal_h, deviceGlobalSize)); + + #define ARRAY_SIZE 16 +@@ -77,7 +78,7 @@ + float myDeviceGlobalArray_h[ARRAY_SIZE]; + float *myDeviceGlobalArray; + size_t myDeviceGlobalArraySize; +- HIP_CHECK(hipModuleGetGlobal((void**)&myDeviceGlobalArray, &myDeviceGlobalArraySize, Module, "myDeviceGlobalArray")); ++ HIP_CHECK(hipModuleGetGlobal((hipDeviceptr_t *)&myDeviceGlobalArray, &myDeviceGlobalArraySize, Module, "myDeviceGlobalArray")); + for (int i = 0; i < ARRAY_SIZE; i++) { + myDeviceGlobalArray_h[i] = i * 1000.0f; + HIP_CHECK(hipMemcpyHtoD(hipDeviceptr_t(myDeviceGlobalArray), &myDeviceGlobalArray_h, myDeviceGlobalArraySize)); +@@ -101,7 +102,7 @@ + HIP_CHECK(hipModuleGetFunction(&Function, Module, "hello_world")); + HIP_CHECK(hipModuleLaunchKernel(Function, 1, 1, 1, LEN, 1, 1, 0, 0, NULL, (void**)&config)); + +- hipMemcpyDtoH(B, Bd, SIZE); ++ hipMemcpyDtoH(B, (hipDeviceptr_t)Bd, SIZE); + + int mismatchCount = 0; + for (uint32_t i = 0; i < LEN; i++) { +@@ -131,7 +132,7 @@ + printf("Num Regs = %d\n",val); + HIP_CHECK(hipModuleLaunchKernel(Function, 1, 1, 1, LEN, 1, 1, 0, 0, NULL, (void**)&config)); + +- hipMemcpyDtoH(B, Bd, SIZE); ++ hipMemcpyDtoH(B, (hipDeviceptr_t)Bd, SIZE); + + int mismatchCount = 0; + for (uint32_t i = 0; i < LEN; i++) { +@@ -156,6 +157,8 @@ + hipFree(Bd); + delete[] A; + delete[] B; ++#ifdef __HIP_PLATFORM_NVIDIA__ + hipCtxDestroy(context); ++#endif + return 0; + } +diff -ruN HIP/samples/0_Intro/square/Makefile samples/0_Intro/square/Makefile +--- HIP/samples/0_Intro/square/Makefile 2021-07-09 10:33:24.804268020 +0530 ++++ samples/0_Intro/square/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,12 +18,17 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif +-HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++ + HIPCC=$(HIP_PATH)/bin/hipcc ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) + + ifeq (${HIP_PLATFORM}, nvidia) + SOURCES=square.cu +diff -ruN HIP/samples/0_Intro/square/README.md samples/0_Intro/square/README.md +--- HIP/samples/0_Intro/square/README.md 2021-07-09 10:33:24.804268020 +0530 ++++ samples/0_Intro/square/README.md 2021-07-09 10:28:19.000000000 +0530 +@@ -1,8 +1,6 @@ + # Square.md + +-Simple test which shows how to use hipify-perl to port CUDA code to HIP. +-See related [blog](http://gpuopen.com/hip-to-be-squared-an-introductory-hip-tutorial) that explains the example. +-Now it is even simpler and requires no manual modification to the hipified source code - just hipify and compile: ++Simple test below is an example, shows how to use hipify-perl to port CUDA code to HIP: + + - Add hip/bin path to the PATH + +diff -ruN HIP/samples/1_Utils/hipBusBandwidth/Makefile samples/1_Utils/hipBusBandwidth/Makefile +--- HIP/samples/1_Utils/hipBusBandwidth/Makefile 2021-07-09 10:33:24.804268020 +0530 ++++ samples/1_Utils/hipBusBandwidth/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,11 +18,17 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif ++ + HIPCC=$(HIP_PATH)/bin/hipcc ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) + + EXE=hipBusBandwidth + CXXFLAGS = -O3 +diff -ruN HIP/samples/1_Utils/hipCommander/Makefile samples/1_Utils/hipCommander/Makefile +--- HIP/samples/1_Utils/hipCommander/Makefile 2021-07-09 10:33:24.804268020 +0530 ++++ samples/1_Utils/hipCommander/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,11 +18,17 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif ++ + HIPCC=$(HIP_PATH)/bin/hipcc ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) + + EXE=hipCommander + OPT=-O3 +diff -ruN HIP/samples/1_Utils/hipDispatchLatency/hipDispatchLatency.cpp samples/1_Utils/hipDispatchLatency/hipDispatchLatency.cpp +--- HIP/samples/1_Utils/hipDispatchLatency/hipDispatchLatency.cpp 2021-07-09 10:33:24.808268042 +0530 ++++ samples/1_Utils/hipDispatchLatency/hipDispatchLatency.cpp 2021-07-09 10:28:19.000000000 +0530 +@@ -65,10 +65,12 @@ + + int main() { + hipStream_t stream0 = 0; ++#ifdef __HIP_PLATFORM_NVIDIA__ + hipDevice_t device; + hipDeviceGet(&device, 0); + hipCtx_t context; + hipCtxCreate(&context, 0, device); ++#endif + hipModule_t module; + hipFunction_t function; + hipModuleLoad(&module, FILE_NAME); +@@ -136,6 +138,8 @@ + + hipEventDestroy(start); + hipEventDestroy(stop); ++#ifdef __HIP_PLATFORM_NVIDIA__ + hipCtxDestroy(context); ++#endif + } + +diff -ruN HIP/samples/1_Utils/hipDispatchLatency/Makefile samples/1_Utils/hipDispatchLatency/Makefile +--- HIP/samples/1_Utils/hipDispatchLatency/Makefile 2021-07-09 10:33:24.808268042 +0530 ++++ samples/1_Utils/hipDispatchLatency/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,11 +18,17 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif +-HIPCC=$(HIP_PATH)/bin/hipcc -std=c++11 ++ ++HIPCC=$(HIP_PATH)/bin/hipcc ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) + + CXXFLAGS = -O3 + +diff -ruN HIP/samples/1_Utils/hipInfo/Makefile samples/1_Utils/hipInfo/Makefile +--- HIP/samples/1_Utils/hipInfo/Makefile 2021-07-09 10:33:24.808268042 +0530 ++++ samples/1_Utils/hipInfo/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,11 +18,17 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif ++ + HIPCC=$(HIP_PATH)/bin/hipcc ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) + + EXE=hipInfo + +diff -ruN HIP/samples/2_Cookbook/0_MatrixTranspose/Makefile samples/2_Cookbook/0_MatrixTranspose/Makefile +--- HIP/samples/2_Cookbook/0_MatrixTranspose/Makefile 2021-07-09 10:33:24.808268042 +0530 ++++ samples/2_Cookbook/0_MatrixTranspose/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,12 +18,17 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif + + HIPCC=$(HIP_PATH)/bin/hipcc ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) + + TARGET=hcc + +diff -ruN HIP/samples/2_Cookbook/10_inline_asm/Makefile samples/2_Cookbook/10_inline_asm/Makefile +--- HIP/samples/2_Cookbook/10_inline_asm/Makefile 2021-07-09 10:33:24.808268042 +0530 ++++ samples/2_Cookbook/10_inline_asm/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,12 +18,17 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif + + HIPCC=$(HIP_PATH)/bin/hipcc ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) + + TARGET=hcc + +@@ -35,7 +40,10 @@ + .PHONY: test + + ++all: ++ifeq ($(HIP_PLATFORM),amd) + all: $(EXECUTABLE) test ++endif + + CXXFLAGS =-g + CXX=$(HIPCC) +diff -ruN HIP/samples/2_Cookbook/11_texture_driver/Makefile samples/2_Cookbook/11_texture_driver/Makefile +--- HIP/samples/2_Cookbook/11_texture_driver/Makefile 2021-07-09 10:33:24.808268042 +0530 ++++ samples/2_Cookbook/11_texture_driver/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,12 +18,17 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif ++ + HIPCC=$(HIP_PATH)/bin/hipcc +-HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) + + all: tex2dKernel.code texture2dDrv.out + +diff -ruN HIP/samples/2_Cookbook/11_texture_driver/texture2dDrv.cpp samples/2_Cookbook/11_texture_driver/texture2dDrv.cpp +--- HIP/samples/2_Cookbook/11_texture_driver/texture2dDrv.cpp 2021-07-09 10:33:24.808268042 +0530 ++++ samples/2_Cookbook/11_texture_driver/texture2dDrv.cpp 2021-07-09 10:28:19.000000000 +0530 +@@ -26,7 +26,31 @@ + #include + + #define fileName "tex2dKernel.code" ++#ifdef __HIP_PLATFORM_NVIDIA__ ++#define CTX_CREATE() \ ++ hipCtx_t context;\ ++ initHipCtx(&context); ++#define CTX_DESTROY() HIP_CHECK(hipCtxDestroy(context)); ++#define ARRAY_DESTROY(array) HIP_CHECK(hipArrayDestroy(array)); ++#define HIP_TEX_REFERENCE hipTexRef ++#define HIP_ARRAY hiparray ++/** ++ * Internal Function ++ */ ++void initHipCtx(hipCtx_t *pcontext) { ++ hipInit(0); ++ hipDevice_t device; ++ hipDeviceGet(&device, 0); ++ hipCtxCreate(pcontext, 0, device); ++} + ++#else ++#define CTX_CREATE() ++#define CTX_DESTROY() ++#define ARRAY_DESTROY(array) HIP_CHECK(hipFreeArray(array)); ++#define HIP_TEX_REFERENCE textureReference* ++#define HIP_ARRAY hipArray* ++#endif + bool testResult = true; + + #define HIP_CHECK(cmd) \ +@@ -50,10 +74,11 @@ + hData[i * width + j] = i * width + j; + } + } ++ CTX_CREATE(); + hipModule_t Module; + HIP_CHECK(hipModuleLoad(&Module, fileName)); + +- hipArray* array; ++ HIP_ARRAY array; + HIP_ARRAY_DESCRIPTOR desc; + desc.Format = HIP_AD_FORMAT_FLOAT; + desc.NumChannels = 1; +@@ -63,23 +88,38 @@ + + hip_Memcpy2D copyParam; + memset(©Param, 0, sizeof(copyParam)); +- copyParam.dstMemoryType = hipMemoryTypeArray; ++#ifdef __HIP_PLATFORM_NVCC__ ++ copyParam.dstMemoryType = CU_MEMORYTYPE_ARRAY; ++ copyParam.srcMemoryType = CU_MEMORYTYPE_HOST; + copyParam.dstArray = array; ++#else ++ copyParam.dstMemoryType = hipMemoryTypeArray; + copyParam.srcMemoryType = hipMemoryTypeHost; ++ copyParam.dstArray = array; ++#endif + copyParam.srcHost = hData; + copyParam.srcPitch = width * sizeof(float); + copyParam.WidthInBytes = copyParam.srcPitch; + copyParam.Height = height; + HIP_CHECK(hipMemcpyParam2D(©Param)); +- +- textureReference* texref; +- HIP_CHECK(hipModuleGetTexRef(&texref, Module, "tex")); +- HIP_CHECK(hipTexRefSetAddressMode(texref, 0, hipAddressModeWrap)); +- HIP_CHECK(hipTexRefSetAddressMode(texref, 1, hipAddressModeWrap)); +- HIP_CHECK(hipTexRefSetFilterMode(texref, hipFilterModePoint)); +- HIP_CHECK(hipTexRefSetFlags(texref, 0)); +- HIP_CHECK(hipTexRefSetFormat(texref, HIP_AD_FORMAT_FLOAT, 1)); +- HIP_CHECK(hipTexRefSetArray(texref, array, HIP_TRSA_OVERRIDE_FORMAT)); ++ HIP_TEX_REFERENCE texref; ++#ifdef __HIP_PLATFORM_NVIDIA__ ++ HIP_CHECK(hipModuleGetTexRef(&texref, Module, "tex")); ++ HIP_CHECK(hipTexRefSetAddressMode(texref, 0, CU_TR_ADDRESS_MODE_WRAP)); ++ HIP_CHECK(hipTexRefSetAddressMode(texref, 1, CU_TR_ADDRESS_MODE_WRAP)); ++ HIP_CHECK(hipTexRefSetFilterMode(texref, HIP_TR_FILTER_MODE_POINT)); ++ HIP_CHECK(hipTexRefSetFlags(texref, 0)); ++ HIP_CHECK(hipTexRefSetFormat(texref, HIP_AD_FORMAT_FLOAT, 1)); ++ HIP_CHECK(hipTexRefSetArray(texref, array, CU_TRSA_OVERRIDE_FORMAT)); ++#else ++ HIP_CHECK(hipModuleGetTexRef(&texref, Module, "tex")); ++ HIP_CHECK(hipTexRefSetAddressMode(texref, 0, hipAddressModeWrap)); ++ HIP_CHECK(hipTexRefSetAddressMode(texref, 1, hipAddressModeWrap)); ++ HIP_CHECK(hipTexRefSetFilterMode(texref, hipFilterModePoint)); ++ HIP_CHECK(hipTexRefSetFlags(texref, 0)); ++ HIP_CHECK(hipTexRefSetFormat(texref, HIP_AD_FORMAT_FLOAT, 1)); ++ HIP_CHECK(hipTexRefSetArray(texref, array, HIP_TRSA_OVERRIDE_FORMAT)); ++#endif + + float* dData = NULL; + HIP_CHECK(hipMalloc((void**)&dData, size)); +@@ -121,9 +161,9 @@ + } + } + } +- HIP_CHECK(hipUnbindTexture(texref)); + HIP_CHECK(hipFree(dData)); +- HIP_CHECK(hipFreeArray(array)); ++ ARRAY_DESTROY(array) ++ CTX_DESTROY(); + return testResult; + } + +diff -ruN HIP/samples/2_Cookbook/13_occupancy/Makefile samples/2_Cookbook/13_occupancy/Makefile +--- HIP/samples/2_Cookbook/13_occupancy/Makefile 2021-07-09 10:33:24.808268042 +0530 ++++ samples/2_Cookbook/13_occupancy/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,11 +18,17 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif ++ + HIPCC=$(HIP_PATH)/bin/hipcc ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) + + EXE=./occupancy + +diff -ruN HIP/samples/2_Cookbook/14_gpu_arch/gpuarch.cpp samples/2_Cookbook/14_gpu_arch/gpuarch.cpp +--- HIP/samples/2_Cookbook/14_gpu_arch/gpuarch.cpp 2021-07-09 10:33:24.808268042 +0530 ++++ samples/2_Cookbook/14_gpu_arch/gpuarch.cpp 2021-07-09 10:28:19.000000000 +0530 +@@ -90,9 +90,9 @@ + } + } + if (flag == false) { +- std::cout << "Error: Kernel is supported for gfx908 architecture\n"; ++ std::cout << "info: Kernel is supported only for gfx908 architecture\n"; + } else { + std::cout << "success\n"; + } + return 0; +-} +\ No newline at end of file ++} +diff -ruN HIP/samples/2_Cookbook/14_gpu_arch/Makefile samples/2_Cookbook/14_gpu_arch/Makefile +--- HIP/samples/2_Cookbook/14_gpu_arch/Makefile 2021-07-09 10:33:24.808268042 +0530 ++++ samples/2_Cookbook/14_gpu_arch/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,17 +18,29 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif ++ + HIPCC=$(HIP_PATH)/bin/hipcc ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) + + EXE=./gpuarch + + .PHONY: test + ++all: ++ifneq (amd, $(HIP_PLATFORM)) ++all: ++ $(info Not supported for platform $(HIP_PLATFORM)!) ++else + all: test ++endif + + $(EXE): gpuarch.cpp + $(HIPCC) $^ -o $@ +diff -ruN HIP/samples/2_Cookbook/15_static_library/device_functions/CMakeLists.txt samples/2_Cookbook/15_static_library/device_functions/CMakeLists.txt +--- HIP/samples/2_Cookbook/15_static_library/device_functions/CMakeLists.txt 1970-01-01 05:30:00.000000000 +0530 ++++ samples/2_Cookbook/15_static_library/device_functions/CMakeLists.txt 2021-07-09 10:28:19.000000000 +0530 +@@ -0,0 +1,36 @@ ++project(static_lib) ++ ++cmake_minimum_required(VERSION 3.10) ++ ++# Search for rocm in common locations ++list(APPEND CMAKE_PREFIX_PATH /opt/rocm/hip /opt/rocm) ++ ++# Find hip ++find_package(hip REQUIRED) ++ ++# Set compiler and linker ++set(CMAKE_CXX_COMPILER ${HIP_HIPCC_EXECUTABLE}) ++set(CMAKE_CXX_LINKER ${HIP_HIPCC_EXECUTABLE}) ++set(CMAKE_BUILD_TYPE Release) ++ ++# Turn static library generation ON ++option(BUILD_SHARED_LIBS "Build as a shared library" OFF) ++ ++set(CPP_SOURCES hipDevice.cpp) ++ ++# Generate static lib libHipDevice.a ++add_library(HipDevice STATIC ${CPP_SOURCES}) ++ ++target_compile_options(HipDevice PRIVATE -fgpu-rdc) ++target_link_libraries(HipDevice PRIVATE -fgpu-rdc) ++target_include_directories(HipDevice PRIVATE /opt/rocm/hsa/include) ++ ++# Create test executable that uses libHipDevice.a ++set(TEST_SOURCES ${CMAKE_SOURCE_DIR}/hipMain2.cpp) ++ ++add_executable(test_device_static ${TEST_SOURCES}) ++add_dependencies(test_device_static HipDevice) ++target_compile_options(test_device_static PRIVATE -fgpu-rdc) ++target_link_libraries(test_device_static PRIVATE HipDevice) ++target_link_libraries(test_device_static PRIVATE -fgpu-rdc hip::host) ++ +diff -ruN HIP/samples/2_Cookbook/15_static_library/device_functions/hipDevice.cpp samples/2_Cookbook/15_static_library/device_functions/hipDevice.cpp +--- HIP/samples/2_Cookbook/15_static_library/device_functions/hipDevice.cpp 1970-01-01 05:30:00.000000000 +0530 ++++ samples/2_Cookbook/15_static_library/device_functions/hipDevice.cpp 2021-07-09 10:28:19.000000000 +0530 +@@ -0,0 +1,28 @@ ++/* ++ * Copyright (c) 2020-present Advanced Micro Devices, Inc. All rights reserved. ++ * ++ * Permission is hereby granted, free of charge, to any person obtaining a copy ++ * of this software and associated documentation files (the "Software"), to deal ++ * in the Software without restriction, including without limitation the rights ++ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell ++ * copies of the Software, and to permit persons to whom the Software is ++ * furnished to do so, subject to the following conditions: ++ * ++ * The above copyright notice and this permission notice shall be included in ++ * all copies or substantial portions of the Software. ++ * ++ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR ++ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, ++ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE ++ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER ++ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, ++ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN ++ * THE SOFTWARE. ++ * */ ++ ++#include ++ ++__device__ int square_me(int A) { ++ return A*A; ++} ++ +diff -ruN HIP/samples/2_Cookbook/15_static_library/device_functions/hipMain2.cpp samples/2_Cookbook/15_static_library/device_functions/hipMain2.cpp +--- HIP/samples/2_Cookbook/15_static_library/device_functions/hipMain2.cpp 1970-01-01 05:30:00.000000000 +0530 ++++ samples/2_Cookbook/15_static_library/device_functions/hipMain2.cpp 2021-07-09 10:28:19.000000000 +0530 +@@ -0,0 +1,69 @@ ++/* ++ * Copyright (c) 2020-present Advanced Micro Devices, Inc. All rights reserved. ++ * ++ * Permission is hereby granted, free of charge, to any person obtaining a copy ++ * of this software and associated documentation files (the "Software"), to deal ++ * in the Software without restriction, including without limitation the rights ++ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell ++ * copies of the Software, and to permit persons to whom the Software is ++ * furnished to do so, subject to the following conditions: ++ * ++ * The above copyright notice and this permission notice shall be included in ++ * all copies or substantial portions of the Software. ++ * ++ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR ++ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, ++ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE ++ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER ++ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, ++ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN ++ * THE SOFTWARE. ++ * */ ++ ++#include ++#include ++#include ++ ++#define HIP_ASSERT(status) assert(status == hipSuccess) ++#define LEN 512 ++ ++extern __device__ int square_me(int); ++ ++__global__ void square_and_save(int* A, int* B) { ++ int tid = threadIdx.x + blockIdx.x * blockDim.x; ++ B[tid] = square_me(A[tid]); ++} ++ ++void run_test2() { ++ int *A_h, *B_h, *A_d, *B_d; ++ A_h = new int[LEN]; ++ B_h = new int[LEN]; ++ for (unsigned i = 0; i < LEN; i++) { ++ A_h[i] = i; ++ B_h[i] = 0; ++ } ++ size_t valbytes = LEN*sizeof(int); ++ ++ HIP_ASSERT(hipMalloc((void**)&A_d, valbytes)); ++ HIP_ASSERT(hipMalloc((void**)&B_d, valbytes)); ++ ++ HIP_ASSERT(hipMemcpy(A_d, A_h, valbytes, hipMemcpyHostToDevice)); ++ hipLaunchKernelGGL(square_and_save, dim3(LEN/64), dim3(64), ++ 0, 0, A_d, B_d); ++ HIP_ASSERT(hipMemcpy(B_h, B_d, valbytes, hipMemcpyDeviceToHost)); ++ ++ for (unsigned i = 0; i < LEN; i++) { ++ assert(A_h[i]*A_h[i] == B_h[i]); ++ } ++ ++ HIP_ASSERT(hipFree(A_d)); ++ HIP_ASSERT(hipFree(B_d)); ++ free(A_h); ++ free(B_h); ++ std::cout << "Test Passed!\n"; ++} ++ ++int main(){ ++ // Run test that generates static lib with ar ++ run_test2(); ++} +diff -ruN HIP/samples/2_Cookbook/15_static_library/device_functions/Makefile samples/2_Cookbook/15_static_library/device_functions/Makefile +--- HIP/samples/2_Cookbook/15_static_library/device_functions/Makefile 1970-01-01 05:30:00.000000000 +0530 ++++ samples/2_Cookbook/15_static_library/device_functions/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -0,0 +1,41 @@ ++HIP_PATH = $(wildcard /opt/rocm/hip) ++ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../../../build ++endif ++ ++ ++HIPCC=$(HIP_PATH)/bin/hipcc ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) ++ ++.PHONY: test ++ ++all: ++ifneq (amd, $(HIP_PLATFORM)) ++all: ++ $(info Not supported for platform $(HIP_PLATFORM)!) ++else ++all: $(RDC_EXE) test ++endif ++ ++STATIC_LIB_SRC=hipDevice.cpp ++STATIC_LIB=./libHipDevice.a ++STATIC_MAIN_SRC=hipMain2.cpp ++RDC_EXE=./test_device_static.out ++ ++$(STATIC_LIB): ++ $(HIPCC) $(STATIC_LIB_SRC) -c -fgpu-rdc -fPIC -o hipDevice.o ++ ar rcsD $@ hipDevice.o ++ ++# Compiles hipMain2 with hipcc and links with libHipDevice.a which contains device function. ++$(RDC_EXE): $(STATIC_LIB) ++ $(HIPCC) $(STATIC_LIB) $(STATIC_MAIN_SRC) -fgpu-rdc -o $@ ++ ++test: $(RDC_EXE) ++ $(RDC_EXE) ++ ++clean: ++ rm -f $(RDC_EXE) ++ rm -f $(STATIC_LIB) ++ rm -f *.o +diff -ruN HIP/samples/2_Cookbook/15_static_library/host_functions/CMakeLists.txt samples/2_Cookbook/15_static_library/host_functions/CMakeLists.txt +--- HIP/samples/2_Cookbook/15_static_library/host_functions/CMakeLists.txt 1970-01-01 05:30:00.000000000 +0530 ++++ samples/2_Cookbook/15_static_library/host_functions/CMakeLists.txt 2021-07-09 10:28:19.000000000 +0530 +@@ -0,0 +1,39 @@ ++project(static_lib) ++ ++cmake_minimum_required(VERSION 3.10) ++ ++# Search for rocm in common locations ++list(APPEND CMAKE_PREFIX_PATH /opt/rocm/hip /opt/rocm) ++ ++# Find hip ++find_package(hip REQUIRED) ++ ++# Set compiler and linker ++set(CMAKE_CXX_COMPILER ${HIP_HIPCC_EXECUTABLE}) ++set(CMAKE_CXX_LINKER ${HIP_HIPCC_EXECUTABLE}) ++set(CMAKE_AR ${HIP_HIPCC_EXECUTABLE}) ++set(CMAKE_BUILD_TYPE Release) ++ ++# Turn static library generation ON ++option(BUILD_SHARED_LIBS "Build as a shared library" OFF) ++ ++set(CPP_SOURCES hipOptLibrary.cpp) ++ ++# Generate static lib libHipOptLibrary.a. ++add_library(HipOptLibrary STATIC ${CPP_SOURCES}) ++ ++# Set-up the correct flags to generate the static library. ++target_link_libraries(HipOptLibrary PRIVATE --emit-static-lib) ++target_include_directories(HipOptLibrary PRIVATE /opt/rocm/hsa/include) ++get_property(link_libraries TARGET HipOptLibrary PROPERTY LINK_LIBRARIES) ++string (REPLACE ";" " " LINK_PROPS "${link_libraries}") ++set(CMAKE_CXX_ARCHIVE_CREATE " -o ${LINK_PROPS} ") ++ ++# Create test executable that uses libHipOptLibrary.a ++set(TEST_SOURCES ${CMAKE_SOURCE_DIR}/hipMain1.cpp) ++ ++add_executable(test_opt_static ${TEST_SOURCES}) ++add_dependencies(test_opt_static HipOptLibrary) ++target_link_libraries(test_opt_static PRIVATE -lHipOptLibrary -L${CMAKE_BINARY_DIR}) ++target_link_libraries(test_opt_static PRIVATE amdhip64 amd_comgr hsa-runtime64::hsa-runtime64) ++ +diff -ruN HIP/samples/2_Cookbook/15_static_library/host_functions/hipMain1.cpp samples/2_Cookbook/15_static_library/host_functions/hipMain1.cpp +--- HIP/samples/2_Cookbook/15_static_library/host_functions/hipMain1.cpp 1970-01-01 05:30:00.000000000 +0530 ++++ samples/2_Cookbook/15_static_library/host_functions/hipMain1.cpp 2021-07-09 10:28:19.000000000 +0530 +@@ -0,0 +1,28 @@ ++/* ++ * Copyright (c) 2020-present Advanced Micro Devices, Inc. All rights reserved. ++ * ++ * Permission is hereby granted, free of charge, to any person obtaining a copy ++ * of this software and associated documentation files (the "Software"), to deal ++ * in the Software without restriction, including without limitation the rights ++ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell ++ * copies of the Software, and to permit persons to whom the Software is ++ * furnished to do so, subject to the following conditions: ++ * ++ * The above copyright notice and this permission notice shall be included in ++ * all copies or substantial portions of the Software. ++ * ++ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR ++ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, ++ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE ++ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER ++ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, ++ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN ++ * THE SOFTWARE. ++ * */ ++ ++extern void run_test1(); ++ ++int main(){ ++ // Run test that generates static lib with -emit-static-lib ++ run_test1(); ++} +diff -ruN HIP/samples/2_Cookbook/15_static_library/host_functions/hipOptLibrary.cpp samples/2_Cookbook/15_static_library/host_functions/hipOptLibrary.cpp +--- HIP/samples/2_Cookbook/15_static_library/host_functions/hipOptLibrary.cpp 1970-01-01 05:30:00.000000000 +0530 ++++ samples/2_Cookbook/15_static_library/host_functions/hipOptLibrary.cpp 2021-07-09 10:28:19.000000000 +0530 +@@ -0,0 +1,62 @@ ++/* ++ * Copyright (c) 2020-present Advanced Micro Devices, Inc. All rights reserved. ++ * ++ * Permission is hereby granted, free of charge, to any person obtaining a copy ++ * of this software and associated documentation files (the "Software"), to deal ++ * in the Software without restriction, including without limitation the rights ++ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell ++ * copies of the Software, and to permit persons to whom the Software is ++ * furnished to do so, subject to the following conditions: ++ * ++ * The above copyright notice and this permission notice shall be included in ++ * all copies or substantial portions of the Software. ++ * ++ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR ++ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, ++ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE ++ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER ++ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, ++ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN ++ * THE SOFTWARE. ++ * */ ++ ++#include ++#include ++#include ++ ++#define HIP_ASSERT(status) assert(status == hipSuccess) ++#define LEN 512 ++ ++__global__ void copy(uint32_t* A, uint32_t* B) { ++ size_t tid = threadIdx.x + blockIdx.x * blockDim.x; ++ B[tid] = A[tid]; ++} ++ ++void run_test1() { ++ uint32_t *A_h, *B_h, *A_d, *B_d; ++ size_t valbytes = LEN * sizeof(uint32_t); ++ ++ A_h = (uint32_t*)malloc(valbytes); ++ B_h = (uint32_t*)malloc(valbytes); ++ for (uint32_t i = 0; i < LEN; i++) { ++ A_h[i] = i; ++ B_h[i] = 0; ++ } ++ ++ HIP_ASSERT(hipMalloc((void**)&A_d, valbytes)); ++ HIP_ASSERT(hipMalloc((void**)&B_d, valbytes)); ++ ++ HIP_ASSERT(hipMemcpy(A_d, A_h, valbytes, hipMemcpyHostToDevice)); ++ hipLaunchKernelGGL(copy, dim3(LEN/64), dim3(64), 0, 0, A_d, B_d); ++ HIP_ASSERT(hipMemcpy(B_h, B_d, valbytes, hipMemcpyDeviceToHost)); ++ ++ for (uint32_t i = 0; i < LEN; i++) { ++ assert(A_h[i] == B_h[i]); ++ } ++ ++ HIP_ASSERT(hipFree(A_d)); ++ HIP_ASSERT(hipFree(B_d)); ++ free(A_h); ++ free(B_h); ++ std::cout << "Test Passed!\n"; ++} +diff -ruN HIP/samples/2_Cookbook/15_static_library/host_functions/Makefile samples/2_Cookbook/15_static_library/host_functions/Makefile +--- HIP/samples/2_Cookbook/15_static_library/host_functions/Makefile 1970-01-01 05:30:00.000000000 +0530 ++++ samples/2_Cookbook/15_static_library/host_functions/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -0,0 +1,49 @@ ++HIP_PATH = $(wildcard /opt/rocm/hip) ++ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../../../build ++endif ++ ++ ++HIPCC=$(HIP_PATH)/bin/hipcc ++GXX=g++ ++ ++EMIT_STATIC_LIB_SRC=hipOptLibrary.cpp ++EMIT_STATIC_LIB=./libHipOptLibrary.a ++EMIT_STATIC_MAIN_SRC=hipMain1.cpp ++HIPCC_EXE=./test_emit_static_hipcc_linker.out ++HOST_EXE=./test_emit_static_host_linker.out ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) ++ ++ ++.PHONY: test ++ ++all: ++ifneq (amd, $(HIP_PLATFORM)) ++all: ++ $(info Not supported for platform $(HIP_PLATFORM)!) ++else ++all: $(HIPCC_EXE) $(HOST_EXE) test ++endif ++ ++$(EMIT_STATIC_LIB): ++ $(HIPCC) $(EMIT_STATIC_LIB_SRC) --emit-static-lib -fPIC -o $@ ++ ++# Compiles hipMain1 with hipcc and links with libHipOptLibrary.a which contains host function. ++$(HIPCC_EXE): $(EMIT_STATIC_LIB) ++ $(HIPCC) $(EMIT_STATIC_MAIN_SRC) -L. -lHipOptLibrary -o $@ ++ ++# Compiles hipMain1 with g++ and links with libHipOptLibrary.a which contains host function. ++$(HOST_EXE): $(EMIT_STATIC_LIB) ++ $(GXX) $(EMIT_STATIC_MAIN_SRC) -L. -lHipOptLibrary -L$(HIP_PATH)/lib -lamdhip64 -Wl,-rpath=$(HIP_PATH)/lib -o $@ ++ ++test: $(HIPCC_EXE) $(HOST_EXE) ++ $(HIPCC_EXE) ++ $(HOST_EXE) ++ ++clean: ++ rm -f $(HIPCC_EXE) ++ rm -f $(HOST_EXE) ++ rm -f $(EMIT_STATIC_LIB) ++ rm -f *.o +diff -ruN HIP/samples/2_Cookbook/15_static_library/README.md samples/2_Cookbook/15_static_library/README.md +--- HIP/samples/2_Cookbook/15_static_library/README.md 1970-01-01 05:30:00.000000000 +0530 ++++ samples/2_Cookbook/15_static_library/README.md 2021-07-09 10:28:19.000000000 +0530 +@@ -0,0 +1,175 @@ ++# Emitting Static Library ++ ++This sample shows how to generate a static library for a simple HIP application. We will evaluate two types of static libraries: the first type exports host functions in a static library generated with --emit-static-lib and is compatible with host linkers, and second type exports device functions in a static library made with system ar. ++ ++Please refer to the hip_programming_guide for limitations. ++ ++## Static libraries with host functions ++ ++### Source files ++The static library source files may contain host functions and kernel `__global__` and `__device__` functions. Here is an example (please refer to the directory host_functions). ++ ++hipOptLibrary.cpp: ++``` ++#define HIP_ASSERT(status) assert(status == hipSuccess) ++#define LEN 512 ++ ++__global__ void copy(uint32_t* A, uint32_t* B) { ++ size_t tid = threadIdx.x + blockIdx.x * blockDim.x; ++ B[tid] = A[tid]; ++} ++ ++void run_test1() { ++ uint32_t *A_h, *B_h, *A_d, *B_d; ++ size_t valbytes = LEN * sizeof(uint32_t); ++ ++ A_h = (uint32_t*)malloc(valbytes); ++ B_h = (uint32_t*)malloc(valbytes); ++ for (uint32_t i = 0; i < LEN; i++) { ++ A_h[i] = i; ++ B_h[i] = 0; ++ } ++ ++ HIP_ASSERT(hipMalloc((void**)&A_d, valbytes)); ++ HIP_ASSERT(hipMalloc((void**)&B_d, valbytes)); ++ ++ HIP_ASSERT(hipMemcpy(A_d, A_h, valbytes, hipMemcpyHostToDevice)); ++ hipLaunchKernelGGL(copy, dim3(LEN/64), dim3(64), 0, 0, A_d, B_d); ++ HIP_ASSERT(hipMemcpy(B_h, B_d, valbytes, hipMemcpyDeviceToHost)); ++ ++ for (uint32_t i = 0; i < LEN; i++) { ++ assert(A_h[i] == B_h[i]); ++ } ++ ++ HIP_ASSERT(hipFree(A_d)); ++ HIP_ASSERT(hipFree(B_d)); ++ free(A_h); ++ free(B_h); ++ std::cout << "Test Passed!\n"; ++} ++``` ++ ++The above source file can be compiled into a static library, libHipOptLibrary.a, using the --emit-static-lib flag, like so: ++``` ++hipcc hipOptLibrary.cpp --emit-static-lib -fPIC -o libHipOptLibrary.a ++``` ++ ++### Main source files ++The main() program source file may link with the above static library using either hipcc or a host compiler (such as g++). A simple source file that calls the host function inside libHipOptLibrary.a: ++ ++hipMain1.cpp: ++``` ++extern void run_test1(); ++ ++int main(){ ++ run_test1(); ++} ++``` ++ ++To link to the static library: ++ ++Using hipcc: ++``` ++hipcc hipMain1.cpp -L. -lHipOptLibrary -o test_emit_static_hipcc_linker.out ++``` ++Using g++: ++``` ++g++ hipMain1.cpp -L. -lHipOptLibrary -L/opt/rocm/hip/lib -lamdhip64 -o test_emit_static_host_linker.out ++``` ++ ++## Static libraries with device functions ++ ++### Source files ++The static library source files which contain only `__device__` functions need to be created using ar. Here is an example (please refer to the directory device_functions). ++ ++hipDevice.cpp: ++``` ++#include ++ ++__device__ int square_me(int A) { ++ return A*A; ++} ++``` ++ ++The above source file may be compiled into a static library, libHipDevice.a, by first compiling into a relocatable object, and then placed in an archive using ar: ++``` ++hipcc hipDevice.cpp -c -fgpu-rdc -fPIC -o hipDevice.o ++ar rcsD libHipDevice.a hipDevice.o ++``` ++ ++### Main source files ++The main() program source file can link with the static library using hipcc. A simple source file that calls the device function inside libHipDevice.a: ++ ++hipMain2.cpp: ++``` ++#include ++#include ++#include ++ ++#define HIP_ASSERT(status) assert(status == hipSuccess) ++#define LEN 512 ++ ++extern __device__ int square_me(int); ++ ++__global__ void square_and_save(int* A, int* B) { ++ int tid = threadIdx.x + blockIdx.x * blockDim.x; ++ B[tid] = square_me(A[tid]); ++} ++ ++void run_test2() { ++ int *A_h, *B_h, *A_d, *B_d; ++ A_h = new int[LEN]; ++ B_h = new int[LEN]; ++ for (unsigned i = 0; i < LEN; i++) { ++ A_h[i] = i; ++ B_h[i] = 0; ++ } ++ size_t valbytes = LEN*sizeof(int); ++ ++ HIP_ASSERT(hipMalloc((void**)&A_d, valbytes)); ++ HIP_ASSERT(hipMalloc((void**)&B_d, valbytes)); ++ ++ HIP_ASSERT(hipMemcpy(A_d, A_h, valbytes, hipMemcpyHostToDevice)); ++ hipLaunchKernelGGL(square_and_save, dim3(LEN/64), dim3(64), ++ 0, 0, A_d, B_d); ++ HIP_ASSERT(hipMemcpy(B_h, B_d, valbytes, hipMemcpyDeviceToHost)); ++ ++ for (unsigned i = 0; i < LEN; i++) { ++ assert(A_h[i]*A_h[i] == B_h[i]); ++ } ++ ++ HIP_ASSERT(hipFree(A_d)); ++ HIP_ASSERT(hipFree(B_d)); ++ free(A_h); ++ free(B_h); ++ std::cout << "Test Passed!\n"; ++} ++ ++int main(){ ++ // Run test that generates static lib with ar ++ run_test2(); ++} ++``` ++ ++To link to the static library: ++``` ++hipcc libHipDevice.a hipMain2.cpp -fgpu-rdc -o test_device_static_hipcc.out ++``` ++ ++## How to build and run this sample: ++Use the make command to build the static libraries, link with it, and execute it. ++- Change directory to either host or device functions folder. ++- To build the static library and link the main executable, use `make all`. ++- To execute, run the generated executable `./test_*.out`. ++ ++Alternatively, use these CMake commands. ++``` ++cd device_functions ++mkdir -p build ++cd build ++cmake .. ++make ++./test_*.out ++``` ++ ++## For More Infomation, please refer to the HIP FAQ. +diff -ruN HIP/samples/2_Cookbook/16_assembly_to_executable/Makefile samples/2_Cookbook/16_assembly_to_executable/Makefile +--- HIP/samples/2_Cookbook/16_assembly_to_executable/Makefile 2021-07-09 10:33:24.808268042 +0530 ++++ samples/2_Cookbook/16_assembly_to_executable/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,11 +18,17 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif + ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) ++ + HIPCC=$(HIP_PATH)/bin/hipcc + CLANG=$(HIP_PATH)/../llvm/bin/clang + LLVM_MC=$(HIP_PATH)/../llvm/bin/llvm-mc +@@ -44,7 +50,13 @@ + + .PHONY: test + ++all: ++ifneq (amd, $(HIP_PLATFORM)) ++all: ++ $(info Not supported for platform $(HIP_PLATFORM)!) ++else + all: src_to_asm asm_to_exec ++endif + + src_to_asm: + $(HIPCC) -c -S --cuda-host-only -target x86_64-linux-gnu -o $(SQ_HOST_ASM) $(SRCS) +diff -ruN HIP/samples/2_Cookbook/17_llvm_ir_to_executable/Makefile samples/2_Cookbook/17_llvm_ir_to_executable/Makefile +--- HIP/samples/2_Cookbook/17_llvm_ir_to_executable/Makefile 2021-07-09 10:33:24.808268042 +0530 ++++ samples/2_Cookbook/17_llvm_ir_to_executable/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,11 +18,17 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif + ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) ++ + HIPCC=$(HIP_PATH)/bin/hipcc + CLANG=$(HIP_PATH)/../llvm/bin/clang + LLVM_MC=$(HIP_PATH)/../llvm/bin/llvm-mc +@@ -47,7 +53,13 @@ + + .PHONY: test + ++all: ++ifneq (amd, $(HIP_PLATFORM)) ++all: ++ $(info Not supported for platform $(HIP_PLATFORM)!) ++else + all: src_to_ir bc_to_ll ll_to_bc ir_to_exec ++endif + + src_to_ir: + $(HIPCC) -c -emit-llvm --cuda-host-only -target x86_64-linux-gnu -o $(SQ_HOST_BC) $(SRCS) +diff -ruN HIP/samples/2_Cookbook/1_hipEvent/Makefile samples/2_Cookbook/1_hipEvent/Makefile +--- HIP/samples/2_Cookbook/1_hipEvent/Makefile 2021-07-09 10:33:24.808268042 +0530 ++++ samples/2_Cookbook/1_hipEvent/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,12 +18,17 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif + + HIPCC=$(HIP_PATH)/bin/hipcc ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) + + TARGET=hcc + +diff -ruN HIP/samples/2_Cookbook/3_shared_memory/Makefile samples/2_Cookbook/3_shared_memory/Makefile +--- HIP/samples/2_Cookbook/3_shared_memory/Makefile 2021-07-09 10:33:24.808268042 +0530 ++++ samples/2_Cookbook/3_shared_memory/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,12 +18,17 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif + + HIPCC=$(HIP_PATH)/bin/hipcc ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) + + TARGET=hcc + +diff -ruN HIP/samples/2_Cookbook/4_shfl/Makefile samples/2_Cookbook/4_shfl/Makefile +--- HIP/samples/2_Cookbook/4_shfl/Makefile 2021-07-09 10:33:24.808268042 +0530 ++++ samples/2_Cookbook/4_shfl/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,17 +18,22 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif + ++HIPCC=$(HIP_PATH)/bin/hipcc ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) ++ + ifeq (gfx701, $(findstring gfx701,$(HCC_AMDGPU_TARGET))) + $(error gfx701 is not a supported device for this sample) + endif + +-HIPCC=$(HIP_PATH)/bin/hipcc +- + TARGET=hcc + + SOURCES = shfl.cpp +diff -ruN HIP/samples/2_Cookbook/5_2dshfl/Makefile samples/2_Cookbook/5_2dshfl/Makefile +--- HIP/samples/2_Cookbook/5_2dshfl/Makefile 2021-07-09 10:33:24.808268042 +0530 ++++ samples/2_Cookbook/5_2dshfl/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,17 +18,22 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif + ++HIPCC=$(HIP_PATH)/bin/hipcc ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) ++ + ifeq (gfx701, $(findstring gfx701,$(HCC_AMDGPU_TARGET))) + $(error gfx701 is not a supported device for this sample) + endif + +-HIPCC=$(HIP_PATH)/bin/hipcc +- + TARGET=hcc + + SOURCES = 2dshfl.cpp +diff -ruN HIP/samples/2_Cookbook/6_dynamic_shared/Makefile samples/2_Cookbook/6_dynamic_shared/Makefile +--- HIP/samples/2_Cookbook/6_dynamic_shared/Makefile 2021-07-09 10:33:24.808268042 +0530 ++++ samples/2_Cookbook/6_dynamic_shared/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,12 +18,17 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif + + HIPCC=$(HIP_PATH)/bin/hipcc ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) + + TARGET=hcc + +diff -ruN HIP/samples/2_Cookbook/7_streams/Makefile samples/2_Cookbook/7_streams/Makefile +--- HIP/samples/2_Cookbook/7_streams/Makefile 2021-07-09 10:33:24.808268042 +0530 ++++ samples/2_Cookbook/7_streams/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,12 +18,17 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif + + HIPCC=$(HIP_PATH)/bin/hipcc ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) + + TARGET=hcc + +diff -ruN HIP/samples/2_Cookbook/8_peer2peer/Makefile samples/2_Cookbook/8_peer2peer/Makefile +--- HIP/samples/2_Cookbook/8_peer2peer/Makefile 2021-07-09 10:33:24.808268042 +0530 ++++ samples/2_Cookbook/8_peer2peer/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,12 +18,17 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif + + HIPCC=$(HIP_PATH)/bin/hipcc ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) + + TARGET=hcc + +diff -ruN HIP/samples/2_Cookbook/9_unroll/Makefile samples/2_Cookbook/9_unroll/Makefile +--- HIP/samples/2_Cookbook/9_unroll/Makefile 2021-07-09 10:33:24.808268042 +0530 ++++ samples/2_Cookbook/9_unroll/Makefile 2021-07-09 10:28:19.000000000 +0530 +@@ -18,17 +18,22 @@ + # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + # THE SOFTWARE. + +-HIP_PATH?= $(wildcard /opt/rocm/hip) + ifeq (,$(HIP_PATH)) +- HIP_PATH=../../.. ++ HIP_PATH?= $(wildcard /opt/rocm/hip) ++ ifeq (,$(HIP_PATH)) ++ HIP_PATH=../../.. ++ endif + endif + ++HIPCC=$(HIP_PATH)/bin/hipcc ++HIP_PLATFORM=$(shell $(HIP_PATH)/bin/hipconfig --platform) ++HIP_COMPILER=$(shell $(HIP_PATH)/bin/hipconfig --compiler) ++HIP_RUNTIME=$(shell $(HIP_PATH)/bin/hipconfig --runtime) ++ + ifeq (gfx701, $(findstring gfx701,$(HCC_AMDGPU_TARGET))) + $(error gfx701 is not a supported device for this sample) + endif + +-HIPCC=$(HIP_PATH)/bin/hipcc +- + TARGET=hcc + + SOURCES = unroll.cpp diff --git a/src/hiptestsuite/applications/hip_samples/__init__.py b/src/hiptestsuite/applications/hip_samples/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/hiptestsuite/applications/hip_samples/hip_samples.py b/src/hiptestsuite/applications/hip_samples/hip_samples.py new file mode 100644 index 0000000..d375f75 --- /dev/null +++ b/src/hiptestsuite/applications/hip_samples/hip_samples.py @@ -0,0 +1,1292 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.TesterRepository import Tester, Test, TestData +from hiptestsuite.Test import HIPTestData, TestResult, HIP_PLATFORM +from typing import Union, List +from hiptestsuite.test_classifier import TestClassifier +from hiptestsuite.applications.hip_samples.hip_samples_build_amd import BuildRunAmd +from hiptestsuite.applications.hip_samples.hip_samples_build_nvidia import BuildRunNvidia +from hiptestsuite.common.hip_get_packages import HipPackages +from hiptestsuite.common.hip_shell import * +import tempfile +import os +import re +# Common class to clone, set up, build and run test +class PrepareTest(): + def __init__(self, path, binary, cwd): + self.cwdAbs = cwd + self.conformancePath = os.path.join(self.cwdAbs, "src/hiptestsuite/conformance/") + self.hippath = os.path.join(self.conformancePath, "HIP/") + self.thistestpath = os.path.join(self.hippath, path) + self.binary = binary + self.testExecOutput = None + self.hiprepo = "" # Default + self.hipbranch = "" + self.hipcommitId = "" + self.prepareobj = None + + def setrepoinfo(self, test_data: HIPTestData): + validrepconfig = True + if test_data.repos["hip"].repo_url != None: + self.hiprepo = test_data.repos["hip"].repo_url + else: + validrepconfig &= False + if test_data.repos["hip"].branch != None: + self.hipbranch = test_data.repos["hip"].branch + if test_data.repos["hip"].commit_id != None: + self.hipcommitId = test_data.repos["hip"].commit_id + + return validrepconfig + + def downloadtest(self, logFile): + ret = HipPackages().pull_repo(logFile, self.hiprepo, self.hipbranch,\ + self.hipcommitId, "HIP") + return ret + + def buildtest(self, logFile, platform, target=None): + isBinaryPresent = True + if platform == HIP_PLATFORM.nvidia: + self.prepareobj = BuildRunNvidia(self.thistestpath, logFile) + elif platform == HIP_PLATFORM.amd: + self.prepareobj = BuildRunAmd(self.thistestpath, logFile) + else: + print("Invalid Platform") + return False + self.prepareobj.buildtest(target) + + if not os.path.isfile(\ + os.path.join(self.thistestpath, self.binary)): + isBinaryPresent &= False + return isBinaryPresent + + def clean(self): + if self.prepareobj != None: + self.prepareobj.clean() + + def runtest(self, logFile): + cmdexc = "cd " + self.thistestpath + ";" + "./" + self.binary + if os.environ.get('AMD_LOG_LEVEL') is None: + self.testExecOutput = execshellcmd(cmdexc, logFile, None) + else: + runlogdump = tempfile.TemporaryFile("w+") + envtoset = os.environ.copy() + execshellcmd_largedump(cmdexc, logFile, runlogdump, envtoset) + runlogdump.seek(0) + self.testExecOutput = runlogdump.read() + +# Common class to parse the result of test execution +class LogParser(): + def __init__(self, numOfExpPassed, testPassCriteria): + self.numOfExpPassed = numOfExpPassed + self.testPassCriteria = testPassCriteria + + def parse_common(self, testResultLog): + countPass = testResultLog.count(self.testPassCriteria) + return countPass + + def utils_parser(self, testcase, testResultLog): + pt1=re.compile(r'(Batch dispatch latency:)\s+(\d+\.\d+)\s+us,\s+std:\s+(\d+\.\d+)\s+us') + pt2=re.compile(r' (total_time,)(\d+\.\d+)') + pt3=re.compile(r'(memInfo.total:\s+)(\d+\.\d+) GB') + pt4=re.compile('(Bidir_Time_pinned\s+\d+\w+\s+ms\s+)(\d+\.\d+)\s+(\d+\.\d+)\s+(\d+\.\d+)\s+(\d+\.\d+)\s+(\d+\.\d+)') + if testcase in 'hipDispatchLatency': + status=re.search(pt1,testResultLog) + if status: + #if float(str(status.group(4)))!=0.0 and float(str(status.group(2)))!=0.0 and float(str(status.group(3)))!=0.0: + return 'PASS' + else: + return'FAIL' + if testcase in 'hipCommander': + status=re.search(pt2,testResultLog) + if status: + if float(str(status.group(2)))!=0.0: + return 'PASS' + else: + return'FAIL' + else: + return 'FAIL' + if testcase in 'hipInfo': + status=re.search(pt3,testResultLog) + if status: + if float(str(status.group(2)))!=0.0: + return 'PASS' + else: + return'FAIL' + else: + return 'FAIL' + + if testcase in 'hipBusBandwidth': + status=re.search(pt4,testResultLog) + if status: + if float(str(status.group(4)))!=0.0 and float(str(status.group(2)))!=0.0 and float(str(status.group(3)))!=0.0: + return 'PASS' + else: + return'FAIL' + else: + return'FAIL' + + +class SAMPLES(TestClassifier): + def __init__(self): + TestClassifier.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + TestClassifier.add_matched_with_names(self, {"samples": matched_with_names}) + + +class INTRO(SAMPLES): + def __init__(self): + SAMPLES.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + SAMPLES.add_matched_with_names(self, {"mini-app": matched_with_names}) + +class COOKBOOK(SAMPLES): + def __init__(self): + SAMPLES.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + SAMPLES.add_matched_with_names(self, {"mini-app": matched_with_names}) + + +class UTILS(SAMPLES): + def __init__(self): + SAMPLES.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + SAMPLES.add_matched_with_names(self, {"performance": matched_with_names}) + + +# Test samples/0_Intro/bit_extract +class BitExtract(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/0_Intro/bit_extract/", "bit_extract", self.cwd) + LogParser.__init__(self, 1, "PASSED") # Number of expected PASSED + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = INTRO() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== BitExtract Test ===============") + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/BitExtract.log", 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if self.numOfExpPassed == self.parse_common(self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/0_Intro/module_api/defaultDriver.hip.out +class ModuleApiDefaultDriver(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/0_Intro/module_api/", "defaultDriver.hip.out", self.cwd) + LogParser.__init__(self, 1, "PASSED") # Number of expected PASSED + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = INTRO() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== ModuleApiDefaultDriver Test ===============") + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/ModuleApiDefaultDriver.log", 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if self.numOfExpPassed == self.parse_common(self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/0_Intro/module_api/launchKernelHcc.hip.out +# This test is skipped for NVIDIA +class ModuleApiLaunchKernelHcc(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/0_Intro/module_api/", + "launchKernelHcc.hip.out", self.cwd) + LogParser.__init__(self, 1, "PASSED") # Number of expected PASSED + self.platform = None + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = INTRO() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + if self.platform != HIP_PLATFORM.nvidia: + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== ModuleApiLaunchKernelHcc Test ===============") + # Set repo info + self.platform = test_data.HIP_PLATFORM + if test_data.HIP_PLATFORM == HIP_PLATFORM.nvidia: + test_data.test_result = TestResult.SKIP + else: + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/ModuleApiLaunchKernelHcc.log", 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if self.numOfExpPassed == self.parse_common(self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/0_Intro/module_api/runKernel.hip.out +class ModuleApiRunKernel(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/0_Intro/module_api/", "runKernel.hip.out", self.cwd) + LogParser.__init__(self, 1, "PASSED") # Number of expected PASSED + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = INTRO() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== ModuleApiRunKernel Test ===============") + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/ModuleApiRunKernel.log", 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if self.numOfExpPassed == self.parse_common(self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/0_Intro/module_api_global +class ModuleApiGlobal(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/0_Intro/module_api_global/", + "runKernel.hip.out", self.cwd) + LogParser.__init__(self, 2, "PASSED") # Number of expected PASSED + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = INTRO() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== ModuleApiGlobal Test ===============") + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/ModuleApiGlobal.log", 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if self.numOfExpPassed == self.parse_common(self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/0_Intro/square/ +class Square(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/0_Intro/square/", "square.out", self.cwd) + LogParser.__init__(self, 1, "PASSED") # Number of expected PASSED + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + intro = INTRO() + intro.add_matched_with_names() + test.classifiers = [intro] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== Square Test ===============") + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/Square.log", 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if self.numOfExpPassed == self.parse_common(self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/2_Cookbook/0_MatrixTranspose +class MatrixTranspose(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/2_Cookbook/0_MatrixTranspose/", + "MatrixTranspose", self.cwd) + LogParser.__init__(self, 1, "PASSED") # Number of expected PASSED + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + cookbook = COOKBOOK() + cookbook.add_matched_with_names() + test.classifiers = [cookbook] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== MatrixTranspose Test ===============") + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + if not os.path.exists(resultLogDir): + os.makedirs(resultLogDir) + with open(resultLogDir + '/MatrixTranspose.log', 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if self.numOfExpPassed == self.parse_common(self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/2_Cookbook/10_inline_asm +# This test is skipped for NVIDIA +class InlineAsm(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/2_Cookbook/10_inline_asm/", "inline_asm", self.cwd) + LogParser.__init__(self, 1, "PASSED") # Number of expected PASSED + self.platform = None + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + cookbook = COOKBOOK() + cookbook.add_matched_with_names() + test.classifiers = [cookbook] + test.tester = self + return [test] + + def clean(self): + if self.platform != HIP_PLATFORM.nvidia: + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== InlineAsm Test ===============") + # Set repo info + self.platform = test_data.HIP_PLATFORM + if test_data.HIP_PLATFORM == HIP_PLATFORM.nvidia: + test_data.test_result = TestResult.SKIP + else: + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + '/InlineAsm.log', 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if self.numOfExpPassed == self.parse_common(self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/2_Cookbook/16_assembly_to_executable +# This test is skipped for nvidia +class SquareAsm(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/2_Cookbook/16_assembly_to_executable/", + "square_asm.out", self.cwd) + LogParser.__init__(self, 1, "PASSED") # Number of expected PASSED + self.platform = None + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + cookbook = COOKBOOK() + cookbook.add_matched_with_names() + test.classifiers = [cookbook] + test.tester = self + return [test] + + def clean(self): + if self.platform != HIP_PLATFORM.nvidia: + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== SquareAsm Test ===============") + self.platform = test_data.HIP_PLATFORM + if test_data.HIP_PLATFORM == HIP_PLATFORM.nvidia: + test_data.test_result = TestResult.SKIP + else: + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + '/SquareAsm.log', 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if self.numOfExpPassed == self.parse_common(self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/2_Cookbook/11_texture_driver +class Texture2dDrv(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/2_Cookbook/11_texture_driver/", + "texture2dDrv.out", self.cwd) + LogParser.__init__(self, 1, "PASSED") # Number of expected PASSED + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + cookbook = COOKBOOK() + cookbook.add_matched_with_names() + test.classifiers = [cookbook] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== Texture2dDrv Test ===============") + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + '/Texture2dDrv.log', 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if self.numOfExpPassed == self.parse_common(self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/2_Cookbook/9_unroll +class Unroll(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/2_Cookbook/9_unroll/", "unroll", self.cwd) + LogParser.__init__(self, 1, "PASSED") # Number of expected PASSED + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + cookbook = COOKBOOK() + cookbook.add_matched_with_names() + test.classifiers = [cookbook] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== Unroll Test ===============") + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + '/Unroll.log', 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if self.numOfExpPassed == self.parse_common(self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/2_Cookbook/4_shfl +class Shfl(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/2_Cookbook/4_shfl/", "shfl", self.cwd) + LogParser.__init__(self, 1, "PASSED") # Number of expected PASSED + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + cookbook = COOKBOOK() + cookbook.add_matched_with_names() + test.classifiers = [cookbook] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== Shfl Test ===============") + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + '/Shfl.log', 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if self.numOfExpPassed == self.parse_common(self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/2_Cookbook/3_shared_memory +class SharedMemory(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/2_Cookbook/3_shared_memory/", "sharedMemory", self.cwd) + LogParser.__init__(self, 1, "PASSED") # Number of expected PASSED + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + cookbook = COOKBOOK() + cookbook.add_matched_with_names() + test.classifiers = [cookbook] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== SharedMemory Test ===============") + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + '/SharedMemory.log', 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if self.numOfExpPassed == self.parse_common(self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/2_Cookbook/17_llvm_ir_to_executable +# This test is skipped for nvidia +class Square_Ir(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/2_Cookbook/17_llvm_ir_to_executable/", + "square_ir.out", self.cwd) + LogParser.__init__(self, 1, "PASSED") # Number of expected PASSED + self.platform = None + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + cookbook = COOKBOOK() + cookbook.add_matched_with_names() + test.classifiers = [cookbook] + test.tester = self + return [test] + + def clean(self): + if self.platform != HIP_PLATFORM.nvidia: + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== Square_Ir Test ===============") + self.platform = test_data.HIP_PLATFORM + if test_data.HIP_PLATFORM == HIP_PLATFORM.nvidia: + test_data.test_result = TestResult.SKIP + else: + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + '/Square_Ir.log', 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if self.numOfExpPassed == self.parse_common(self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/2_Cookbook/5_2dshfl +class Dshfl(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/2_Cookbook/5_2dshfl/", "2dshfl", self.cwd) + LogParser.__init__(self, 1, "PASSED") # Number of expected PASSED + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + cookbook = COOKBOOK() + cookbook.add_matched_with_names() + test.classifiers = [cookbook] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== Dshfl Test ===============") + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + '/Dshfl.log', 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if self.numOfExpPassed == self.parse_common(self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/2_Cookbook/1_hipEvent +class HipEvent(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/2_Cookbook/1_hipEvent/", "hipEvent", self.cwd) + LogParser.__init__(self, 1, "PASSED") # Number of expected PASSED + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + cookbook = COOKBOOK() + cookbook.add_matched_with_names() + test.classifiers = [cookbook] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== HipEvent Test ===============") + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + '/HipEvent.log', 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if self.numOfExpPassed == self.parse_common(self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/2_Cookbook/6_dynamic_shared +class Dynamic_Shared(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/2_Cookbook/6_dynamic_shared/", + "dynamic_shared", self.cwd) + LogParser.__init__(self, 1, "PASSED") # Number of expected PASSED + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + cookbook = COOKBOOK() + cookbook.add_matched_with_names() + test.classifiers = [cookbook] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== Dynamic_Shared Test ===============") + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + '/Dynamic_Shared.log', 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if self.numOfExpPassed == self.parse_common(self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/2_Cookbook/8_peer2peer +class Peer2peer(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/2_Cookbook/8_peer2peer/", "peer2peer", self.cwd) + LogParser.__init__(self, 1, "PASSED") # Number of expected PASSED + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + cookbook = COOKBOOK() + cookbook.add_matched_with_names() + test.classifiers = [cookbook] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== Peer2peer Test ===============") + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + '/Peer2peer.log', 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if self.numOfExpPassed == self.parse_common(self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/2_Cookbook/7_streams +class Stream(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/2_Cookbook/7_streams/", "stream", self.cwd) + LogParser.__init__(self, 1, "PASSED") # Number of expected PASSED + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + cookbook = COOKBOOK() + cookbook.add_matched_with_names() + test.classifiers = [cookbook] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== Stream Test ===============") + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + '/Stream.log', 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if self.numOfExpPassed == self.parse_common(self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/2_Cookbook/13_occupancy +class Occupancy(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/2_Cookbook/13_occupancy/", "occupancy", self.cwd) + LogParser.__init__(self, 2, "PASSED") # Number of expected PASSED + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + cookbook = COOKBOOK() + cookbook.add_matched_with_names() + test.classifiers = [cookbook] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== Occupancy Test ===============") + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + '/Occupancy.log', 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if self.numOfExpPassed == self.parse_common(self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/1_Utils/hipBusBandwidth +class HipBusBandwidth(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/1_Utils/hipBusBandwidth/", "hipBusBandwidth", self.cwd) + LogParser.__init__(self, 2, "PASSED") # Number of expected PASSED + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + utils = UTILS() + utils.add_matched_with_names() + test.classifiers = [utils] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== HipBusBandwidth Test ===============") + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + '/hipBusBandwidth.log', 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if "PASS" == self.utils_parser("hipBusBandwidth", self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/1_Utils/hipCommander +class HipCommander(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/1_Utils/hipCommander/", "hipCommander", self.cwd) + LogParser.__init__(self, 2, "PASSED") # Number of expected PASSED + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + utils = UTILS() + utils.add_matched_with_names() + test.classifiers = [utils] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== HipCommander Test ===============") + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + '/hipCommander.log', 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if "PASS" == self.utils_parser("hipCommander", self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/1_Utils/hipDispatchLatency +class HipDispatchLatency(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/1_Utils/hipDispatchLatency/", "hipDispatchLatency.out", self.cwd) + LogParser.__init__(self, 2, "PASSED") # Number of expected PASSED + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + utils = UTILS() + utils.add_matched_with_names() + test.classifiers = [utils] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== HipDispatchLatency Test ===============") + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + '/hipDispatchLatency.log', 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if "PASS" == self.utils_parser("hipDispatchLatency", self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test samples/1_Utils/hipInfo +class HipInfo(Tester, PrepareTest, LogParser): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, "samples/1_Utils/hipInfo/", "hipInfo", self.cwd) + LogParser.__init__(self, 2, "PASSED") # Number of expected PASSED + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + utils = UTILS() + utils.add_matched_with_names() + test.classifiers = [utils] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== HipInfo Test ===============") + # Set repo info + isrepocfgvalid = self.setrepoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + '/hipInfo.log', 'w+') as testLogger: + res = self.downloadtest(testLogger) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, "hipInfo") + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(testLogger) + if "PASS" == self.utils_parser("hipInfo", self.testExecOutput): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL diff --git a/src/hiptestsuite/applications/hip_samples/hip_samples_build_amd.py b/src/hiptestsuite/applications/hip_samples/hip_samples_build_amd.py new file mode 100644 index 0000000..ba02894 --- /dev/null +++ b/src/hiptestsuite/applications/hip_samples/hip_samples_build_amd.py @@ -0,0 +1,36 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os +from hiptestsuite.applications.hip_samples.hip_samples_build_common import BuildRunCommon + +class BuildRunAmd(BuildRunCommon): + def __init__(self, path, logfile): + BuildRunCommon.__init__(self, path, logfile) + + def buildtest(self, target): + # In this function put the build steps for test cases + # which differ across platforms (amd/nvidia/intel) else + # invoke BuildRunCommon.buildtest + if not os.path.exists("/opt/rocm"): + print("ROCm not installed. Exiting!") + return False + ret = BuildRunCommon.buildtest(self, target) + return ret diff --git a/src/hiptestsuite/applications/hip_samples/hip_samples_build_common.py b/src/hiptestsuite/applications/hip_samples/hip_samples_build_common.py new file mode 100644 index 0000000..488c2d9 --- /dev/null +++ b/src/hiptestsuite/applications/hip_samples/hip_samples_build_common.py @@ -0,0 +1,47 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os +from hiptestsuite.common.hip_shell import * + +class BuildRunCommon(): + ''' + In this class insert the build and execution steps for test cases + which are identical across different platforms (amd/nvidia/intel). + ''' + def __init__(self, path, logfile): + self.thistestpath = path + self.logfile = logfile + + def buildtest(self, target, env = None): + cmdcd = "cd " + self.thistestpath + ";" + cmd_clean = "make clean;" + if target != None: + cmd_build = "make " + target + ";" + else: + cmd_build = "make;" + cmdexc = cmdcd + cmd_clean + cmd_build + execshellcmd(cmdexc, self.logfile, env) + + def clean(self): + cmdcd = "cd " + self.thistestpath + ";" + cmd_clean = "make clean;" + cmdexc = cmdcd + cmd_clean + execshellcmd(cmdexc, None, None) diff --git a/src/hiptestsuite/applications/hip_samples/hip_samples_build_nvidia.py b/src/hiptestsuite/applications/hip_samples/hip_samples_build_nvidia.py new file mode 100644 index 0000000..7920c91 --- /dev/null +++ b/src/hiptestsuite/applications/hip_samples/hip_samples_build_nvidia.py @@ -0,0 +1,58 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os +import tempfile +from hiptestsuite.common.hip_shell import * +from hiptestsuite.applications.hip_samples.hip_samples_build_common import BuildRunCommon + +class BuildRunNvidia(BuildRunCommon): + def __init__(self, path, logfile): + self.hippath = os.path.join(os.getcwd(), "src/hiptestsuite/conformance/HIP/") + BuildRunCommon.__init__(self, path, logfile) + + def getenvironmentvariables(self): + envtoset = os.environ.copy() + envtoset["HIP_PLATFORM"] = "nvidia" + envtoset["HIP_COMPILER"] = "nvcc" + envtoset["HIP_RUNTIME"] = "cuda" + envtoset["HIP_PATH"] = "/opt/rocm/hip" + return envtoset + + def applypatch(self): # To be deleted + if not os.path.isfile(\ + os.path.join(self.hippath, "patched")): + cmd = "cd " + self.hippath + ";" + cmd += "patch -p0 < ../../applications/hip_samples/Samples_Patch_4.2.x; touch patched;" + execshellcmd(cmd, self.logfile, None) + + def buildtest(self, target): + # In this function put the build steps for test cases + # which differ across platforms (testsuite/nvidia/intel) else + # invoke BuildRunCommon.buildtest + if not os.path.exists("/opt/rocm"): + print("ROCm not installed. Exiting!") + return False + # Apply Patch. This is temporary and will be removed once Hip Samples changes + # are available in HIP public repository. + envtoset = self.getenvironmentvariables() + self.applypatch() + ret = BuildRunCommon.buildtest(self, target, envtoset) + return ret diff --git a/src/hiptestsuite/applications/hpc_apps/__init__.py b/src/hiptestsuite/applications/hpc_apps/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/hiptestsuite/applications/hpc_apps/cp2k/__init__.py b/src/hiptestsuite/applications/hpc_apps/cp2k/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/hiptestsuite/applications/hpc_apps/cp2k/cp2k.py b/src/hiptestsuite/applications/hpc_apps/cp2k/cp2k.py new file mode 100644 index 0000000..dfa0ae6 --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/cp2k/cp2k.py @@ -0,0 +1,181 @@ +# Copyright (c) 2023 Advanced Micro Devices, Inc. All rights reserved. +# Copyright (c) 2023 Intel Finland Oy. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.TesterRepository import Tester, Test, TestData +from hiptestsuite.Test import HIPTestData, TestResult, HIP_PLATFORM +from typing import Union, List +from hiptestsuite.test_classifier import TestClassifier +from hiptestsuite.applications.hpc_apps.cp2k.cp2k_build_amd import BuildRunAmd +from hiptestsuite.common.hip_get_packages import HipPackages +from hiptestsuite.common.hip_shell import execshellcmd_largedump + +import os +import re +import tempfile + +# Common class to clone, set up, build and run test +class PrepareTest(): + def __init__(self, cwd): + self.cwdAbs = cwd + self.app_path = os.path.join(self.cwdAbs, + "src/hiptestsuite/applications/hpc_apps/", + "cp2k/") + self.thistestpath = self.app_path + self.prepareobj = None + self.apprepo = "" # Default + self.appbranch = "" + self.appcommitId = "" + self.rocm_path = os.environ.get('ROCM_PATH') + if not self.rocm_path: + self.rocm_path = "/opt/rocm" + self.gpu_version = None + + def pick_gpu(self): + """Picks a GPU supported by the CP2K""" + if self.gpu_version: + return self.gpu_version + + selected_gpu = None + cmd = self.rocm_path + "/bin/rocm_agent_enumerator" + with tempfile.TemporaryFile("w+") as output: + ret_code = execshellcmd_largedump(cmd, None, output, None) + if ret_code != 0: + return None + output.seek(0) + for line in output: + # See cp2k/tools/toolchain/scripts/generate_arch_files.sh + # for supported GPUs. + if line.startswith("gfx90a"): + selected_gpu = "Mi250" + elif line.startswith("gfx908"): + selected_gpu = "Mi100" + elif line.startswith("gfx906"): + selected_gpu = "Mi50" + + self.gpu_version = selected_gpu + return selected_gpu + + def set_cp2k_repoinfo(self, test_data: HIPTestData): + validrepconfig = True + if test_data.repos["cp2k"].repo_url != None: + self.apprepo = test_data.repos["cp2k"].repo_url + else: + validrepconfig &= False + if test_data.repos["cp2k"].branch != None: + self.appbranch = test_data.repos["cp2k"].branch + if test_data.repos["cp2k"].commit_id != None: + self.appcommitId = test_data.repos["cp2k"].commit_id + return validrepconfig + + def downloadtest(self, logFile, test_data: HIPTestData): + ret = HipPackages().pull_repo(logFile, self.apprepo, + self.appbranch, self.appcommitId, "cp2k") + return ret + + def buildtest(self, logFile, platform): + if platform == HIP_PLATFORM.amd: + self.prepareobj = BuildRunAmd(self.thistestpath, logFile, + self.rocm_path, self.pick_gpu()) + else: + print("Unsupported Platform") + return False + return self.prepareobj.buildtest() == True + + def clean(self): + if self.prepareobj != None: + self.prepareobj.clean() + + def runtest(self, testnum): + if self.prepareobj != None: + self.prepareobj.runtest(testnum) + + def parse_result(self, testnum): + if self.prepareobj != None: + return self.prepareobj.parse_result(testnum) + return False + +class CP2K(TestClassifier): + def __init__(self): + TestClassifier.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + TestClassifier.add_matched_with_names(self, {"cp2k": matched_with_names}) + +class AllRegressionTests(CP2K): + def __init__(self): + CP2K.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + CP2K.add_matched_with_names(self, {"all-tests": matched_with_names}) + +# Cp2k Unit/Regression Test +class CP2K_UNIT_TEST(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + classifier = AllRegressionTests() + classifier.add_matched_with_names() + test.classifiers = [classifier] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== CP2K Unit test ===============") + if test_data.HIP_PLATFORM != HIP_PLATFORM.amd: + print("CP2K test is not supported on NVIDIA") + test_data.test_result = TestResult.SKIP + return + if not self.pick_gpu(): + print("Could not find supported GPU version.") + test_data.test_result = TestResult.SKIP + return + + # Set repo info + isrepocfgvalid = self.set_cp2k_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/cp2k_unit.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(0) + # Parse the test result + if True == self.parse_result(0): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + diff --git a/src/hiptestsuite/applications/hpc_apps/cp2k/cp2k_build.bash b/src/hiptestsuite/applications/hpc_apps/cp2k/cp2k_build.bash new file mode 100644 index 0000000..7d1f03b --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/cp2k/cp2k_build.bash @@ -0,0 +1,9 @@ +#!/bin/bash +set -eu +cd ${CP2K_ROOT} +rm -f build.stamp +set +eu +source ./tools/toolchain/install/setup +set -eu +make -j${JOB_COUNT:-1} ARCH=local_hip VERSION=ssmp +touch build.stamp diff --git a/src/hiptestsuite/applications/hpc_apps/cp2k/cp2k_build_amd.py b/src/hiptestsuite/applications/hpc_apps/cp2k/cp2k_build_amd.py new file mode 100644 index 0000000..85cabd8 --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/cp2k/cp2k_build_amd.py @@ -0,0 +1,124 @@ +# Copyright (c) 2023 Advanced Micro Devices, Inc. All rights reserved. +# Copyright (c) 2023 Intel Finland Oy. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os +import re +import tempfile +from multiprocessing import cpu_count + +from hiptestsuite.common.hip_shell import * +from hiptestsuite.applications.hpc_apps.cp2k.cp2k_parser_common import Cp2kParser + +class BuildRunAmd(): + def __init__(self, thistestpath, logFile, rocm_path, gpu_version): + self.thistestpath = thistestpath + self.logFile = logFile + self.runlog = None + self.rocm_path = rocm_path + self.gpu_version = gpu_version + + def configure_build_cp2k(self): + + if not os.path.exists(self.rocm_path): + print("Error: ROCm path does not exist. Exiting Test!") + print("NOTE: tried path '{}'".format(self.rocm_path)) + return False + + repo_root = os.path.join(self.thistestpath, "cp2k") + + submodule_cmd = """ +set -e +cd {repo_root} +git submodule update --init --recursive +""".format(repo_root=repo_root) + print("cp2k submodule fetching in progress ..") + with tempfile.TemporaryFile("w+") as runlogdump: + retcode = execshellcmd_largedump(submodule_cmd, self.logFile, + runlogdump, None) + if retcode != 0: + print("cp2k submodule fetch Failed", ) + return False + + configure_cmd = """ +set -e +export ROCM_PATH={rocm_path} +export LIBRARY_PATH=$ROCM_PATH/lib:$LIBRARY_PATH +cd {repo_root} +rm -f configure.stamp +cd tools/toolchain +./install_cp2k_toolchain.sh --enable-hip --with-cmake=system \ +--with-libint=system --with-libxc=system --with-fftw=system \ +--with-openblas=system --with-sirius=no --with-gsl=no \ +--gpu-ver={gpu_version} +cd {repo_root} +cp tools/toolchain/install/arch/* arch/ +touch configure.stamp +""".format(repo_root=repo_root, rocm_path=self.rocm_path, + gpu_version=self.gpu_version) + print("cp2k dependency fetch and configuration in progress ..") + with tempfile.TemporaryFile("w+") as runlogdump: + execshellcmd_largedump(configure_cmd, self.logFile, runlogdump, + None) + if not os.path.exists(repo_root + "/configure.stamp"): + print("cp2k dependency fetch and/or configuration Failed", ) + return False + + # The build step relies on bash shell. + build_script = os.path.join(self.thistestpath, "cp2k_build.bash") + build_cmd = "/bin/bash " + build_script + build_env = os.environ.copy() + build_env['CP2K_ROOT'] = str(repo_root) + build_env['JOB_COUNT'] = str(cpu_count()) + print("cp2k build in progress ..") + with tempfile.TemporaryFile("w+") as runlogdump: + execshellcmd_largedump(build_cmd, self.logFile, runlogdump, + build_env) + if not os.path.exists(repo_root + "/build.stamp"): + print("cp2k build Failed", ) + return False + return True + + def buildtest(self): + if not self.configure_build_cp2k(): + return False + return True + + def runtest(self, testnum): + print("Running cp2k Test" + str(testnum)) + repo_root = os.path.join(self.thistestpath, "cp2k") + + cmd = """ +set -e +export LD_LIBRARY_PATH={rocm_path}/lib:$LD_LIBRARY_PATH +cd {repo_root} +make -j{job_count} TESTOPTS="--skipdir QS/regtest-rs-dhft --skipdir QS/regtest-hfx-ri" ARCH=local_hip VERSION=ssmp test +""".format(repo_root=repo_root, job_count=cpu_count(), rocm_path=self.rocm_path) + + self.runlog = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmd, self.logFile, self.runlog, None) + + def clean(self): + print("Cleaning cp2k..") + if self.runlog != None: + self.runlog.close() + + def parse_result(self, testnum): + return Cp2kParser(self.runlog).parse(testnum) diff --git a/src/hiptestsuite/applications/hpc_apps/cp2k/cp2k_parser_common.py b/src/hiptestsuite/applications/hpc_apps/cp2k/cp2k_parser_common.py new file mode 100644 index 0000000..e7838a0 --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/cp2k/cp2k_parser_common.py @@ -0,0 +1,35 @@ +# Copyright (c) 2023 Advanced Micro Devices, Inc. All rights reserved. +# Copyright (c) 2023 Intel Finland Oy. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import re + +class Cp2kParser(): + def __init__(self, results): + results.seek(0) + logbytes = results.read() + self.results = logbytes + + def parse(self, testnum): + no_failed_tests = re.search("Number of FAILED \s*tests 0", self.results, + re.MULTILINE) is not None + no_wrong_tests = re.search("Number of WRONG \s*tests 0", self.results, + re.MULTILINE) is not None + return no_failed_tests and no_wrong_tests diff --git a/src/hiptestsuite/applications/hpc_apps/gridtools/__init__.py b/src/hiptestsuite/applications/hpc_apps/gridtools/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/hiptestsuite/applications/hpc_apps/gridtools/gridtools.patch b/src/hiptestsuite/applications/hpc_apps/gridtools/gridtools.patch new file mode 100644 index 0000000..b18427f --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/gridtools/gridtools.patch @@ -0,0 +1,22 @@ +diff --git a/cmake/definitions.cmake b/cmake/definitions.cmake +index 8c49b56d..cd3743bd 100644 +--- a/cmake/definitions.cmake ++++ b/cmake/definitions.cmake +@@ -61,6 +61,17 @@ if(CUDA_AVAILABLE) + elseif(GT_CUDA_COMPILATION_TYPE MATCHES "HIPCC-AMDGPU") + set(hipcc_options_ -xhip --amdgpu-target=${GT_CUDA_ARCH}) + target_compile_options(gridtools INTERFACE $<$:${hipcc_options_}>) ++ target_compile_definitions(gridtools INTERFACE GT_USE_HIP) ++ elseif(GT_CUDA_COMPILATION_TYPE MATCHES "HIPCC-NVCC") ++ ++ # allow to call constexpr __host__ from constexpr __device__, e.g. call std::max in constexpr context ++ target_compile_options(gridtools INTERFACE ++ $<$:--expt-relaxed-constexpr>) ++ ++ if(${GT_CXX_STANDARD} STREQUAL "c++17") ++ message(FATAL_ERROR "c++17 is not supported for CUDA compilation") ++ endif() ++ + target_compile_definitions(gridtools INTERFACE GT_USE_HIP) + endif() + diff --git a/src/hiptestsuite/applications/hpc_apps/gridtools/gridtools.py b/src/hiptestsuite/applications/hpc_apps/gridtools/gridtools.py new file mode 100644 index 0000000..c3d101a --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/gridtools/gridtools.py @@ -0,0 +1,237 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.TesterRepository import Tester, Test, TestData +from hiptestsuite.Test import HIPTestData, TestResult, HIP_PLATFORM +from typing import Union, List +from hiptestsuite.test_classifier import TestClassifier +from hiptestsuite.applications.hpc_apps.gridtools.gridtools_build_amd import BuildRunAmd +from hiptestsuite.applications.hpc_apps.gridtools.gridtools_build_nvidia import BuildRunNvidia +from hiptestsuite.common.hip_get_packages import HipPackages +from hiptestsuite.common.hip_shell import execshellcmd + +import os +import re +# Common class to clone, set up, build and run test +class PrepareTest(): + def __init__(self, cwd): + self.cwdAbs = cwd + self.app_path = os.path.join(self.cwdAbs,\ + "src/hiptestsuite/applications/hpc_apps/gridtools") + self.thistestpath = self.app_path + self.prepareobj = None + self.gridtoolsrepo = "" # Default + self.gridtoolsbranch = "" + self.gridtoolscommitId = "" + self.gtbenchrepo = "" # Default + self.gtbenchbranch = "" + self.gtbenchcommitId = "" + + def set_gridtools_repoinfo(self, test_data: HIPTestData): + validrepconfig = True + if test_data.repos["gridtools"].repo_url != None: + self.gridtoolsrepo = test_data.repos["gridtools"].repo_url + else: + validrepconfig &= False + if test_data.repos["gridtools"].branch != None: + self.gridtoolsbranch = test_data.repos["gridtools"].branch + if test_data.repos["gridtools"].commit_id != None: + self.gridtoolscommitId = test_data.repos["gridtools"].commit_id + return validrepconfig + + def set_gtbench_repoinfo(self, test_data: HIPTestData): + validrepconfig = True + if test_data.repos["gtbench"].repo_url != None: + self.gtbenchrepo = test_data.repos["gtbench"].repo_url + else: + validrepconfig &= False + if test_data.repos["gtbench"].branch != None: + self.gtbenchbranch = test_data.repos["gtbench"].branch + if test_data.repos["gtbench"].commit_id != None: + self.gtbenchcommitId = test_data.repos["gtbench"].commit_id + return validrepconfig + + def downloadtest(self, logFile, test_data: HIPTestData): + ret = True + ret = ret & HipPackages().pull_repo(logFile, self.gridtoolsrepo,\ + self.gridtoolsbranch, self.gridtoolscommitId, "gridtools") + ret = ret & HipPackages().pull_repo(logFile, self.gtbenchrepo,\ + self.gtbenchbranch, self.gtbenchcommitId, "gtbench") + return ret + + def buildtest(self, logFile, platform, cuda_target): + isBinaryPresent = True + if platform == HIP_PLATFORM.amd: + self.prepareobj = BuildRunAmd(self.thistestpath, logFile) + elif platform == HIP_PLATFORM.nvidia: + self.prepareobj = BuildRunNvidia(self.thistestpath, logFile, cuda_target) + else: + print("Invalid Platform") + return False + buildstatus = self.prepareobj.buildtest() + return buildstatus + + def clean(self): + if self.prepareobj != None: + self.prepareobj.clean() + + def runtest(self, testnum): + if self.prepareobj != None: + self.prepareobj.runtest(testnum) + + def parse_result(self, testnum): + if self.prepareobj != None: + return self.prepareobj.parse_result(testnum) + return False + +class GDTOOLS(TestClassifier): + def __init__(self): + TestClassifier.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + TestClassifier.add_matched_with_names(self, {"gridtools": matched_with_names}) + +class PERFORMANCE(GDTOOLS): + def __init__(self): + GDTOOLS.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + GDTOOLS.add_matched_with_names(self, {"performance": matched_with_names}) + +# Gridtool Convergence Test +class GRIDTOOLSCONVG(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + classifier = PERFORMANCE() + classifier.add_matched_with_names() + test.classifiers = [classifier] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== Gridtool Convergence Test ===============") + # Check if Boost package exists + if not os.path.isfile(\ + os.path.join(self.thistestpath, "boost_1_72_0.tar.bz2")): + print("Boost Package boost_1_72_0.tar.bz2 not available under src/hiptestsuite/applications/hpc_apps/gridtools/.") + print("Please download and copy Boost package in src/hiptestsuite/applications/hpc_apps/gridtools folder.") + test_data.test_result = TestResult.ERROR + return + # Create GT_TREE_DIR + gt_tree_dir = os.path.join(self.thistestpath, "GridTools") + if not os.path.exists(gt_tree_dir): + os.mkdir(gt_tree_dir) + # Set repo info + isrepocfgvalid = self.set_gridtools_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + isrepocfgvalid = self.set_gtbench_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/gridtoolsconv.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, test_data.build_for_cuda_target) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(0) + # Parse the test result + if True == self.parse_result(0): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + +# Gridtool PERFORMANCE Test +class GRIDTOOLSPERF(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + classifier = PERFORMANCE() + classifier.add_matched_with_names() + test.classifiers = [classifier] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== Gridtool Perf Test ===============") + # Check if Boost package exists + if not os.path.isfile(\ + os.path.join(self.thistestpath, "boost_1_72_0.tar.bz2")): + print("Boost Package boost_1_72_0.tar.bz2 not available under src/hiptestsuite/applications/hpc_apps/gridtools/.") + print("Please download and copy Boost package in src/hiptestsuite/applications/hpc_apps/gridtools folder.") + test_data.test_result = TestResult.ERROR + return + + # Create GT_TREE_DIR + gt_tree_dir = os.path.join(self.thistestpath, "GridTools") + if not os.path.exists(gt_tree_dir): + os.mkdir(gt_tree_dir) + + # Set repo info + isrepocfgvalid = self.set_gridtools_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + isrepocfgvalid = self.set_gtbench_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/gridtoolsperf.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, test_data.build_for_cuda_target) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(1) + # Parse the test result + if True == self.parse_result(1): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL diff --git a/src/hiptestsuite/applications/hpc_apps/gridtools/gridtools_build_amd.py b/src/hiptestsuite/applications/hpc_apps/gridtools/gridtools_build_amd.py new file mode 100644 index 0000000..6cb7594 --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/gridtools/gridtools_build_amd.py @@ -0,0 +1,124 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os +import tempfile +from hiptestsuite.common.hip_shell import * +from hiptestsuite.applications.hpc_apps.gridtools.gridtools_parser_common import GridtoolsParser + +class BuildRunAmd(): + def __init__(self, thistestpath, logFile): + self.thistestpath = thistestpath + self.logFile = logFile + self.runlog = None + + def setenv(self, gpu_arch): + env = "export HIP_PLATFORM=`/opt/rocm/bin/hipconfig --platform`;" + env += "export GT_CUDA_COMPILATION_TYPE=HIPCC-AMDGPU;" + env += "export HCC_AMDGPU_TARGET=\""+ gpu_arch +"\";" + env += "export ROCMHOME=/opt/rocm;" + env += "export PATH=$ROCMHOME/bin:$ROCMHOME/llvm/bin:$ROCMHOME/hip/bin:$ROCMHOME/opencl/bin:$ROCMHOME/rocprofiler/bin:$PATH;" + env += "export LD_LIBRARY_PATH=/$ROCMHOME/lib:$ROCMHOME/llvm/lib:$LD_LIBRARY_PATH;" + env += "export GT_ALL_DIR=$PWD/src/hiptestsuite/applications/hpc_apps/gridtools;" + env += "export GT_TREE_DIR=$GT_ALL_DIR/GridTools;" + env += "export BOOST_TREE_DIR=$GT_TREE_DIR/boost_1_72_0;" + env += "export BOOST_INSTALL_DIR=$GT_ALL_DIR/boost_1_72_0;" + env += "export GRIDTOOLS_TREE_DIR=$GT_TREE_DIR/gridtools;" + env += "export GRIDTOOLS_BUILD_DIR=$GRIDTOOLS_TREE_DIR/build;" + env += "export GRIDTOOLS_INSTALL_DIR=$GT_ALL_DIR/gridtools;" + env += "export GTBENCH_TREE_DIR=$GT_TREE_DIR/gtbench;" + env += "export GTBENCH_BUILD_DIR=$GTBENCH_TREE_DIR/build;" + return env + + def buildtest(self): + # In this function put the build steps for test cases + # which differ across platforms (amd/nvidia/intel) + # Build Boost + gpuarch = get_gpuarch(self.logFile) + if gpuarch is None: + return False + + if not os.path.exists(os.path.join(self.thistestpath, "boost_1_72_0")): + print("Building and Installing Boost..") + cmdexc = self.setenv(gpuarch) + cmdexc += "cd $GT_TREE_DIR;" + cmdexc += "tar -xvjf ../boost_1_72_0.tar.bz2;cd $BOOST_TREE_DIR;" + cmdexc += "./bootstrap.sh --prefix=$BOOST_INSTALL_DIR --with-python=python3;" + cmdexc += "./b2 install -j8 threading=multi link=shared;" + runlogdump = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, runlogdump, None) + runlogdump.close() + else: + print("Boost already installed..") + # Build Gridtools + if not os.path.exists(os.path.join(self.thistestpath, "GridTools/gridtools/build")): + print("Building and Installing GridTools..") + cmdexc = self.setenv(gpuarch) + cmdexc += "export CXX=/opt/rocm/bin/hipcc;" + cmdexc += "cd $GT_TREE_DIR;cd $GRIDTOOLS_TREE_DIR;git apply ../../gridtools.patch;mkdir -p $GRIDTOOLS_BUILD_DIR;cd $GRIDTOOLS_BUILD_DIR;" + cmdexc += "CXX=/opt/rocm/bin/hipcc cmake .. -DBUILD_TESTING=OFF -DBoost_INCLUDE_DIR=$BOOST_INSTALL_DIR/include " +\ + "-DGT_CUDA_COMPILATION_TYPE=$GT_CUDA_COMPILATION_TYPE -DGT_CUDA_ARCH=" + gpuarch + " " +\ + "-DGT_ENABLE_BACKEND_CUDA=ON -DGT_ENABLE_BACKEND_MC=OFF -DGT_ENABLE_BACKEND_X86=OFF -DGT_ENABLE_BACKEND_NAIVE=OFF " +\ + "-DGT_USE_MPI=OFF -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$GRIDTOOLS_INSTALL_DIR;" + cmdexc += "make -j;" + cmdexc += "make install;" + runlogdump = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, runlogdump, None) + runlogdump.close() + else: + print("GridTools already built..") + + # Build Gtbench + if not os.path.exists(os.path.join(self.thistestpath, "GridTools/gtbench/build")): + print("Building and Installing Gtbench..") + cmdexc = self.setenv(gpuarch) + cmdexc += "export CXX=/opt/rocm/bin/hipcc;" + cmdexc += "cd $GT_TREE_DIR;cd $GTBENCH_TREE_DIR;git apply ../../gtbench.patch;mkdir -p $GTBENCH_BUILD_DIR;cd $GTBENCH_BUILD_DIR;" + cmdexc += "CXX=/opt/rocm/bin/hipcc cmake .. -DGridTools_DIR=$GRIDTOOLS_INSTALL_DIR/lib/cmake -DGTBENCH_BACKEND=cuda " +\ + "-DGTBENCH_RUNTIME=single_node -DCMAKE_CXX_FLAGS=-D__HIPCC__ -DBoost_INCLUDE_DIR=$BOOST_INSTALL_DIR/include;" + cmdexc += "make -j8;" + runlogdump = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, runlogdump, None) + runlogdump.close() + else: + print("Gtbench already built..") + + return True + + def runtest(self, testnum): + cmdcd = "cd " + os.path.join(self.thistestpath, "GridTools/gtbench/build") + ";" + if testnum == 0: + print("Testing Convergence Test") + cmdrun = "./convergence_tests;" + elif testnum == 1: + print("Testing Performance Test") + cmdrun = "./benchmark --domain-size 256 256 --runs 100;" + cmdexc = cmdcd + cmdrun + env = os.environ.copy() + self.runlog = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, self.runlog, env) + + def clean(self): + print("Cleaning Gridtools..") + if self.runlog != None: + self.runlog.close() + + def parse_result(self, testnum): + return GridtoolsParser(self.runlog).parse(testnum) diff --git a/src/hiptestsuite/applications/hpc_apps/gridtools/gridtools_build_nvidia.py b/src/hiptestsuite/applications/hpc_apps/gridtools/gridtools_build_nvidia.py new file mode 100644 index 0000000..62a3173 --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/gridtools/gridtools_build_nvidia.py @@ -0,0 +1,114 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os +import tempfile +from hiptestsuite.common.hip_shell import * +from hiptestsuite.applications.hpc_apps.gridtools.gridtools_parser_common import GridtoolsParser + +class BuildRunNvidia(): + def __init__(self, thistestpath, logFile, cuda_target): + self.thistestpath = thistestpath + self.logFile = logFile + self.runlog = None + self.cuda_target = cuda_target + + def setenv(self): + env = "export HIP_PLATFORM=nvidia; export HIP_COMPILER=nvcc; export HIP_RUNTIME=cuda;" + env += "export GT_CUDA_COMPILATION_TYPE=HIPCC-NVCC;" + env += "export GT_ALL_DIR=$PWD/src/hiptestsuite/applications/hpc_apps/gridtools;" + env += "export GT_TREE_DIR=$GT_ALL_DIR/GridTools;" + env += "export BOOST_TREE_DIR=$GT_TREE_DIR/boost_1_72_0;" + env += "export BOOST_INSTALL_DIR=$GT_ALL_DIR/boost_1_72_0;" + env += "export GRIDTOOLS_TREE_DIR=$GT_TREE_DIR/gridtools;" + env += "export GRIDTOOLS_BUILD_DIR=$GRIDTOOLS_TREE_DIR/build;" + env += "export GRIDTOOLS_INSTALL_DIR=$GT_ALL_DIR/gridtools;" + env += "export GTBENCH_TREE_DIR=$GT_TREE_DIR/gtbench;" + env += "export GTBENCH_BUILD_DIR=$GTBENCH_TREE_DIR/build;" + return env + + def buildtest(self): + if not os.path.exists(os.path.join(self.thistestpath, "boost_1_72_0")): + print("Building and Installing Boost..") + cmdexc = self.setenv() + cmdexc += "cd $GT_TREE_DIR;" + cmdexc += "tar -xvjf ../boost_1_72_0.tar.bz2;cd $BOOST_TREE_DIR;" + cmdexc += "./bootstrap.sh --prefix=$BOOST_INSTALL_DIR --with-python=python3;" + cmdexc += "./b2 install -j8 threading=multi link=shared;" + runlogdump = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, runlogdump, None) + runlogdump.close() + else: + print("Boost already installed..") + # Build Gridtools + if not os.path.exists(os.path.join(self.thistestpath, "GridTools/gridtools/build")): + print("Building and Installing GridTools..") + cmdexc = self.setenv() + cmdexc += "export CXX=/opt/rocm/bin/hipcc;" + cmdexc += "cd $GT_TREE_DIR;cd $GRIDTOOLS_TREE_DIR;git apply ../../gridtools.patch;mkdir -p $GRIDTOOLS_BUILD_DIR;cd $GRIDTOOLS_BUILD_DIR;" + cmdexc += "CXX=/opt/rocm/bin/hipcc cmake .. -DBUILD_TESTING=OFF -DBoost_INCLUDE_DIR=$BOOST_INSTALL_DIR/include " +\ + "-DGT_CUDA_COMPILATION_TYPE=$GT_CUDA_COMPILATION_TYPE -DGT_CUDA_ARCH=" + self.cuda_target + " " +\ + "-DGT_ENABLE_BACKEND_CUDA=ON -DGT_ENABLE_BACKEND_MC=OFF -DGT_ENABLE_BACKEND_X86=OFF -DGT_ENABLE_BACKEND_NAIVE=OFF " +\ + "-DGT_USE_MPI=OFF -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$GRIDTOOLS_INSTALL_DIR;" + cmdexc += "make -j;" + cmdexc += "make install;" + runlogdump = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, runlogdump, None) + runlogdump.close() + else: + print("GridTools already built..") + + # Build Gtbench + if not os.path.exists(os.path.join(self.thistestpath, "GridTools/gtbench/build")): + print("Building and Installing Gtbench..") + cmdexc = self.setenv() + cmdexc += "export CXX=/opt/rocm/bin/hipcc;" + cmdexc += "cd $GT_TREE_DIR;cd $GTBENCH_TREE_DIR;git apply ../../gtbench.patch;mkdir -p $GTBENCH_BUILD_DIR;cd $GTBENCH_BUILD_DIR;" + cmdexc += "CXX=/opt/rocm/bin/hipcc cmake .. -DGridTools_DIR=$GRIDTOOLS_INSTALL_DIR/lib/cmake -DGTBENCH_BACKEND=cuda " +\ + "-DGTBENCH_RUNTIME=single_node -DCMAKE_CXX_FLAGS=--expt-relaxed-constexpr -DBoost_INCLUDE_DIR=$BOOST_INSTALL_DIR/include;" + cmdexc += "make CFLAGS=--expt-relaxed-constexpr CXXFLAGS=--expt-relaxed-constexpr -j8;" + runlogdump = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, runlogdump, None) + runlogdump.close() + else: + print("Gtbench already built..") + + return True + + def runtest(self, testnum): + cmdcd = "cd " + os.path.join(self.thistestpath, "GridTools/gtbench/build") + ";" + if testnum == 0: + print("Testing Convergence Test") + cmdrun = "./convergence_tests;" + elif testnum == 1: + print("Testing Performance Test") + cmdrun = "./benchmark --domain-size 256 256 --runs 100;" + cmdexc = cmdcd + cmdrun + env = os.environ.copy() + self.runlog = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, self.runlog, env) + + def clean(self): + print("Cleaning Gridtools..") + if self.runlog != None: + self.runlog.close() + + def parse_result(self, testnum): + return GridtoolsParser(self.runlog).parse(testnum) diff --git a/src/hiptestsuite/applications/hpc_apps/gridtools/gridtools_parser_common.py b/src/hiptestsuite/applications/hpc_apps/gridtools/gridtools_parser_common.py new file mode 100644 index 0000000..957d10b --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/gridtools/gridtools_parser_common.py @@ -0,0 +1,49 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import re + +class GridtoolsParser(): + def __init__(self, results): + results.seek(0) + logbytes = results.read() + self.results = logbytes + self.conv_tests = ["HORIZONTAL DIFFUSION", "VERTICAL DIFFUSION", "FULL DIFFUSION",\ + "HORIZONTAL ADVECTION", "VERTICAL ADVECTION", "RUNGE-KUTTA ADVECTION", "ADVECTION-DIFFUSION"] + def parse(self, testnum): + test_passed = False + if testnum == 0: + count = 0 + for test in self.conv_tests: + if re.search(test, self.results): + count = count + 1 + if count == len(self.conv_tests): + test_passed = True + + elif testnum == 1: + test1 = False + test2 = False + if re.search("Median time:\s*\d*\.\d*s", self.results): + test1 = True + if re.search("Columns per second:\s*\d+", self.results): + test2 = True + test_passed = test1 & test2 + + return test_passed diff --git a/src/hiptestsuite/applications/hpc_apps/gridtools/gtbench.patch b/src/hiptestsuite/applications/hpc_apps/gridtools/gtbench.patch new file mode 100644 index 0000000..e8a6756 --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/gridtools/gtbench.patch @@ -0,0 +1,120 @@ +diff --git a/runtime/computation.hpp b/runtime/computation.hpp +index 81b4497..1d29deb 100644 +--- a/runtime/computation.hpp ++++ b/runtime/computation.hpp +@@ -67,8 +67,8 @@ numerics::solver_state init_state(Discrete const &discrete, real_t t = 0_r) { + } + + inline void sync(numerics::solver_state &state) { +-#ifdef __CUDACC__ +- if (cudaDeviceSynchronize() != cudaSuccess) ++#ifdef __HIPCC__ ++ if (hipDeviceSynchronize() != hipSuccess) + throw std::runtime_error("device sync failed"); + #endif + } +diff --git a/runtime/device/set_device.cpp b/runtime/device/set_device.cpp +index c3c6e8f..4efc8d7 100644 +--- a/runtime/device/set_device.cpp ++++ b/runtime/device/set_device.cpp +@@ -9,25 +9,24 @@ + */ + #include "set_device.hpp" + #include +- ++#include + namespace runtime { +- +-#ifdef __CUDACC__ ++#ifdef __HIPCC__ + int set_device(int device_id) { + int device_count = 1; +- if (cudaGetDeviceCount(&device_count) != cudaSuccess) +- throw std::runtime_error("cudaGetDeviceCount failed"); ++ if (hipGetDeviceCount(&device_count) != hipSuccess) ++ throw std::runtime_error("hipGetDeviceCount failed"); + device_id %= device_count; +- if (cudaSetDevice(device_id) != cudaSuccess) +- throw std::runtime_error("cudaSetDevice failed"); ++ if (hipSetDevice(device_id) != hipSuccess) ++ throw std::runtime_error("hipSetDevice failed"); + if (device_count > 1) { + for (int i = 0; i < device_count; ++i) { + if (i != device_id) { + int flag; +- if (cudaDeviceCanAccessPeer(&flag, device_id, i) != cudaSuccess) ++ if (hipDeviceCanAccessPeer(&flag, device_id, i) != hipSuccess) + throw std::runtime_error("cudaDeviceAccessPeer failed"); + if (flag) { +- cudaDeviceEnablePeerAccess(i, 0); ++ hipDeviceEnablePeerAccess(i, 0); + } + } + } +diff --git a/runtime/gcl/run.cpp b/runtime/gcl/run.cpp +index 827e567..8bcdcfe 100644 +--- a/runtime/gcl/run.cpp ++++ b/runtime/gcl/run.cpp +@@ -19,7 +19,7 @@ namespace gcl_impl { + + using pattern_t = gt::halo_exchange_dynamic_ut, real_t, +-#ifdef __CUDACC__ ++#ifdef __HIPCC__ + gt::gcl_gpu + #else + gt::gcl_cpu +@@ -103,8 +103,8 @@ struct process_grid::impl { + + pattern->setup(1); + +-#ifdef __CUDACC__ +- cudaStreamSynchronize(0); ++#ifdef __HIPCC__ ++ hipStreamSynchronize(0); + #endif + + return [pattern = std::move(pattern)](storage_t &storage) { +diff --git a/runtime/ghex_comm/run.cpp b/runtime/ghex_comm/run.cpp +index 60e9e35..21c2777 100644 +--- a/runtime/ghex_comm/run.cpp ++++ b/runtime/ghex_comm/run.cpp +@@ -82,7 +82,7 @@ runtime::runtime(int num_threads, std::array cart_dims, + "the product of thread cart dims must be equal to the number of " + "threads per rank."); + } +-#ifdef __CUDACC__ ++#ifdef __HIPCC__ + MPI_Comm shmem_comm; + MPI_Comm_split_type(MPI_COMM_WORLD, MPI_COMM_TYPE_SHARED, 0, MPI_INFO_NULL, + &shmem_comm); +@@ -308,8 +308,8 @@ public: + &patterns = *m_patterns](storage_t &storage) mutable { + auto field = gt::ghex::wrap_gt_field(domain, storage); + +-#ifdef __CUDACC__ +- cudaStreamSynchronize(0); ++#ifdef __HIPCC__ ++ hipStreamSynchronize(0); + #endif + comm_obj->exchange(patterns(field)).wait(); + }; +@@ -356,7 +356,7 @@ void runtime_register_options(ghex_comm, options &options) { + "dimensons of cartesian decomposition " + "among sub-domains", + "TX TY", 2); +-#ifdef __CUDACC__ ++#ifdef __HIPCC__ + options("device-mapping", + "node device mapping: device id per sub-domain in the format " + "I_0:I_1:...:I_(N-1) " +@@ -368,7 +368,7 @@ void runtime_register_options(ghex_comm, options &options) { + + runtime runtime_init(ghex_comm, options_values const &options) { + std::vector device_mapping; +-#ifdef __CUDACC__ ++#ifdef __HIPCC__ + if (options.has("device-mapping")) { + const std::regex delimiter(":"); + const auto input = options.get("device-mapping"); + diff --git a/src/hiptestsuite/applications/hpc_apps/kokkos/__init__.py b/src/hiptestsuite/applications/hpc_apps/kokkos/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/hiptestsuite/applications/hpc_apps/kokkos/kokkos.py b/src/hiptestsuite/applications/hpc_apps/kokkos/kokkos.py new file mode 100644 index 0000000..b8fe373 --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/kokkos/kokkos.py @@ -0,0 +1,202 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.TesterRepository import Tester, Test, TestData +from hiptestsuite.Test import HIPTestData, TestResult, HIP_PLATFORM +from typing import Union, List +from hiptestsuite.test_classifier import TestClassifier +from hiptestsuite.applications.hpc_apps.kokkos.kokkos_build_amd import BuildRunAmd +from hiptestsuite.common.hip_get_packages import HipPackages +from hiptestsuite.common.hip_shell import execshellcmd + +import os +import re +# Common class to clone, set up, build and run test +class PrepareTest(): + def __init__(self, cwd): + self.cwdAbs = cwd + self.app_path = os.path.join(self.cwdAbs,\ + "src/hiptestsuite/applications/hpc_apps/kokkos/") + self.app_root = os.path.join(self.app_path, "kokkos/") + self.thistestpath = self.app_root + self.prepareobj = None + self.apprepo = "" # Default + self.appbranch = "" + self.appcommitId = "" + + def set_kokkos_repoinfo(self, test_data: HIPTestData): + validrepconfig = True + if test_data.repos["kokkos"].repo_url != None: + self.apprepo = test_data.repos["kokkos"].repo_url + else: + validrepconfig &= False + if test_data.repos["kokkos"].branch != None: + self.appbranch = test_data.repos["kokkos"].branch + if test_data.repos["kokkos"].commit_id != None: + self.appcommitId = test_data.repos["kokkos"].commit_id + return validrepconfig + + def downloadtest(self, logFile, test_data: HIPTestData): + ret = True + ret = ret & HipPackages().pull_repo(logFile, self.apprepo,\ + self.appbranch, self.appcommitId, "kokkos") + return ret + + def buildtest(self, logFile, platform): + if platform == HIP_PLATFORM.amd: + self.prepareobj = BuildRunAmd(self.thistestpath, logFile) + else: + print("Invalid Platform") + return False + buildstatus = self.prepareobj.buildtest() + if buildstatus == False: + return False + + return True + + def clean(self): + if self.prepareobj != None: + self.prepareobj.clean() + + def runtest(self, testnum): + if self.prepareobj != None: + self.prepareobj.runtest(testnum) + + def parse_result(self, testnum): + if self.prepareobj != None: + return self.prepareobj.parse_result(testnum) + return False + +class KOKKOS(TestClassifier): + def __init__(self): + TestClassifier.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + TestClassifier.add_matched_with_names(self, {"kokkos": matched_with_names}) + +class PERFORMANCE(KOKKOS): + def __init__(self): + KOKKOS.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + KOKKOS.add_matched_with_names(self, {"performance": matched_with_names}) + +class MINIAPP(KOKKOS): + def __init__(self): + KOKKOS.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + KOKKOS.add_matched_with_names(self, {"mini-app": matched_with_names}) + +# Kokkos Unit/Regression Test +class KOKKOS_UNIT_TEST(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + classifier = MINIAPP() + classifier.add_matched_with_names() + test.classifiers = [classifier] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== KOKKOS Unit test ===============") + if test_data.HIP_PLATFORM == HIP_PLATFORM.nvidia: + print("KOKKOS test is not supported on NVIDIA") + test_data.test_result = TestResult.SKIP + return + # Set repo info + isrepocfgvalid = self.set_kokkos_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/kokkos_unit.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(0) + # Parse the test result + if True == self.parse_result(0): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + +# Performance Unit/Regression Test +class KOKKOS_PERF_TEST(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + classifier = PERFORMANCE() + classifier.add_matched_with_names() + test.classifiers = [classifier] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== KOKKOS Performance test ===============") + if test_data.HIP_PLATFORM == HIP_PLATFORM.nvidia: + print("KOKKOS test is not supported on NVIDIA") + test_data.test_result = TestResult.SKIP + return + # Set repo info + isrepocfgvalid = self.set_kokkos_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/kokkos_perf.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(1) + # Parse the test result + if True == self.parse_result(1): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL diff --git a/src/hiptestsuite/applications/hpc_apps/kokkos/kokkos_build_amd.py b/src/hiptestsuite/applications/hpc_apps/kokkos/kokkos_build_amd.py new file mode 100644 index 0000000..2ef02bb --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/kokkos/kokkos_build_amd.py @@ -0,0 +1,76 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os +import re +import tempfile +from hiptestsuite.common.hip_shell import * +from hiptestsuite.applications.hpc_apps.kokkos.kokkos_parser_common import KokkosParser + +class BuildRunAmd(): + def __init__(self, thistestpath, logFile): + self.thistestpath = thistestpath + self.logFile = logFile + self.runlog = None + + def buildtest(self): + # In this function put the build steps for test cases + # which differ across platforms (amd/nvidia/intel) + gpuarch = get_gpuarch(self.logFile) + if gpuarch is None: + return False + # extract the gpu architecture number + arch_num = re.search("\d+", gpuarch).group(0) + if not os.path.exists(os.path.join(self.thistestpath, "build")): + print("Building Kokkos..") + cmdcd = "cd " + self.thistestpath + "; mkdir build; cd build;" + cmd_cmake = "cmake -DKokkos_ARCH_SKX=ON -DKokkos_ARCH_VEGA" + arch_num + "=ON -DCMAKE_CXX_COMPILER=/opt/rocm/bin/hipcc " +\ + "-DKokkos_ENABLE_HIP=ON -DKokkos_ENABLE_SERIAL=ON -DKokkos_ENABLE_TESTS=ON -DKokkos_CXX_STANDARD=14 -DCMAKE_CXX_STANDARD=14 " +\ + "-DCMAKE_INSTALL_PREFIX=../install -DKokkos_ENABLE_HIP_RELOCATABLE_DEVICE_CODE=OFF " +\ + "-DCMAKE_CXX_FLAGS=\"-O3 -DNDEBUG --amdgpu-target=gfx" + arch_num + "\" ..;" + cmd_build = "make -j; make -j install;" + cmdexc = cmdcd + cmd_cmake + cmd_build + runlogdump = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, runlogdump, None) + runlogdump.close() + else: + print("Kokkos already built..") + + return True + + def runtest(self, testnum): + print("Running Kokkos Test..") + cmdcd = "cd " + self.thistestpath + "/build;" + if testnum == 0: + cmdrun = "ctest -R;" + elif testnum == 1: + cmdrun = "./core/perf_test/KokkosCore_PerfTestExec;" + cmdexc = cmdcd + cmdrun + env = os.environ.copy() + self.runlog = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, self.runlog, env) + + def clean(self): + print("Cleaning Kokkos..") + if self.runlog != None: + self.runlog.close() + + def parse_result(self, testnum): + return KokkosParser(self.runlog).parse(testnum) diff --git a/src/hiptestsuite/applications/hpc_apps/kokkos/kokkos_parser_common.py b/src/hiptestsuite/applications/hpc_apps/kokkos/kokkos_parser_common.py new file mode 100644 index 0000000..1afe1ea --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/kokkos/kokkos_parser_common.py @@ -0,0 +1,37 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import re + +class KokkosParser(): + def __init__(self, results): + results.seek(0) + logbytes = results.read() + self.results = logbytes + + def parse(self, testnum): + testpassed = False + if testnum == 0: + if re.search("100% tests passed, 0 tests failed out of \d+", self.results): + testpassed = True + elif testnum == 1: + if re.search("[\s*PASSED\s*]\s*\d+\s*tests", self.results): + testpassed = True + return testpassed diff --git a/src/hiptestsuite/applications/hpc_apps/laghos/__init__.py b/src/hiptestsuite/applications/hpc_apps/laghos/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/hiptestsuite/applications/hpc_apps/laghos/hip_on_nvcc.patch b/src/hiptestsuite/applications/hpc_apps/laghos/hip_on_nvcc.patch new file mode 100644 index 0000000..835cb81 --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/laghos/hip_on_nvcc.patch @@ -0,0 +1,44 @@ +diff --git a/general/error.hpp b/general/error.hpp +index 957aef944..a59cee0a3 100644 +--- a/general/error.hpp ++++ b/general/error.hpp +@@ -150,16 +150,20 @@ void mfem_warning(const char *msg = NULL); + + // Additional abort functions for HIP + #if defined(MFEM_USE_HIP) +-template +-__host__ void abort_msg(T & msg) +-{ +- MFEM_ABORT(msg); +-} ++//template ++//__host__ void abort_msg(T & msg) ++//{ ++// MFEM_ABORT(msg); ++//} + + template +-__device__ void abort_msg(T & msg) ++__host__ __device__ void abort_msg(T & msg) + { ++#ifndef __HIP_DEVICE_COMPILE_ ++ MFEM_ABORT(msg); ++#else + abort(); ++#endif + } + #endif + +diff --git a/general/hip.hpp b/general/hip.hpp +index 9ad804fff..62d8ed625 100644 +--- a/general/hip.hpp ++++ b/general/hip.hpp +@@ -20,7 +20,7 @@ + + #ifdef MFEM_USE_HIP + #define MFEM_DEVICE __device__ +-#define MFEM_LAMBDA __host__ __device__ ++#define MFEM_LAMBDA __host__ + #define MFEM_HOST_DEVICE __host__ __device__ + #define MFEM_DEVICE_SYNC MFEM_GPU_CHECK(hipDeviceSynchronize()) + #define MFEM_STREAM_SYNC MFEM_GPU_CHECK(hipStreamSynchronize(0)) diff --git a/src/hiptestsuite/applications/hpc_apps/laghos/laghos-multinode.patch b/src/hiptestsuite/applications/hpc_apps/laghos/laghos-multinode.patch new file mode 100644 index 0000000..8f6981c --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/laghos/laghos-multinode.patch @@ -0,0 +1,27 @@ +diff --git a/laghos.cpp b/laghos.cpp +index cfcb0ed..bee1447 100644 +--- a/laghos.cpp ++++ b/laghos.cpp +@@ -201,6 +201,7 @@ int main(int argc, char *argv[]) + if (mpi.Root()) { args.PrintOptions(cout); } + + // Configure the device from the command line options ++ dev=myid; + Device backend; + backend.Configure(device, dev); + if (mpi.Root()) { backend.Print(); } +diff --git a/laghos_solver.cpp b/laghos_solver.cpp +index 2a04477..255ce57 100644 +--- a/laghos_solver.cpp ++++ b/laghos_solver.cpp +@@ -689,6 +689,10 @@ void LagrangianHydroOperator::PrintTimingData(bool IamRoot, int steps, + cout << "Major kernels total time (seconds): " << T[4] << endl; + cout << "Major kernels total rate (megadofs x time steps / second): " + << FOM << endl; ++ cout <<" "<< endl; ++ cout <<" FOM= "< List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + classifier = PERFORMANCE() + classifier.add_matched_with_names() + test.classifiers = [classifier] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== Laghos cube01_hex test ===============") + # Set repo info + isrepocfgvalid = self.set_laghos_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/cube01_hex.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, test_data.build_for_cuda_target) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(0) + # Parse the test result + if True == self.parse_result(0): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + +# cube_12_hex test +class LAGHOS_CUBE_12_HEX(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + classifier = PERFORMANCE() + classifier.add_matched_with_names() + test.classifiers = [classifier] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== Laghos cube_12_hex test ===============") + # Set repo info + isrepocfgvalid = self.set_laghos_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/cube_12_hex.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, test_data.build_for_cuda_target) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest(1) + # Parse the test result + if True == self.parse_result(1): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL diff --git a/src/hiptestsuite/applications/hpc_apps/laghos/laghos_build_amd.py b/src/hiptestsuite/applications/hpc_apps/laghos/laghos_build_amd.py new file mode 100644 index 0000000..adb0648 --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/laghos/laghos_build_amd.py @@ -0,0 +1,167 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os +import re +import tempfile +from hiptestsuite.common.hip_shell import * +from hiptestsuite.applications.hpc_apps.laghos.laghos_parser_common import LaghosParser + +class BuildRunAmd(): + def __init__(self, thistestpath, logFile): + self.thistestpath = thistestpath + self.logFile = logFile + self.runlog = None + + def set_env(self): + cmd = "export MPI_PATH=/usr/local/openmpi;" + cmd += "export ROCM_PATH=/opt/rocm;" + cmd += "export PATH=${MPI_PATH}/bin:$PATH;" + cmd += "export PATH=${ROCM_PATH}/bin:$PATH;" + return cmd + + def configure_build_hypre(self): + if os.path.exists(os.path.join(self.thistestpath, "hypre/src/lib/libHYPRE.a")): + print("Hypre already built") + return True + print("Hypre build in progress ..") + if not os.path.exists("/opt/rocm"): + print("Rocm backend is not installed under /opt/. Exiting Test!") + return False + if not os.path.exists("/usr/local/openmpi"): + print("Openmpi backend is not installed under /usr/local/. Exiting Test!") + return False + + cmd = self.set_env() + cmd += "cd " + self.thistestpath + ";" + cmd += "wget https://github.com/hypre-space/hypre/archive/v2.16.0.tar.gz;" + cmd += "mv v2.16.0.tar.gz hypre-2.16.0.tar.gz;tar -zxvf hypre-2.16.0.tar.gz;rm hypre-2.16.0.tar.gz;" + cmd += "cd hypre-2.16.0/src/;" + cmd += "./configure --disable-fortran --enable-bigint --with-MPI --with-MPI-include=${MPI_PATH}/include --with-MPI-lib-dirs=${MPI_PATH}/lib;" + cmd += "make -j; cd ../..; ln -s hypre-2.16.0 hypre;" + runlogdump = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmd, self.logFile, runlogdump, None) + runlogdump.close() + if not os.path.exists(os.path.join(self.thistestpath, "hypre/src/lib/libHYPRE.a")): + print("Hypre Build Failed") + return False + return True + + def configure_build_metis(self): + if os.path.exists(os.path.join(self.thistestpath, "metis-4.0/libmetis.a")): + print("Metis already built") + return True + print("Metis build in progress ..") + cmd = self.set_env() + cmd += "cd " + self.thistestpath + ";" + cmd += "wget http://glaros.dtc.umn.edu/gkhome/fetch/sw/metis/OLD/metis-4.0.3.tar.gz;" + cmd += "tar -zxvf metis-4.0.3.tar.gz;rm metis-4.0.3.tar.gz;" + cmd += "cd metis-4.0.3;make -j;cd ..;" + cmd += "ln -s metis-4.0.3 metis-4.0;" + runlogdump = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmd, self.logFile, runlogdump, None) + runlogdump.close() + if not os.path.exists(os.path.join(self.thistestpath, "metis-4.0/libmetis.a")): + print("Metis Build Failed") + return False + return True + + def configure_build_mfem(self): + if os.path.exists(os.path.join(self.thistestpath, "mfem/libmfem.a")): + print("Mfem already built") + return True + print("Mfem build in progress ..") + cmd = self.set_env() + cmd += "cd " + self.thistestpath + "; cd mfem;" + cmd += "make config;" + cmd += "make phip -j CXXFLAGS=\"-O3 -std=c++11 --gpu-max-threads-per-block=256\" \ + MFEM_TPLFLAGS=\"-I./../hypre/src/hypre/include -I${MPI_PATH}/include\" MFEM_EXT_LIBS=\"-L./../hypre/src/hypre/lib \ + -lHYPRE -L./../metis-4.0 -lmetis -lrt -L${MPI_PATH}/lib -lmpi\";" + runlogdump = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmd, self.logFile, runlogdump, None) + runlogdump.close() + if not os.path.exists(os.path.join(self.thistestpath, "mfem/libmfem.a")): + print("Mfem Build Failed") + return False + return True + + def configure_build_laghos(self): + if os.path.exists(os.path.join(self.thistestpath, "Laghos/laghos")): + print("Laghos already built") + return True + print("Laghos build in progress ..") + cmd = self.set_env() + cmd += "cd " + self.thistestpath + "; cd Laghos;" + cmd += "make -j;" + runlogdump = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmd, self.logFile, runlogdump, None) + runlogdump.close() + if not os.path.exists(os.path.join(self.thistestpath, "mfem/libmfem.a")): + print("Laghos Build Failed") + return False + return True + + def buildtest(self): + # In this function put the build steps for test cases + # which differ across platforms (amd/nvidia/intel) + # Check if OpenMPI is already installed. Else return. + if os.path.exists("/usr/local/openmpi/bin") and os.path.exists("/usr/local/openmpi/etc")\ + and os.path.exists("/usr/local/openmpi/lib") and os.path.exists("/usr/local/openmpi/include")\ + and os.path.exists("/usr/local/openmpi/share"): + print("Openmpi installed.") + else: + print("Openmpi not installed. Exiting test!") + return False + + if not self.configure_build_hypre(): + print("Hypre configuration and build failed ..") + return False + if not self.configure_build_metis(): + print("Metis configuration and build failed ..") + return False + if not self.configure_build_mfem(): + print("Mfem configuration and build failed ..") + return False + if not self.configure_build_laghos(): + print("Laghos configuration and build failed ..") + return False + + return True + + def runtest(self, testnum): + print("Running Laghos Test" + str(testnum)) + cmd = self.set_env() + cmd += "cd " + self.thistestpath + "; cd Laghos;" + cmd += "export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/openmpi/lib;" + if testnum == 0: + cmd += "mpirun -np 1 laghos -pa -p 1 -tf 0.6 -no-vis -m data/cube01_hex.mesh --cg-tol 0 --cg-max-steps 50 --max-steps 2 -ok 2 -ot 1 -rs 5 -d hip;" + elif testnum == 1: + cmd += "mpirun -np 1 laghos -pa -p 1 -tf 0.6 -no-vis -m data/cube_12_hex.mesh --cg-tol 0 --cg-max-steps 50 --max-steps 2 -ok 3 -ot 2 -rs 4 -d hip;" + env = os.environ.copy() + self.runlog = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmd, self.logFile, self.runlog, env) + + def clean(self): + print("Cleaning Laghos..") + if self.runlog != None: + self.runlog.close() + + def parse_result(self, testnum): + return LaghosParser(self.runlog).parse(testnum) diff --git a/src/hiptestsuite/applications/hpc_apps/laghos/laghos_build_nvidia.py b/src/hiptestsuite/applications/hpc_apps/laghos/laghos_build_nvidia.py new file mode 100644 index 0000000..ddc7e15 --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/laghos/laghos_build_nvidia.py @@ -0,0 +1,168 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os +import re +import tempfile +from hiptestsuite.common.hip_shell import * +from hiptestsuite.applications.hpc_apps.laghos.laghos_parser_common import LaghosParser + +class BuildRunNvidia(): + def __init__(self, thistestpath, logFile, cuda_target): + self.thistestpath = thistestpath + self.logFile = logFile + self.runlog = None + self.cuda_target = cuda_target + + def set_env(self): + cmd = "export MPI_PATH=/usr/local/openmpi;" + cmd += "export ROCM_PATH=/opt/rocm;" + cmd += "export PATH=${MPI_PATH}/bin:$PATH;" + cmd += "export PATH=${ROCM_PATH}/bin:$PATH;" + cmd += "export CUDA_PATH=/usr/local/cuda;" + cmd += "export HIP_PLATFORM=nvidia;" + return cmd + + def configure_build_hypre(self): + if os.path.exists(os.path.join(self.thistestpath, "hypre/src/lib/libHYPRE.a")): + print("Hypre already built") + return True + print("Hypre build in progress ..") + cmd = self.set_env() + cmd += "cd " + self.thistestpath + ";" + cmd += "wget https://github.com/hypre-space/hypre/archive/v2.16.0.tar.gz;" + cmd += "mv v2.16.0.tar.gz hypre-2.16.0.tar.gz;tar -zxvf hypre-2.16.0.tar.gz;rm hypre-2.16.0.tar.gz;" + cmd += "cd hypre-2.16.0/src/;" + cmd += "./configure --disable-fortran --enable-bigint --with-MPI --with-MPI-include=${MPI_PATH}/include --with-MPI-lib-dirs=${MPI_PATH}/lib;" + cmd += "make -j; cd ../..; ln -s hypre-2.16.0 hypre;" + runlogdump = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmd, self.logFile, runlogdump, None) + runlogdump.close() + if not os.path.exists(os.path.join(self.thistestpath, "hypre/src/lib/libHYPRE.a")): + print("Hypre Build Failed") + return False + return True + + def configure_build_metis(self): + if os.path.exists(os.path.join(self.thistestpath, "metis-4.0/libmetis.a")): + print("Metis already built") + return True + print("Metis build in progress ..") + cmd = self.set_env() + cmd += "cd " + self.thistestpath + ";" + cmd += "wget http://glaros.dtc.umn.edu/gkhome/fetch/sw/metis/OLD/metis-4.0.3.tar.gz;" + cmd += "tar -zxvf metis-4.0.3.tar.gz;rm metis-4.0.3.tar.gz;" + cmd += "cd metis-4.0.3;make -j;cd ..;" + cmd += "ln -s metis-4.0.3 metis-4.0;" + runlogdump = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmd, self.logFile, runlogdump, None) + runlogdump.close() + if not os.path.exists(os.path.join(self.thistestpath, "metis-4.0/libmetis.a")): + print("Metis Build Failed") + return False + return True + + def configure_build_mfem(self): + if os.path.exists(os.path.join(self.thistestpath, "mfem/libmfem.a")): + print("Mfem already built") + return True + print("Mfem build in progress ..") + cmd = self.set_env() + cmd += "cd " + self.thistestpath + "; cd mfem;" + if not os.path.isfile(os.path.join(self.thistestpath, "mfem/patched")): + cmd += "git apply ../hip_on_nvcc.patch; touch patched;" + cmd += "make phip CXXFLAGS=\"-std=c++11 -x=cu --extended-lambda -arch=" + self.cuda_target + " -O3 -g -I/opt/rocm/include\" MFEM_TPLFLAGS=\"-I./../hypre/src/hypre/include -I${MPI_PATH}/include\" MFEM_EXT_LIBS=\"-L./../hypre/src/hypre/lib -lHYPRE -L./../metis-4.0 -lmetis -lrt -L${MPI_PATH}/lib -lmpi\" -j;" + runlogdump = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmd, self.logFile, runlogdump, None) + runlogdump.close() + if not os.path.exists(os.path.join(self.thistestpath, "mfem/libmfem.a")): + print("Mfem Build Failed") + return False + return True + + def configure_build_laghos(self): + if os.path.exists(os.path.join(self.thistestpath, "Laghos/laghos")): + print("Laghos already built") + return True + print("Laghos build in progress ..") + cmd = self.set_env() + cmd += "cd " + self.thistestpath + "; cd Laghos;" + if not os.path.isfile(os.path.join(self.thistestpath, "Laghos/patched")): + cmd += "git apply ../laghos-multinode.patch; touch patched;" + cmd += "make -j;" + runlogdump = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmd, self.logFile, runlogdump, None) + runlogdump.close() + if not os.path.exists(os.path.join(self.thistestpath, "mfem/libmfem.a")): + print("Laghos Build Failed") + return False + return True + + def buildtest(self): + # In this function put the build steps for test cases + # which differ across platforms (amd/nvidia/intel) + # Check if OpenMPI is already installed. Else return. + if os.path.exists("/usr/local/openmpi/bin") and os.path.exists("/usr/local/openmpi/etc")\ + and os.path.exists("/usr/local/openmpi/lib") and os.path.exists("/usr/local/openmpi/include")\ + and os.path.exists("/usr/local/openmpi/share"): + print("Openmpi installed.") + else: + print("Openmpi not installed. Exiting test!") + return False + + if not os.path.exists("/opt/rocm"): + print("Rocm backend is not installed under /opt/. Exiting Test!") + return False + + if not self.configure_build_hypre(): + print("Hypre configuration and build failed ..") + return False + if not self.configure_build_metis(): + print("Metis configuration and build failed ..") + return False + if not self.configure_build_mfem(): + print("Mfem configuration and build failed ..") + return False + if not self.configure_build_laghos(): + print("Laghos configuration and build failed ..") + return False + + return True + + def runtest(self, testnum): + print("Running Laghos Test" + str(testnum)) + cmd = self.set_env() + cmd += "cd " + self.thistestpath + "; cd Laghos;" + cmd += "export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/openmpi/lib;" + if testnum == 0: + cmd += "mpirun -np 1 laghos -pa -p 1 -tf 0.6 -no-vis -m data/cube01_hex.mesh --cg-tol 0 --cg-max-steps 50 --max-steps 2 -ok 2 -ot 1 -rs 5 -d hip;" + elif testnum == 1: + cmd += "mpirun -np 1 laghos -pa -p 1 -tf 0.6 -no-vis -m data/cube_12_hex.mesh --cg-tol 0 --cg-max-steps 50 --max-steps 2 -ok 3 -ot 2 -rs 4 -d hip;" + env = os.environ.copy() + self.runlog = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmd, self.logFile, self.runlog, env) + + def clean(self): + print("Cleaning Laghos..") + if self.runlog != None: + self.runlog.close() + + def parse_result(self, testnum): + return LaghosParser(self.runlog).parse(testnum) diff --git a/src/hiptestsuite/applications/hpc_apps/laghos/laghos_parser_common.py b/src/hiptestsuite/applications/hpc_apps/laghos/laghos_parser_common.py new file mode 100644 index 0000000..a98cc09 --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/laghos/laghos_parser_common.py @@ -0,0 +1,40 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import re + +class LaghosParser(): + def __init__(self, results): + results.seek(0) + logbytes = results.read() + self.results = logbytes + + def parse(self, testnum): + count = 0 + if testnum == 0 or testnum == 1: + if re.search("Major\s*kernels\s*total\s*time\s*\(seconds\):\s*\d+\.\d+", self.results): + count = count + 1 + if re.search("Major\s*kernels\s*total\s*rate\s*\(megadofs\s*x\s*time\s*steps\s*/\s*second\):\s*\d+\.\d+", self.results): + count = count + 1 + if re.search("Energy\s+diff:\s*\d+\.\d+", self.results): + count = count + 1 + if count != 3: + return False + return True diff --git a/src/hiptestsuite/applications/hpc_apps/quicksilver/__init__.py b/src/hiptestsuite/applications/hpc_apps/quicksilver/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/hiptestsuite/applications/hpc_apps/quicksilver/qs_diff_patch_nvidia b/src/hiptestsuite/applications/hpc_apps/quicksilver/qs_diff_patch_nvidia new file mode 100644 index 0000000..86c6201 --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/quicksilver/qs_diff_patch_nvidia @@ -0,0 +1,32 @@ +--- ../../Makefile 2021-11-12 16:14:28.748243195 +0530 ++++ Makefile 2021-11-12 16:19:31.194839331 +0530 +@@ -119,10 +119,10 @@ + ##################################################################################### + #hip, no MPI + ##################################################################################### +-CXX = $(ROCM_PATH)/bin/hipcc +-CXXFLAGS = -I$(ROCM_PATH)/include/ +-CPPFLAGS = -DHAVE_HIP=1 -DMaxIt=15 +-LDFLAGS = -L$(ROCM_PATH)/lib -L$(ROCM_PATH)/lib ++#CXX = $(ROCM_PATH)/bin/hipcc ++#CXXFLAGS = -I$(ROCM_PATH)/include/ ++#CPPFLAGS = -DHAVE_HIP=1 -DMaxIt=15 ++#LDFLAGS = -L$(ROCM_PATH)/lib -L$(ROCM_PATH)/lib + + ############################################################################ + #hip + mpi +@@ -137,10 +137,10 @@ + #hip + nvcc + ###################################################################### + ###export HIP_PLATFORM=nvidia +-#CXX = $(ROCM_PATH)/bin/hipcc +-#CXXFLAGS = -I$(ROCM_PATH)/include/ -I$(CUDA_PATH)/include -I${MPIPATH}/include +-#CPPFLAGS = -DHAVE_HIP=1 -DHAVE_MPI -DHAVE_ASYNC_MPI -DMaxIt=15 -DHAVE_CUDA=1 -arch=compute_80 +-#LDFLAGS = -L$(CUDA_PATH)/lib64 -lcuda -lcudart -L${MPIPATH}/lib -lmpi ++CXX = $(ROCM_PATH)/bin/hipcc ++CXXFLAGS = -I$(ROCM_PATH)/include/ -I$(CUDA_PATH)/include -I${MPIPATH}/include ++CPPFLAGS = -DHAVE_HIP=1 -DHAVE_MPI -DHAVE_ASYNC_MPI -DMaxIt=15 -DHAVE_CUDA=1 -arch=$(CUDA_ARCH) ++LDFLAGS = -L$(CUDA_PATH)/lib64 -lcuda -lcudart -L${MPIPATH}/lib -lmpi + + ############################################################################### + #hip openmp 4.5 not supported yet diff --git a/src/hiptestsuite/applications/hpc_apps/quicksilver/quicksilver.py b/src/hiptestsuite/applications/hpc_apps/quicksilver/quicksilver.py new file mode 100644 index 0000000..05098c0 --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/quicksilver/quicksilver.py @@ -0,0 +1,154 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.TesterRepository import Tester, Test, TestData +from hiptestsuite.Test import HIPTestData, TestResult, HIP_PLATFORM +from typing import Union, List +from hiptestsuite.test_classifier import TestClassifier +from hiptestsuite.applications.hpc_apps.quicksilver.quicksilver_build_amd import BuildRunAmd +from hiptestsuite.applications.hpc_apps.quicksilver.quicksilver_build_nvidia import BuildRunNvidia +from hiptestsuite.common.hip_get_packages import HipPackages +from hiptestsuite.common.hip_shell import execshellcmd + +import os +import re +# Common class to clone, set up, build and run test +class PrepareTest(): + def __init__(self, cwd, binary): + self.cwdAbs = cwd + self.binary = binary + self.app_path = os.path.join(self.cwdAbs,\ + "src/hiptestsuite/applications/hpc_apps/quicksilver/") + self.app_root = os.path.join(self.app_path, "Quicksilver/") + self.thistestpath = self.app_root + self.prepareobj = None + self.apprepo = "" # Default + self.appbranch = "" + self.appcommitId = "" + + def set_quicksilver_repoinfo(self, test_data: HIPTestData): + validrepconfig = True + if test_data.repos["quicksilver"].repo_url != None: + self.apprepo = test_data.repos["quicksilver"].repo_url + else: + validrepconfig &= False + if test_data.repos["quicksilver"].branch != None: + self.appbranch = test_data.repos["quicksilver"].branch + if test_data.repos["quicksilver"].commit_id != None: + self.appcommitId = test_data.repos["quicksilver"].commit_id + return validrepconfig + + def downloadtest(self, logFile, test_data: HIPTestData): + ret = True + ret = ret & HipPackages().pull_repo(logFile, self.apprepo,\ + self.appbranch, self.appcommitId, "quicksilver") + return ret + + def buildtest(self, logFile, platform, cuda_target): + isBinaryPresent = True + if platform == HIP_PLATFORM.amd: + self.prepareobj = BuildRunAmd(self.thistestpath, logFile) + elif platform == HIP_PLATFORM.nvidia: + self.prepareobj = BuildRunNvidia(self.thistestpath, logFile, cuda_target) + else: + print("Invalid Platform") + return False + + buildstatus = self.prepareobj.buildtest() + if buildstatus == False: + return False + # Check if test binary is created + binarypath = os.path.join(self.thistestpath, "src") + if not os.path.isfile(\ + os.path.join(binarypath, self.binary)): + isBinaryPresent &= False + return isBinaryPresent + + def clean(self): + if self.prepareobj != None: + self.prepareobj.clean() + + def runtest(self): + if self.prepareobj != None: + self.prepareobj.runtest() + + def parse_result(self): + if self.prepareobj != None: + return self.prepareobj.parse_result() + return False + +class QUICKSILVER(TestClassifier): + def __init__(self): + TestClassifier.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + TestClassifier.add_matched_with_names(self, {"quicksilver": matched_with_names}) + +class PERFORMANCE(QUICKSILVER): + def __init__(self): + QUICKSILVER.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + QUICKSILVER.add_matched_with_names(self, {"performance": matched_with_names}) + + +# Quicksilver test +class QUICKSILVERTEST(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd, "qs") + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + classifier = PERFORMANCE() + classifier.add_matched_with_names() + test.classifiers = [classifier] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== Quicksilver test ===============") + # Set repo info + isrepocfgvalid = self.set_quicksilver_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/qs.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM, test_data.build_for_cuda_target) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest() + # Parse the test result + if True == self.parse_result(): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL diff --git a/src/hiptestsuite/applications/hpc_apps/quicksilver/quicksilver_build_amd.py b/src/hiptestsuite/applications/hpc_apps/quicksilver/quicksilver_build_amd.py new file mode 100644 index 0000000..6fd157d --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/quicksilver/quicksilver_build_amd.py @@ -0,0 +1,70 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import tempfile +import os +from hiptestsuite.common.hip_shell import execshellcmd_largedump, execshellcmd +from hiptestsuite.applications.hpc_apps.quicksilver.quicksilver_parser_common import QuicksilverParser + +class BuildRunAmd(): + def __init__(self, thistestpath, logFile): + self.thistestpath = thistestpath + self.logFile = logFile + self.resultlogFile = None + self.runlog = "" + + def buildtest(self): + # In this function put the build steps for test cases + # which differ across platforms (amd/nvidia/intel) + print("Building Quicksilver..") + if not os.path.exists("/opt/rocm"): + print("Rocm backend is not installed under /opt/. Exiting Test!") + return False + env = "export ROCM_PATH=/opt/rocm;" + cmdcd = "cd " + self.thistestpath + ";" + cmd_build = "cd src; make -j;" + cmdexc = env + cmdcd + cmd_build + runlogdump = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, runlogdump, None) + runlogdump.close() + if not os.path.exists(os.path.join(self.thistestpath, "src/qs")): + print("Quicksilver Build Failed. Exiting Test!") + return False + return True + + def runtest(self): + print("Running Quicksilver Test..") + cmdcd = "cd " + self.thistestpath + ";" + "cd src;" + cmdrun = "./qs -i ../Examples/CORAL2_Benchmark/Problem1/Coral2_P1.inp -X 16 -Y 16 -Z 16 -x 16 -y 16 -z 16 -I 1 -J 1 -K 1 -b 2 -n 2621440;" + cmdexc = cmdcd + cmdrun + self.resultlogFile = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, self.resultlogFile, None) + + def clean(self): + print("Cleaning Quicksilver..") + if self.resultlogFile != None: + self.resultlogFile.close() + cmdcd = "cd " + self.thistestpath + ";" + "cd src;" + cmdrm = "make clean;" + cmdexc = cmdcd + cmdrm + execshellcmd(cmdexc, None, None) + + def parse_result(self): + return QuicksilverParser(self.resultlogFile).parse() diff --git a/src/hiptestsuite/applications/hpc_apps/quicksilver/quicksilver_build_nvidia.py b/src/hiptestsuite/applications/hpc_apps/quicksilver/quicksilver_build_nvidia.py new file mode 100644 index 0000000..c9830e5 --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/quicksilver/quicksilver_build_nvidia.py @@ -0,0 +1,82 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import tempfile +import os +from hiptestsuite.common.hip_shell import execshellcmd_largedump, execshellcmd +from hiptestsuite.applications.hpc_apps.quicksilver.quicksilver_parser_common import QuicksilverParser + +class BuildRunNvidia(): + def __init__(self, thistestpath, logFile, cuda_arch): + self.thistestpath = thistestpath + self.logFile = logFile + self.resultlogFile = None + self.runlog = "" + self.cuda_arch = cuda_arch + + def buildtest(self): + # In this function put the build steps for test cases + # which differ across platforms (amd/nvidia/intel) + print("Building Quicksilver..") + if not os.path.exists("/opt/rocm"): + print("Rocm backend is not installed under /opt/. Exiting Test!") + return False + if not os.path.exists("/usr/local/openmpi"): + print("Openmpi backend is not installed under /usr/local/. Exiting Test!") + return False + if not os.path.exists("/usr/local/cuda"): + print("Cuda is not installed under /usr/local/. Exiting Test!") + return False + + env = "export ROCM_PATH=/opt/rocm;export MPIPATH=/usr/local/openmpi;export CUDA_PATH=/usr/local/cuda;export HIP_PLATFORM=nvidia;" + testpath = os.path.join(self.thistestpath, "src/") + cmdcd = "cd " + testpath + ";" + cmd_modify = "" + if not os.path.isfile(os.path.join(testpath, "patched")): + cmd_modify = "patch -p0 < ../../qs_diff_patch_nvidia; touch patched;" + cmd_build = "CUDA_ARCH=" + self.cuda_arch + " make -j;" + cmdexc = env + cmdcd + cmd_modify + cmd_build + runlogdump = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, runlogdump, None) + runlogdump.close() + if not os.path.exists(os.path.join(self.thistestpath, "src/qs")): + print("Quicksilver Build Failed. Exiting Test!") + return False + return True + + def runtest(self): + print("Running Quicksilver Test..") + cmdcd = "cd " + self.thistestpath + ";" + "cd src;" + cmdrun = "./qs -i ../Examples/CORAL2_Benchmark/Problem1/Coral2_P1.inp -X 16 -Y 16 -Z 16 -x 16 -y 16 -z 16 -I 1 -J 1 -K 1 -b 2 -n 2621440;" + cmdexc = cmdcd + cmdrun + self.resultlogFile = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, self.resultlogFile, None) + + def clean(self): + print("Cleaning Quicksilver..") + if self.resultlogFile != None: + self.resultlogFile.close() + cmdcd = "cd " + self.thistestpath + ";" + "cd src;" + cmdrm = "make clean;" + cmdexc = cmdcd + cmdrm + execshellcmd(cmdexc, None, None) + + def parse_result(self): + return QuicksilverParser(self.resultlogFile).parse() diff --git a/src/hiptestsuite/applications/hpc_apps/quicksilver/quicksilver_parser_common.py b/src/hiptestsuite/applications/hpc_apps/quicksilver/quicksilver_parser_common.py new file mode 100644 index 0000000..db35dda --- /dev/null +++ b/src/hiptestsuite/applications/hpc_apps/quicksilver/quicksilver_parser_common.py @@ -0,0 +1,42 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import re + +class QuicksilverParser(): + def __init__(self, results): + results.seek(0) + self.results = results.read() + + def parse(self): + passed1 = False + passed2 = False + passed3 = False + passed4 = False + if re.search("PASS:: Absorption / Fission / Scatter Ratios maintained with \d+% tolerance", self.results): + passed1 = True + if re.search("PASS:: Collision to Facet Crossing Ratio maintained even balanced within \d+% tolerance", self.results): + passed2 = True + if re.search("PASS:: No Particles Lost During Run", self.results): + passed3 = True + if re.search("PASS:: Fluence is homogenous across cells with \d+% tolerance", self.results): + passed4 = True + + return (passed1 & passed2 & passed3 & passed4) diff --git a/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/KeccakF.c b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/KeccakF.c new file mode 100644 index 0000000..0d65981 --- /dev/null +++ b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/KeccakF.c @@ -0,0 +1,393 @@ +/* +GPU Implementation of Keccak by Guillaume Sevestre, 2010 + +This code is hereby put in the public domain. +It is given as is, without any guarantee. +*/ + +#include +#include + +#include "KeccakF.h" +#include "KeccakTree.h" + +// 22 rounds constants +// Study constant memory best placement with Cuda (textures ?) + + + +const tKeccakLane KeccakF_RoundConstantsCPU[22] = +{ + (tKeccakLane)0x00000001 , + (tKeccakLane)0x00008082 , + (tKeccakLane)0x0000808a , + (tKeccakLane)0x80008000 , + (tKeccakLane)0x0000808b , + (tKeccakLane)0x80000001 , + (tKeccakLane)0x80008081 , + (tKeccakLane)0x00008009 , + (tKeccakLane)0x0000008a , + (tKeccakLane)0x00000088 , + (tKeccakLane)0x80008009 , + (tKeccakLane)0x8000000a , + (tKeccakLane)0x8000808b , + (tKeccakLane)0x0000008b , + (tKeccakLane)0x00008089 , + (tKeccakLane)0x00008003 , + (tKeccakLane)0x00008002 , + (tKeccakLane)0x00000080 , + (tKeccakLane)0x0000800a , + (tKeccakLane)0x8000000a , + (tKeccakLane)0x80008081 , + (tKeccakLane)0x00008080 +}; + + + +//INFO It could be more optimized to use unsigned char on an 8-bit CPU +const unsigned int KeccakF_RotationConstants[25] = +{ + // 1, 3, 6, 10, 15, 21, 28, 36, 45, 55, 2, 14, 27, 41, 56, 8, 25, 43, 62, 18, 39, 61, 20, 44 + 1, 3, 6, 10, 15, 21, 28, 4, 13, 23, 2, 14, 27, 9, 24, 8, 25, 11, 30, 18, 7, 29 , 20, 12 +}; + +//INFO It could be more optimized to use unsigned char on an 8-bit CPU +const unsigned int KeccakF_PiLane[25] = +{ + 10, 7, 11, 17, 18, 3, 5, 16, 8, 21, 24, 4, 15, 23, 19, 13, 12, 2, 20, 14, 22, 9, 6, 1 +}; + +//INFO It could be more optimized to use unsigned char on an 8-bit CPU +const unsigned int KeccakF_Mod5[10] = +{ + 0, 1, 2, 3, 4, 0, 1, 2, 3, 4 +}; + + + + void KeccakF( tKeccakLane * state) +{ + unsigned int x,y,round; //try to avoid to many registers + tKeccakLane BC[5]; + tKeccakLane temp; + + for ( round = 0; round < cKeccakNumberOfRounds; ++round ) + { + // Theta + for ( x = 0; x < 5; ++x ) // derouler + { + BC[x] = state[x] ^ state[5 + x] ^ state[10 + x] ^ state[15 + x] ^ state[20 + x]; + } + for ( x = 0; x < 5; ++x ) // derouler + { + temp = BC[KeccakF_Mod5[x+4]] ^ ROL32(BC[KeccakF_Mod5[x+1]], 1); //expliciter + for ( y = 0; y < 25; y += 5 ) // derouler + { + state[y + x] ^= temp; + } + } + + // Rho Pi + temp = state[1]; + for ( x = 0; x < 24; ++x ) //expliciter + rotation modulo 32 + { + BC[0] = state[KeccakF_PiLane[x]]; + state[KeccakF_PiLane[x]] = ROL32( temp, KeccakF_RotationConstants[x] ); + temp = BC[0]; + } + + // Chi + for ( y = 0; y < 25; y += 5 ) + { + BC[0] = state[y + 0]; + BC[1] = state[y + 1]; + BC[2] = state[y + 2]; + BC[3] = state[y + 3]; + BC[4] = state[y + 4]; + for ( x = 0; x < 5; ++x ) + { + state[y + x] = BC[x] ^((~BC[KeccakF_Mod5[x+1]]) & BC[KeccakF_Mod5[x+2]]); + } + } + + // Iota + state[0] ^= KeccakF_RoundConstantsCPU[round]; + } + + +} + + void KeccakF_CPU( tKeccakLane * state ) +{ + unsigned int round; //try to avoid to many registers + tKeccakLane BC[5]; + tKeccakLane temp; + + for ( round = 0; round < cKeccakNumberOfRounds; ++round ) + { + + { + // Theta + BC[0] = state[0] ^ state[5] ^ state[10] ^ state[15] ^ state[20]; + BC[1] = state[1] ^ state[6] ^ state[11] ^ state[16] ^ state[21]; + BC[2] = state[2] ^ state[7] ^ state[12] ^ state[17] ^ state[22]; + BC[3] = state[3] ^ state[8] ^ state[13] ^ state[18] ^ state[23]; + BC[4] = state[4] ^ state[9] ^ state[14] ^ state[19] ^ state[24]; + + temp = BC[4] ^ ROL32(BC[1], 1);//x=0 + state[0] ^= temp; + state[5] ^= temp; + state[10] ^= temp; + state[15] ^= temp; + state[20] ^= temp; + temp = BC[0] ^ ROL32(BC[2], 1);//x=1 + state[1] ^= temp; + state[6] ^= temp; + state[11] ^= temp; + state[16] ^= temp; + state[21] ^= temp; + temp = BC[1] ^ ROL32(BC[3], 1);//x=2 + state[2] ^= temp; + state[7] ^= temp; + state[12] ^= temp; + state[17] ^= temp; + state[22] ^= temp; + temp = BC[2] ^ ROL32(BC[4], 1);//x=0 + state[3] ^= temp; + state[8] ^= temp; + state[13] ^= temp; + state[18] ^= temp; + state[23] ^= temp; + temp = BC[3] ^ ROL32(BC[0], 1);//x=0 + state[4] ^= temp; + state[9] ^= temp; + state[14] ^= temp; + state[19] ^= temp; + state[24] ^= temp; + }//end Theta + + { + // Rho Pi + temp = state[1]; + BC[0] = state[10]; + state[10] = ROL32( temp, 1); + temp = BC[0];//x=0 + BC[0] = state[7]; + state[7] = ROL32( temp, 3); + temp = BC[0]; + BC[0] = state[11]; + state[11] = ROL32( temp, 6); + temp = BC[0]; + BC[0] = state[17]; + state[17] = ROL32( temp,10); + temp = BC[0]; + BC[0] = state[18]; + state[18] = ROL32( temp,15); + temp = BC[0]; + BC[0] = state[3]; + state[3] = ROL32( temp,21); + temp = BC[0];//x=5 + BC[0] = state[5]; + state[5] = ROL32( temp,28); + temp = BC[0]; + BC[0] = state[16]; + state[16] = ROL32( temp, 4); + temp = BC[0]; + BC[0] = state[8]; + state[8] = ROL32( temp,13); + temp = BC[0]; + BC[0] = state[21]; + state[21] = ROL32( temp,23); + temp = BC[0]; + BC[0] = state[24]; + state[24] = ROL32( temp, 2); + temp = BC[0];//x=10 + BC[0] = state[4]; + state[4] = ROL32( temp,14); + temp = BC[0]; + BC[0] = state[15]; + state[15] = ROL32( temp,27); + temp = BC[0]; + BC[0] = state[23]; + state[23] = ROL32( temp, 9); + temp = BC[0]; + BC[0] = state[19]; + state[19] = ROL32( temp,24); + temp = BC[0]; + BC[0] = state[13]; + state[13] = ROL32( temp, 8); + temp = BC[0];//x=15 + BC[0] = state[12]; + state[12] = ROL32( temp,25); + temp = BC[0]; + BC[0] = state[2]; + state[2] = ROL32( temp,11); + temp = BC[0]; + BC[0] = state[20]; + state[20] = ROL32( temp,30); + temp = BC[0]; + BC[0] = state[14]; + state[14] = ROL32( temp,18); + temp = BC[0]; + BC[0] = state[22]; + state[22] = ROL32( temp, 7); + temp = BC[0];//x=20 + BC[0] = state[9]; + state[9] = ROL32( temp,29); + temp = BC[0]; + BC[0] = state[6]; + state[6] = ROL32( temp,20); + temp = BC[0]; + BC[0] = state[1]; + state[1] = ROL32( temp,12); + temp = BC[0];//x=23 + }//end Rho Pi + + { + // Chi + BC[0] = state[0]; + BC[1] = state[1]; + BC[2] = state[2]; + BC[3] = state[3]; + BC[4] = state[4]; + state[0] = BC[0] ^((~BC[1]) & BC[2]); + state[1] = BC[1] ^((~BC[2]) & BC[3]); + state[2] = BC[2] ^((~BC[3]) & BC[4]); + state[3] = BC[3] ^((~BC[4]) & BC[0]); + state[4] = BC[4] ^((~BC[0]) & BC[1]); + BC[0] = state[5]; + BC[1] = state[6]; + BC[2] = state[7]; + BC[3] = state[8]; + BC[4] = state[9]; + state[5] = BC[0] ^((~BC[1]) & BC[2]); + state[6] = BC[1] ^((~BC[2]) & BC[3]); + state[7] = BC[2] ^((~BC[3]) & BC[4]); + state[8] = BC[3] ^((~BC[4]) & BC[0]); + state[9] = BC[4] ^((~BC[0]) & BC[1]); + BC[0] = state[10]; + BC[1] = state[11]; + BC[2] = state[12]; + BC[3] = state[13]; + BC[4] = state[14]; + state[10] = BC[0] ^((~BC[1]) & BC[2]); + state[11] = BC[1] ^((~BC[2]) & BC[3]); + state[12] = BC[2] ^((~BC[3]) & BC[4]); + state[13] = BC[3] ^((~BC[4]) & BC[0]); + state[14] = BC[4] ^((~BC[0]) & BC[1]); + BC[0] = state[15]; + BC[1] = state[16]; + BC[2] = state[17]; + BC[3] = state[18]; + BC[4] = state[19]; + state[15] = BC[0] ^((~BC[1]) & BC[2]); + state[16] = BC[1] ^((~BC[2]) & BC[3]); + state[17] = BC[2] ^((~BC[3]) & BC[4]); + state[18] = BC[3] ^((~BC[4]) & BC[0]); + state[19] = BC[4] ^((~BC[0]) & BC[1]); + BC[0] = state[20]; + BC[1] = state[21]; + BC[2] = state[22]; + BC[3] = state[23]; + BC[4] = state[24]; + state[20] = BC[0] ^((~BC[1]) & BC[2]); + state[21] = BC[1] ^((~BC[2]) & BC[3]); + state[22] = BC[2] ^((~BC[3]) & BC[4]); + state[23] = BC[3] ^((~BC[4]) & BC[0]); + state[24] = BC[4] ^((~BC[0]) & BC[1]); + }//end Chi + + // Iota + state[0] ^= KeccakF_RoundConstantsCPU[round]; + } + +} + + + + + +// Absorb blocks in top of tree keccak hash function +// inBuffer supposed to have block_number * output_block_size of data + void Keccak_top(tKeccakLane * Kstate, tKeccakLane *inBuffer , int block_number) +{ + int ind_word,k; + + for (k=0;k>(32-offset) ) ) + +//implementation of Keccak function on CPU +void KeccakF( tKeccakLane * state ); + +//implementation of Keccak function on CPU, unrolled +void KeccakF_CPU( tKeccakLane * state ); + +//set the state to zero +void zeroize( tKeccakLane * state ); + +//Keccak final node hashing results of previous nodes in sequential mode +// inBuffer supposed to have block_number * output_block_size of data +void Keccak_top(tKeccakLane * Kstate, tKeccakLane *inBuffer , int block_number); + +//test equility of 2 keccak states +int isEqual_KS(tKeccakLane * Ks1, tKeccakLane * Ks2); + +//print functions +void print_KS(tKeccakLane * state); +void print_KS_256(tKeccakLane * state); + +#endif // KECCAKF_H_INCLUDED diff --git a/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/KeccakTree.h b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/KeccakTree.h new file mode 100644 index 0000000..7193fe3 --- /dev/null +++ b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/KeccakTree.h @@ -0,0 +1,29 @@ +/* +GPU Implementation of Keccak by Guillaume Sevestre, 2010 + +This code is hereby put in the public domain. +It is given as is, without any guarantee. +*/ + +#ifndef KECCAKTREE_H_INCLUDED +#define KECCAKTREE_H_INCLUDED + +#define NB_THREADS 64 // 96 // 192 // Numbers of threads PER BLOCK MUST BE a multiple of NB_SNCD_STAGE_NODES + //MUST BE > 8 for streamcipher mode + +#define NB_THREADS_BLOCKS 64 // 64 // 32 + +#define NB_STREAMS 2 // 4 // 2 MUST DIVIDE NB_THREADS_BLOCKS + +#define INPUT_BLOCK_SIZE_B 32 // 256 bits in : 32 Bytes MUST BE multiple of 4 +#define OUTPUT_BLOCK_SIZE_B 32 // 256 bits out of each keccak hash MUST BE multiple of 4 +#define NB_INPUT_BLOCK 64 // 128 // 64 number of input block of 256 bits + +// 2 stage Treehash +#define NB_SCND_STAGE_THREADS 16 // MUST DIVIDE NB_THREADS +#define NB_INPUT_BLOCK_SNCD_STAGE 2*NB_THREADS/NB_SCND_STAGE_THREADS // + +//StreamCipher +#define SC_NB_OUTPUT_BLOCK 64 // number of output blocks in stream cipher mode + +#endif // KECCAKTREE_H_INCLUDED diff --git a/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/KeccakTreeCPU.c b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/KeccakTreeCPU.c new file mode 100644 index 0000000..f45e039 --- /dev/null +++ b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/KeccakTreeCPU.c @@ -0,0 +1,195 @@ +/* +GPU Implementation of Keccak by Guillaume Sevestre, 2010 + +This code is hereby put in the public domain. +It is given as is, without any guarantee. +*/ + +#include +#include +#include + +#include "KeccakTreeCPU.h" +#include "KeccakF.h" + +void KeccakTreeCPU(tKeccakLane * inBuffer, tKeccakLane * outBuffer) +{ + + //int thread_i, + int thrIdx,blkIdx; + int k,ind_word; + + + for(blkIdx=0;blkIdx +#include +#include + + +//#include "KeccakTreeGPU.h" + +#include "KeccakTree.h" +#include "KeccakF.h" + + +extern "C" +__host__ void checkCUDAError(const char *msg) +{ + hipError_t err = hipGetLastError(); + if( hipSuccess != err) + { + fprintf(stderr, "Cuda error: %s: %s.\n", msg, + hipGetErrorString( err) ); + exit(EXIT_FAILURE); + } +} + +//GPU constants + __constant__ tKeccakLane KeccakF_RoundConstants[22] = +{ + (tKeccakLane)0x00000001 , + (tKeccakLane)0x00008082 , + (tKeccakLane)0x0000808a , + (tKeccakLane)0x80008000 , + (tKeccakLane)0x0000808b , + (tKeccakLane)0x80000001 , + (tKeccakLane)0x80008081 , + (tKeccakLane)0x00008009 , + (tKeccakLane)0x0000008a , + (tKeccakLane)0x00000088 , + (tKeccakLane)0x80008009 , + (tKeccakLane)0x8000000a , + (tKeccakLane)0x8000808b , + (tKeccakLane)0x0000008b , + (tKeccakLane)0x00008089 , + (tKeccakLane)0x00008003 , + (tKeccakLane)0x00008002 , + (tKeccakLane)0x00000080 , + (tKeccakLane)0x0000800a , + (tKeccakLane)0x8000000a , + (tKeccakLane)0x80008081 , + (tKeccakLane)0x00008080 +}; + +//host constants + tKeccakLane KeccakF_RoundConstants_h[22] = +{ + (tKeccakLane)0x00000001 , + (tKeccakLane)0x00008082 , + (tKeccakLane)0x0000808a , + (tKeccakLane)0x80008000 , + (tKeccakLane)0x0000808b , + (tKeccakLane)0x80000001 , + (tKeccakLane)0x80008081 , + (tKeccakLane)0x00008009 , + (tKeccakLane)0x0000008a , + (tKeccakLane)0x00000088 , + (tKeccakLane)0x80008009 , + (tKeccakLane)0x8000000a , + (tKeccakLane)0x8000808b , + (tKeccakLane)0x0000008b , + (tKeccakLane)0x00008089 , + (tKeccakLane)0x00008003 , + (tKeccakLane)0x00008002 , + (tKeccakLane)0x00000080 , + (tKeccakLane)0x0000800a , + (tKeccakLane)0x8000000a , + (tKeccakLane)0x80008081 , + (tKeccakLane)0x00008080 +}; + +// Device (GPU) Keccak-f function implementation +// unrolled +__device__ void KeccakFunr( tKeccakLane * state ) +{ + unsigned int round; //try to avoid to many registers + tKeccakLane BC[5]; + tKeccakLane temp; + + for ( round = 0; round < cKeccakNumberOfRounds; ++round ) + { + + { + // Theta + BC[0] = state[0] ^ state[5] ^ state[10] ^ state[15] ^ state[20]; + BC[1] = state[1] ^ state[6] ^ state[11] ^ state[16] ^ state[21]; + BC[2] = state[2] ^ state[7] ^ state[12] ^ state[17] ^ state[22]; + BC[3] = state[3] ^ state[8] ^ state[13] ^ state[18] ^ state[23]; + BC[4] = state[4] ^ state[9] ^ state[14] ^ state[19] ^ state[24]; + + temp = BC[4] ^ ROL32(BC[1], 1);//x=0 + state[0] ^= temp; + state[5] ^= temp; + state[10] ^= temp; + state[15] ^= temp; + state[20] ^= temp; + temp = BC[0] ^ ROL32(BC[2], 1);//x=1 + state[1] ^= temp; + state[6] ^= temp; + state[11] ^= temp; + state[16] ^= temp; + state[21] ^= temp; + temp = BC[1] ^ ROL32(BC[3], 1);//x=2 + state[2] ^= temp; + state[7] ^= temp; + state[12] ^= temp; + state[17] ^= temp; + state[22] ^= temp; + temp = BC[2] ^ ROL32(BC[4], 1);//x=3 + state[3] ^= temp; + state[8] ^= temp; + state[13] ^= temp; + state[18] ^= temp; + state[23] ^= temp; + temp = BC[3] ^ ROL32(BC[0], 1);//x=4 + state[4] ^= temp; + state[9] ^= temp; + state[14] ^= temp; + state[19] ^= temp; + state[24] ^= temp; + }//end Theta + + { + // Rho Pi + temp = state[1]; + BC[0] = state[10]; + state[10] = ROL32( temp, 1); + temp = BC[0];//x=0 + BC[0] = state[7]; + state[7] = ROL32( temp, 3); + temp = BC[0]; + BC[0] = state[11]; + state[11] = ROL32( temp, 6); + temp = BC[0]; + BC[0] = state[17]; + state[17] = ROL32( temp,10); + temp = BC[0]; + BC[0] = state[18]; + state[18] = ROL32( temp,15); + temp = BC[0]; + BC[0] = state[3]; + state[3] = ROL32( temp,21); + temp = BC[0];//x=5 + BC[0] = state[5]; + state[5] = ROL32( temp,28); + temp = BC[0]; + BC[0] = state[16]; + state[16] = ROL32( temp, 4); + temp = BC[0]; + BC[0] = state[8]; + state[8] = ROL32( temp,13); + temp = BC[0]; + BC[0] = state[21]; + state[21] = ROL32( temp,23); + temp = BC[0]; + BC[0] = state[24]; + state[24] = ROL32( temp, 2); + temp = BC[0];//x=10 + BC[0] = state[4]; + state[4] = ROL32( temp,14); + temp = BC[0]; + BC[0] = state[15]; + state[15] = ROL32( temp,27); + temp = BC[0]; + BC[0] = state[23]; + state[23] = ROL32( temp, 9); + temp = BC[0]; + BC[0] = state[19]; + state[19] = ROL32( temp,24); + temp = BC[0]; + BC[0] = state[13]; + state[13] = ROL32( temp, 8); + temp = BC[0];//x=15 + BC[0] = state[12]; + state[12] = ROL32( temp,25); + temp = BC[0]; + BC[0] = state[2]; + state[2] = ROL32( temp,11); + temp = BC[0]; + BC[0] = state[20]; + state[20] = ROL32( temp,30); + temp = BC[0]; + BC[0] = state[14]; + state[14] = ROL32( temp,18); + temp = BC[0]; + BC[0] = state[22]; + state[22] = ROL32( temp, 7); + temp = BC[0];//x=20 + BC[0] = state[9]; + state[9] = ROL32( temp,29); + temp = BC[0]; + BC[0] = state[6]; + state[6] = ROL32( temp,20); + temp = BC[0]; + BC[0] = state[1]; + state[1] = ROL32( temp,12); + temp = BC[0];//x=23 + }//end Rho Pi + + { + // Chi + BC[0] = state[0]; + BC[1] = state[1]; + BC[2] = state[2]; + BC[3] = state[3]; + BC[4] = state[4]; + state[0] = BC[0] ^((~BC[1]) & BC[2]); + state[1] = BC[1] ^((~BC[2]) & BC[3]); + state[2] = BC[2] ^((~BC[3]) & BC[4]); + state[3] = BC[3] ^((~BC[4]) & BC[0]); + state[4] = BC[4] ^((~BC[0]) & BC[1]); + BC[0] = state[5]; + BC[1] = state[6]; + BC[2] = state[7]; + BC[3] = state[8]; + BC[4] = state[9]; + state[5] = BC[0] ^((~BC[1]) & BC[2]); + state[6] = BC[1] ^((~BC[2]) & BC[3]); + state[7] = BC[2] ^((~BC[3]) & BC[4]); + state[8] = BC[3] ^((~BC[4]) & BC[0]); + state[9] = BC[4] ^((~BC[0]) & BC[1]); + BC[0] = state[10]; + BC[1] = state[11]; + BC[2] = state[12]; + BC[3] = state[13]; + BC[4] = state[14]; + state[10] = BC[0] ^((~BC[1]) & BC[2]); + state[11] = BC[1] ^((~BC[2]) & BC[3]); + state[12] = BC[2] ^((~BC[3]) & BC[4]); + state[13] = BC[3] ^((~BC[4]) & BC[0]); + state[14] = BC[4] ^((~BC[0]) & BC[1]); + BC[0] = state[15]; + BC[1] = state[16]; + BC[2] = state[17]; + BC[3] = state[18]; + BC[4] = state[19]; + state[15] = BC[0] ^((~BC[1]) & BC[2]); + state[16] = BC[1] ^((~BC[2]) & BC[3]); + state[17] = BC[2] ^((~BC[3]) & BC[4]); + state[18] = BC[3] ^((~BC[4]) & BC[0]); + state[19] = BC[4] ^((~BC[0]) & BC[1]); + BC[0] = state[20]; + BC[1] = state[21]; + BC[2] = state[22]; + BC[3] = state[23]; + BC[4] = state[24]; + state[20] = BC[0] ^((~BC[1]) & BC[2]); + state[21] = BC[1] ^((~BC[2]) & BC[3]); + state[22] = BC[2] ^((~BC[3]) & BC[4]); + state[23] = BC[3] ^((~BC[4]) & BC[0]); + state[24] = BC[4] ^((~BC[0]) & BC[1]); + }//end Chi + + // Iota + state[0] ^= KeccakF_RoundConstants[round]; + } + +} +//end unrolled + +//Host Keccak-f function (pb with using the same constants between host and device) +//unrolled +__host__ void KeccakFunr_h( tKeccakLane * state ) +{ + unsigned int round; //try to avoid to many registers + tKeccakLane BC[5]; + tKeccakLane temp; + + for ( round = 0; round < cKeccakNumberOfRounds; ++round ) + { + + { + // Theta + BC[0] = state[0] ^ state[5] ^ state[10] ^ state[15] ^ state[20]; + BC[1] = state[1] ^ state[6] ^ state[11] ^ state[16] ^ state[21]; + BC[2] = state[2] ^ state[7] ^ state[12] ^ state[17] ^ state[22]; + BC[3] = state[3] ^ state[8] ^ state[13] ^ state[18] ^ state[23]; + BC[4] = state[4] ^ state[9] ^ state[14] ^ state[19] ^ state[24]; + + temp = BC[4] ^ ROL32(BC[1], 1);//x=0 + state[0] ^= temp; + state[5] ^= temp; + state[10] ^= temp; + state[15] ^= temp; + state[20] ^= temp; + temp = BC[0] ^ ROL32(BC[2], 1);//x=1 + state[1] ^= temp; + state[6] ^= temp; + state[11] ^= temp; + state[16] ^= temp; + state[21] ^= temp; + temp = BC[1] ^ ROL32(BC[3], 1);//x=2 + state[2] ^= temp; + state[7] ^= temp; + state[12] ^= temp; + state[17] ^= temp; + state[22] ^= temp; + temp = BC[2] ^ ROL32(BC[4], 1);//x=3 + state[3] ^= temp; + state[8] ^= temp; + state[13] ^= temp; + state[18] ^= temp; + state[23] ^= temp; + temp = BC[3] ^ ROL32(BC[0], 1);//x=4 + state[4] ^= temp; + state[9] ^= temp; + state[14] ^= temp; + state[19] ^= temp; + state[24] ^= temp; + }//end Theta + + { + // Rho Pi + temp = state[1]; + BC[0] = state[10]; + state[10] = ROL32( temp, 1); + temp = BC[0];//x=0 + BC[0] = state[7]; + state[7] = ROL32( temp, 3); + temp = BC[0]; + BC[0] = state[11]; + state[11] = ROL32( temp, 6); + temp = BC[0]; + BC[0] = state[17]; + state[17] = ROL32( temp,10); + temp = BC[0]; + BC[0] = state[18]; + state[18] = ROL32( temp,15); + temp = BC[0]; + BC[0] = state[3]; + state[3] = ROL32( temp,21); + temp = BC[0];//x=5 + BC[0] = state[5]; + state[5] = ROL32( temp,28); + temp = BC[0]; + BC[0] = state[16]; + state[16] = ROL32( temp, 4); + temp = BC[0]; + BC[0] = state[8]; + state[8] = ROL32( temp,13); + temp = BC[0]; + BC[0] = state[21]; + state[21] = ROL32( temp,23); + temp = BC[0]; + BC[0] = state[24]; + state[24] = ROL32( temp, 2); + temp = BC[0];//x=10 + BC[0] = state[4]; + state[4] = ROL32( temp,14); + temp = BC[0]; + BC[0] = state[15]; + state[15] = ROL32( temp,27); + temp = BC[0]; + BC[0] = state[23]; + state[23] = ROL32( temp, 9); + temp = BC[0]; + BC[0] = state[19]; + state[19] = ROL32( temp,24); + temp = BC[0]; + BC[0] = state[13]; + state[13] = ROL32( temp, 8); + temp = BC[0];//x=15 + BC[0] = state[12]; + state[12] = ROL32( temp,25); + temp = BC[0]; + BC[0] = state[2]; + state[2] = ROL32( temp,11); + temp = BC[0]; + BC[0] = state[20]; + state[20] = ROL32( temp,30); + temp = BC[0]; + BC[0] = state[14]; + state[14] = ROL32( temp,18); + temp = BC[0]; + BC[0] = state[22]; + state[22] = ROL32( temp, 7); + temp = BC[0];//x=20 + BC[0] = state[9]; + state[9] = ROL32( temp,29); + temp = BC[0]; + BC[0] = state[6]; + state[6] = ROL32( temp,20); + temp = BC[0]; + BC[0] = state[1]; + state[1] = ROL32( temp,12); + temp = BC[0];//x=23 + }//end Rho Pi + + { + // Chi + BC[0] = state[0]; + BC[1] = state[1]; + BC[2] = state[2]; + BC[3] = state[3]; + BC[4] = state[4]; + state[0] = BC[0] ^((~BC[1]) & BC[2]); + state[1] = BC[1] ^((~BC[2]) & BC[3]); + state[2] = BC[2] ^((~BC[3]) & BC[4]); + state[3] = BC[3] ^((~BC[4]) & BC[0]); + state[4] = BC[4] ^((~BC[0]) & BC[1]); + BC[0] = state[5]; + BC[1] = state[6]; + BC[2] = state[7]; + BC[3] = state[8]; + BC[4] = state[9]; + state[5] = BC[0] ^((~BC[1]) & BC[2]); + state[6] = BC[1] ^((~BC[2]) & BC[3]); + state[7] = BC[2] ^((~BC[3]) & BC[4]); + state[8] = BC[3] ^((~BC[4]) & BC[0]); + state[9] = BC[4] ^((~BC[0]) & BC[1]); + BC[0] = state[10]; + BC[1] = state[11]; + BC[2] = state[12]; + BC[3] = state[13]; + BC[4] = state[14]; + state[10] = BC[0] ^((~BC[1]) & BC[2]); + state[11] = BC[1] ^((~BC[2]) & BC[3]); + state[12] = BC[2] ^((~BC[3]) & BC[4]); + state[13] = BC[3] ^((~BC[4]) & BC[0]); + state[14] = BC[4] ^((~BC[0]) & BC[1]); + BC[0] = state[15]; + BC[1] = state[16]; + BC[2] = state[17]; + BC[3] = state[18]; + BC[4] = state[19]; + state[15] = BC[0] ^((~BC[1]) & BC[2]); + state[16] = BC[1] ^((~BC[2]) & BC[3]); + state[17] = BC[2] ^((~BC[3]) & BC[4]); + state[18] = BC[3] ^((~BC[4]) & BC[0]); + state[19] = BC[4] ^((~BC[0]) & BC[1]); + BC[0] = state[20]; + BC[1] = state[21]; + BC[2] = state[22]; + BC[3] = state[23]; + BC[4] = state[24]; + state[20] = BC[0] ^((~BC[1]) & BC[2]); + state[21] = BC[1] ^((~BC[2]) & BC[3]); + state[22] = BC[2] ^((~BC[3]) & BC[4]); + state[23] = BC[3] ^((~BC[4]) & BC[0]); + state[24] = BC[4] ^((~BC[0]) & BC[1]); + }//end Chi + + // Iota + state[0] ^= KeccakF_RoundConstants_h[round]; + } + +} +//end unrolled + +//Keccak final node hashing results of previous nodes in sequential mode +__host__ void Keccak_top_GPU(tKeccakLane * Kstate, tKeccakLane *inBuffer , int block_number) +{ + int ind_word,k; + + for (k=0;k 8 + if(threadIdx.x <8) + { + Key[threadIdx.x]= d_KeyNonce_inBuffer[threadIdx.x]; + Nonce[threadIdx.x]=d_KeyNonce_inBuffer[threadIdx.x + 8 ]; + } + __syncthreads(); + + //zeroize the state + for(ind_word=0; ind_word<25; ind_word++) {Kstate[ind_word]=0; } + + //input the key + //xor Key into state from shared mem + for (ind_word=0; ind_word<8; ind_word++) + { + Kstate[ind_word] ^=Key[ind_word]; + } + //apply GPU Keccak permutation + KeccakFunr(Kstate); + + //input the Nonce + //xor Key into state from shared mem + for (ind_word=0; ind_word<8; ind_word++) + { + Kstate[ind_word] ^=Nonce[ind_word]; + } + //apply GPU Keccak permutation + KeccakFunr(Kstate); + + //input the threadIdx.x AND blockIdx.x to have different KeyStreams + Kstate[0]^= threadIdx.x ; + Kstate[1]^= blockIdx.x ; + KeccakFunr(Kstate); + + + for (k=0;k= 8800 GT), even without the cuda toolkit should be ok with the executable (to be renamed in .exe) and the runtime dll included in release folder of VS project. + + +Compiling under Linux : +After installing Cuda SDK and toolkit, use the included Makefile. the Path to cuda SDK and toolkit must be changed in this Makefile. Beware of the 64 bits target used (could be easily changed to 32 bits). + +Description of Source files: + +KeccakF.c : Implementation of Keccak function on CPU + +KeccakTree.h : Definition of constants (number of threads, size of input blocks, output blocks, etc...) + +KeccakTreeCPU.c : Implementation of Keccak tree hash mode on CPU only + +KeccakTreeGPU.cu : Cuda code of implementation of Keccak tree hash on GPU + +KeccakTypes.h : Types definition + +main.c : main file + +Test.c : Implementation of test functions of different modes, performance mesure. + diff --git a/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/Test.c b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/Test.c new file mode 100644 index 0000000..71ee7c0 --- /dev/null +++ b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/Test.c @@ -0,0 +1,1023 @@ +/* +GPU Implementation of Keccak by Guillaume Sevestre, 2010 + +This code is hereby put in the public domain. +It is given as is, without any guarantee. +*/ + +#include +#include +#include + +#include + +//Cuda +#include +#include + + + +#include "KeccakF.h" +#include "KeccakTreeCPU.h" +#include "KeccakTreeGPU.h" + +#define IMAX 1600 //2400 // 1600 for high speed mesures // iteration for speed mesure loops + + +//debug print function +void print_out(tKeccakLane * h_outBuffer,int nb_threads) +{ +printf("%08x ",h_outBuffer[0]);printf("%08x ",h_outBuffer[1]); +printf("%08x ",h_outBuffer[nb_threads]);printf("%08x ",h_outBuffer[nb_threads +1]); +printf("\n\n"); +} + + + +void TestCPU(int reduc) +{ + time_t t1,t2; + double speed1; + int i; + + tKeccakLane *h_inBuffer;// Host in buffer for data to be hashed + tKeccakLane *h_outBuffer;// Host out buffer + + tKeccakLane Kstate[25]; //Keccak State for top node + memset(Kstate, 0, 25 * sizeof(tKeccakLane)); + + + //init host inBuffer + h_inBuffer=(tKeccakLane *) malloc( INPUT_BLOCK_SIZE_B * NB_THREADS*NB_THREADS_BLOCKS * NB_INPUT_BLOCK ); + memset(h_inBuffer, 0, INPUT_BLOCK_SIZE_B * NB_THREADS*NB_THREADS_BLOCKS * NB_INPUT_BLOCK); + + //init host outBuffer + h_outBuffer=(tKeccakLane *) malloc( OUTPUT_BLOCK_SIZE_B * NB_THREADS*NB_THREADS_BLOCKS ); + memset(h_outBuffer, 0, OUTPUT_BLOCK_SIZE_B * NB_THREADS*NB_THREADS_BLOCKS ); + + //*************************** + //init h_inBuffer with values + for(i=0;i + + + + + + + + + diff --git a/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/Release/VCKeccakTree.pdb b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/Release/VCKeccakTree.pdb new file mode 100644 index 0000000..ccfece5 Binary files /dev/null and b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/Release/VCKeccakTree.pdb differ diff --git a/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/Release/cudart32_30_14.dll b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/Release/cudart32_30_14.dll new file mode 100644 index 0000000..380d7af Binary files /dev/null and b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/Release/cudart32_30_14.dll differ diff --git a/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/Release/main.obj b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/Release/main.obj new file mode 100644 index 0000000..323cc86 Binary files /dev/null and b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/Release/main.obj differ diff --git a/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/Release/mt.dep b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/Release/mt.dep new file mode 100644 index 0000000..56cecb2 --- /dev/null +++ b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/Release/mt.dep @@ -0,0 +1 @@ +Manifest resource last updated at 23:22:52,54 on 28/11/2010 diff --git a/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/Release/vc90.idb b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/Release/vc90.idb new file mode 100644 index 0000000..cad23ea Binary files /dev/null and b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/Release/vc90.idb differ diff --git a/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/Release/vc90.pdb b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/Release/vc90.pdb new file mode 100644 index 0000000..0e9fa34 Binary files /dev/null and b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/Release/vc90.pdb differ diff --git a/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/VCKeccakTree.ncb b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/VCKeccakTree.ncb new file mode 100644 index 0000000..8aa3927 Binary files /dev/null and b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/VCKeccakTree.ncb differ diff --git a/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/VCKeccakTree.sln b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/VCKeccakTree.sln new file mode 100644 index 0000000..27b8ca9 --- /dev/null +++ b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/VCKeccakTree.sln @@ -0,0 +1,20 @@ + +Microsoft Visual Studio Solution File, Format Version 10.00 +# Visual Studio 2008 +Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "VCKeccakTree", "VCKeccakTree.vcproj", "{82FDBE31-A332-400C-85CE-8618C34D2F45}" +EndProject +Global + GlobalSection(SolutionConfigurationPlatforms) = preSolution + Debug|Win32 = Debug|Win32 + Release|Win32 = Release|Win32 + EndGlobalSection + GlobalSection(ProjectConfigurationPlatforms) = postSolution + {82FDBE31-A332-400C-85CE-8618C34D2F45}.Debug|Win32.ActiveCfg = Debug|Win32 + {82FDBE31-A332-400C-85CE-8618C34D2F45}.Debug|Win32.Build.0 = Debug|Win32 + {82FDBE31-A332-400C-85CE-8618C34D2F45}.Release|Win32.ActiveCfg = Release|Win32 + {82FDBE31-A332-400C-85CE-8618C34D2F45}.Release|Win32.Build.0 = Release|Win32 + EndGlobalSection + GlobalSection(SolutionProperties) = preSolution + HideSolutionNode = FALSE + EndGlobalSection +EndGlobal diff --git a/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/VCKeccakTree.suo b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/VCKeccakTree.suo new file mode 100644 index 0000000..fbe60bf Binary files /dev/null and b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/VCKeccakTree.suo differ diff --git a/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/VCKeccakTree.vcproj b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/VCKeccakTree.vcproj new file mode 100644 index 0000000..bf87f1a --- /dev/null +++ b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/VCKeccakTree.vcproj @@ -0,0 +1,243 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/VCKeccakTree.vcproj.DELLM24.guigui.user b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/VCKeccakTree.vcproj.DELLM24.guigui.user new file mode 100644 index 0000000..61fa7c7 --- /dev/null +++ b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/VCKeccakTree.vcproj.DELLM24.guigui.user @@ -0,0 +1,65 @@ + + + + + + + + + + + diff --git a/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/vc90.pdb b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/vc90.pdb new file mode 100644 index 0000000..06bee27 Binary files /dev/null and b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/VCKeccakTree/vc90.pdb differ diff --git a/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/main.c b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/main.c new file mode 100644 index 0000000..503faae --- /dev/null +++ b/src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/main.c @@ -0,0 +1,68 @@ +/* +GPU Implementation of Keccak by Guillaume Sevestre, 2010 + +This code is hereby put in the public domain. +It is given as is, without any guarantee. +*/ + +#include +#include +#include + +#include + +//Cuda +#include +#include + + + +#include "KeccakF.h" +#include "KeccakTreeCPU.h" +#include "KeccakTreeGPU.h" +#include "Test.h" + + + + + +int main() +{ + + + + Device_Info(); + + Print_Param(); + + //Test_Completness(); + + /* + TestCPU(1); + + TestGPU(); + + TestGPU_OverlapCPU(); + + //TestGPU_Split(); + + TestGPU_Stream(); + + TestGPU_Stream_OverlapCPU(); + */ + + + + TestCPU_2stg(80); + + TestGPU_2stg(); + + TestGPU_2stg_Stream_OverlapCPU(); + + + + TestGPU_SCipher(); + + + return 0; +} diff --git a/src/hiptestsuite/applications/keccaktreegpu/__init__.py b/src/hiptestsuite/applications/keccaktreegpu/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/hiptestsuite/applications/keccaktreegpu/keccaktreegpu.py b/src/hiptestsuite/applications/keccaktreegpu/keccaktreegpu.py new file mode 100644 index 0000000..66f00ca --- /dev/null +++ b/src/hiptestsuite/applications/keccaktreegpu/keccaktreegpu.py @@ -0,0 +1,119 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.TesterRepository import Tester, Test, TestData +from hiptestsuite.Test import HIPTestData, TestResult, HIP_PLATFORM +from typing import Union, List +from hiptestsuite.test_classifier import TestClassifier +from hiptestsuite.applications.keccaktreegpu.keccaktreegpu_build_amd import BuildRunAmd +from hiptestsuite.applications.keccaktreegpu.keccaktreegpu_build_nvidia import BuildRunNvidia +from hiptestsuite.common.hip_get_packages import HipPackages +from hiptestsuite.common.hip_shell import execshellcmd + +import os +import re +# Common class to clone, set up, build and run test +class PrepareTest(): + def __init__(self, cwd, binary): + self.cwdAbs = cwd + self.binary = binary + self.app_path = os.path.join(self.cwdAbs,\ + "src/hiptestsuite/applications/keccaktreegpu/KeccakTreeGpu/") + self.thistestpath = self.app_path + self.prepareobj = None + + def buildtest(self, logFile, platform): + isBinaryPresent = True + if platform == HIP_PLATFORM.nvidia: + self.prepareobj = BuildRunNvidia(self.thistestpath, logFile, self.binary) + elif platform == HIP_PLATFORM.amd: + self.prepareobj = BuildRunAmd(self.thistestpath, logFile, self.binary) + else: + print("Invalid Platform") + return False + self.prepareobj.buildtest() + + if not os.path.isfile(\ + os.path.join(self.thistestpath, self.binary)): + isBinaryPresent &= False + return isBinaryPresent + + def clean(self): + if self.prepareobj != None: + self.prepareobj.clean() + + def runtest(self): + if self.prepareobj != None: + self.prepareobj.runtest() + + def parse_result(self): + if self.prepareobj != None: + return self.prepareobj.parse_result() + return False + +class KECCAKTREEGPU(TestClassifier): + def __init__(self): + TestClassifier.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + TestClassifier.add_matched_with_names(self, {"keccaktreegpu": matched_with_names}) + +class PERFORMANCE(KECCAKTREEGPU): + def __init__(self): + KECCAKTREEGPU.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + KECCAKTREEGPU.add_matched_with_names(self, {"performance": matched_with_names}) + + +# Test keccaktreetest +class keccaktreetest(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd, "LinKeccakTree") + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + perf = PERFORMANCE() + perf.add_matched_with_names() + test.classifiers = [perf] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== keccaktreetest Test ===============") + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/keccaktreetest.log", 'w+') as testLogger: + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest() + # Parse the test result + if True == self.parse_result(): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL diff --git a/src/hiptestsuite/applications/keccaktreegpu/keccaktreegpu_build_amd.py b/src/hiptestsuite/applications/keccaktreegpu/keccaktreegpu_build_amd.py new file mode 100644 index 0000000..ed16d94 --- /dev/null +++ b/src/hiptestsuite/applications/keccaktreegpu/keccaktreegpu_build_amd.py @@ -0,0 +1,68 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os +import tempfile +from hiptestsuite.common.hip_shell import * +from hiptestsuite.applications.keccaktreegpu.keccaktreegpu_parser_common import KeccakTreeParser + +class BuildRunAmd(): + def __init__(self, thistestpath, logFile, binary): + self.thistestpath = thistestpath + self.logFile = logFile + self.runlog = None + self.binary = binary + + def getenvironmentvariables(self): + envtoset = os.environ.copy() + envtoset["HIP_PATH"] = "/opt/rocm" + return envtoset + + def buildtest(self): + if not os.path.exists("/opt/rocm"): + print("ROCm not installed. Exiting!") + return False + # In this function put the build steps for test cases + # which differ across platforms (amd/nvidia/intel) + print("Building keccaktreegpu..") + cmdcd = "cd " + self.thistestpath + ";" + env = self.getenvironmentvariables() + cmd_build = "make CFLAGS=-D__HIP_PLATFORM_AMD__ ;" + cmdexc = cmdcd + cmd_build + execshellcmd(cmdexc, self.logFile, env) + + def runtest(self): + print("Running keccaktreegpu..") + env = self.getenvironmentvariables() + cmdexc = "cd " + self.thistestpath + ";" + "./" + self.binary + ";" + self.runlog = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, self.runlog, env) + + def clean(self): + print("Cleaning keccaktreegpu..") + cmdcd = "cd " + self.thistestpath + ";" + cmdclean = "make clean;" + cmdexc = cmdcd + cmdclean + if self.runlog != None: + self.runlog.close() + execshellcmd(cmdexc, None, None) + + def parse_result(self): + return KeccakTreeParser(self.runlog).parse() diff --git a/src/hiptestsuite/applications/keccaktreegpu/keccaktreegpu_build_nvidia.py b/src/hiptestsuite/applications/keccaktreegpu/keccaktreegpu_build_nvidia.py new file mode 100644 index 0000000..fc8e888 --- /dev/null +++ b/src/hiptestsuite/applications/keccaktreegpu/keccaktreegpu_build_nvidia.py @@ -0,0 +1,71 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os +import tempfile +from hiptestsuite.common.hip_shell import * +from hiptestsuite.applications.keccaktreegpu.keccaktreegpu_parser_common import KeccakTreeParser + +class BuildRunNvidia(): + def __init__(self, thistestpath, logFile, binary): + self.thistestpath = thistestpath + self.logFile = logFile + self.runlog = None + self.binary = binary + + def getenvironmentvariables(self): + envtoset = os.environ.copy() + envtoset["HIP_PLATFORM"] = "nvidia" + envtoset["HIP_COMPILER"] = "nvcc" + envtoset["HIP_RUNTIME"] = "cuda" + envtoset["HIP_PATH"] = "/opt/rocm" + return envtoset + + def buildtest(self): + if not os.path.exists("/opt/rocm"): + print("ROCm not installed. Exiting!") + return False + # In this function put the build steps for test cases + # which differ across platforms (amd/nvidia/intel) + print("Building keccaktreegpu..") + cmdcd = "cd " + self.thistestpath + ";" + env = self.getenvironmentvariables() + cmd_build = "make CFLAGS=-D__HIP_PLATFORM_NVIDIA__ ;" + cmdexc = cmdcd + cmd_build + execshellcmd(cmdexc, self.logFile, env) + + def runtest(self): + print("Running keccaktreegpu..") + env = self.getenvironmentvariables() + cmdexc = "cd " + self.thistestpath + ";" + "./" + self.binary + ";" + self.runlog = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, self.runlog, env) + + def clean(self): + print("Cleaning keccaktreegpu..") + cmdcd = "cd " + self.thistestpath + ";" + cmdclean = "make clean;" + cmdexc = cmdcd + cmdclean + if self.runlog != None: + self.runlog.close() + execshellcmd(cmdexc, None, None) + + def parse_result(self): + return KeccakTreeParser(self.runlog).parse() diff --git a/src/hiptestsuite/applications/keccaktreegpu/keccaktreegpu_parser_common.py b/src/hiptestsuite/applications/keccaktreegpu/keccaktreegpu_parser_common.py new file mode 100644 index 0000000..8cd4a24 --- /dev/null +++ b/src/hiptestsuite/applications/keccaktreegpu/keccaktreegpu_parser_common.py @@ -0,0 +1,45 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import re +from hiptestsuite.common.hip_shell import execshellcmd + +class KeccakTreeParser(): + def __init__(self, results): + results.seek(0) + logbytes = results.read() + self.results = logbytes + + def parse(self): + passedts1 = False + passedts2 = False + passedts3 = False + passedts4 = False + + if re.search("CPU_2stg speed :\s*\d+\.\d+\s*kB/s", self.results): + passedts1 = True + if re.search("GPU_2stg speed :\s*\d+\.\d+\s*kB/s", self.results): + passedts2 = True + if re.search("GPU_2stg Stream OverlapCPU speed :\s*\d+\.\d+\s*kB/s", self.results): + passedts3 = True + if re.search("GPU SCipher speed :\s*\d+\.\d+\s*kB/s", self.results): + passedts4 = True + + return (passedts1 & passedts2 & passedts3 & passedts4) diff --git a/src/hiptestsuite/applications/mgbench/__init__.py b/src/hiptestsuite/applications/mgbench/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/hiptestsuite/applications/mgbench/mgbench.py b/src/hiptestsuite/applications/mgbench/mgbench.py new file mode 100644 index 0000000..bf34da8 --- /dev/null +++ b/src/hiptestsuite/applications/mgbench/mgbench.py @@ -0,0 +1,242 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.TesterRepository import Tester, Test, TestData +from hiptestsuite.Test import HIPTestData, TestResult, HIP_PLATFORM +from typing import Union, List +from hiptestsuite.test_classifier import TestClassifier +from hiptestsuite.applications.mgbench.mgbench_build_amd import BuildRunAmd +from hiptestsuite.applications.mgbench.mgbench_build_nvidia import BuildRunNvidia +from hiptestsuite.common.hip_get_packages import HipPackages +from hiptestsuite.common.hip_shell import execshellcmd + +import os +import re +# Common class to clone, set up, build and run test +class PrepareTest(): + def __init__(self, cwd, testpath, mgtestfile, binary): + self.cwdAbs = cwd + self.mgtestfile = mgtestfile + self.binary = binary + self.app_path = os.path.join(self.cwdAbs,\ + "src/hiptestsuite/applications/mgbench/") + self.app_root = os.path.join(self.app_path, "mgbench/") + self.thistestpath = os.path.join(self.app_root, testpath) + self.prepareobj = None + self.apprepo = "" # Default + self.appbranch = "" + self.appcommitId = "" + + def set_mgbench_repoinfo(self, test_data: HIPTestData): + validrepconfig = True + if test_data.repos["mgbench"].repo_url != None: + self.apprepo = test_data.repos["mgbench"].repo_url + else: + validrepconfig &= False + if test_data.repos["mgbench"].branch != None: + self.appbranch = test_data.repos["mgbench"].branch + if test_data.repos["mgbench"].commit_id != None: + self.appcommitId = test_data.repos["mgbench"].commit_id + return validrepconfig + + def downloadtest(self, logFile, test_data: HIPTestData): + ret = HipPackages().pull_repo(logFile, self.apprepo,\ + self.appbranch, self.appcommitId, "mgbench") + return ret + + def buildtest(self, logFile, platform): + isBinaryPresent = True + if platform == HIP_PLATFORM.nvidia: + self.prepareobj = BuildRunNvidia(self.app_root, self.thistestpath, logFile, self.mgtestfile, self.binary) + elif platform == HIP_PLATFORM.amd: + self.prepareobj = BuildRunAmd(self.app_root, self.thistestpath, logFile, self.mgtestfile, self.binary) + else: + print("Invalid Platform") + return False + buildstatus = self.prepareobj.buildtest() + if buildstatus == False: + return False + # Check if test binary is created + if not os.path.isfile(\ + os.path.join(self.thistestpath, self.binary)): + isBinaryPresent &= False + return isBinaryPresent + + def clean(self): + if self.prepareobj != None: + self.prepareobj.clean() + + def runtest(self): + if self.prepareobj != None: + self.prepareobj.runtest() + + def parse_result(self, test): + if self.prepareobj != None: + return self.prepareobj.parse_result(test) + return False + +class MGBENCH(TestClassifier): + def __init__(self): + TestClassifier.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + TestClassifier.add_matched_with_names(self, {"mgbench": matched_with_names}) + +class PERFORMANCE(MGBENCH): + def __init__(self): + MGBENCH.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + MGBENCH.add_matched_with_names(self, {"performance": matched_with_names}) + + +# Test src/L1/fullduplex +class mgbench_fullduplex(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd, "src/L1/", "fullduplex.cpp", "fullduplex") + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + perf = PERFORMANCE() + perf.add_matched_with_names() + test.classifiers = [perf] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== mgbench_fullduplex Test ===============") + # Set repo info + isrepocfgvalid = self.set_mgbench_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/mgbench_fullduplex.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest() + # Parse the test result + if True == self.parse_result("fullduplex"): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test src/L1/halfduplex +class mgbench_halfduplex(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd, "src/L1/", "halfduplex.cpp", "halfduplex") + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + perf = PERFORMANCE() + perf.add_matched_with_names() + test.classifiers = [perf] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== mgbench_halfduplex Test ===============") + # Set repo info + isrepocfgvalid = self.set_mgbench_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/mgbench_halfduplex.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest() + # Parse the test result + if True == self.parse_result("halfduplex"): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL + + +# Test src/L1/uva +class mgbench_uva(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + self.cwd = os.getcwd() + PrepareTest.__init__(self, self.cwd, "src/L1/", "uva.cu", "uva") + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + perf = PERFORMANCE() + perf.add_matched_with_names() + test.classifiers = [perf] + test.tester = self + return [test] + + def clean(self): + PrepareTest.clean(self) + + def test(self, test_data: HIPTestData): + print("=============== mgbench_uva Test ===============") + # Set repo info + isrepocfgvalid = self.set_mgbench_repoinfo(test_data) + if not isrepocfgvalid: + test_data.test_result = TestResult.ERROR + return + # Create the log directory + resultLogDir = test_data.log_location + with open(resultLogDir + "/mgbench_uva.log", 'w+') as testLogger: + res = self.downloadtest(testLogger, test_data) + if not res: + test_data.test_result = TestResult.FAIL + return + res = self.buildtest(testLogger, test_data.HIP_PLATFORM) + if not res: + test_data.test_result = TestResult.FAIL + return + self.runtest() + # Parse the test result + if True == self.parse_result("uva"): + test_data.test_result = TestResult.PASS + else: + test_data.test_result = TestResult.FAIL diff --git a/src/hiptestsuite/applications/mgbench/mgbench_build_amd.py b/src/hiptestsuite/applications/mgbench/mgbench_build_amd.py new file mode 100644 index 0000000..47894cb --- /dev/null +++ b/src/hiptestsuite/applications/mgbench/mgbench_build_amd.py @@ -0,0 +1,79 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os +import tempfile +from hiptestsuite.common.hip_shell import * +from hiptestsuite.applications.mgbench.mgbench_parser_common import MgbenchParser + +class BuildRunAmd(): + def __init__(self, app_root, thistestpath, logFile, mgtestfile, binary): + self.app_root = app_root + self.thistestpath = thistestpath + self.logFile = logFile + self.mgtestfile = mgtestfile + self.binary = binary + self.runlog = None + + def buildtest(self): + # In this function put the build steps for test cases + # which differ across platforms (amd/nvidia/intel) + # Build dependencies + if not os.path.exists("/opt/rocm"): + print("ROCm not installed. Exiting!") + return False + print("Building mgbench..") + depfolder = os.path.join(self.app_root, "deps/gflags/") + cmdcd = "cd " + depfolder + ";" + cmdbuild = "cmake .; make clean; make;" + cmdexcdep = cmdcd + cmdbuild + execshellcmd(cmdexcdep, self.logFile, None) + if not os.path.isfile(depfolder + "lib/libgflags.a"): + print("Dependency (gflags) build failed") + return False + # Build Test + mgtestfile_hipified = "hip_" + self.mgtestfile + cmdcd = "cd " + self.thistestpath + ";" + cmd_hipify = "/opt/rocm/bin/hipify-perl " + self.mgtestfile + ">> " + mgtestfile_hipified + ";" + cmd_build = "/opt/rocm/bin/hipcc " + mgtestfile_hipified +\ + " -lgflags -L../../deps/gflags/lib/ -I ../../deps/gflags/include/ -o " + self.binary + ";" + cmd_clean = "rm -f " + mgtestfile_hipified + ";" + cmdexc = cmdcd + cmd_hipify + cmd_build + cmd_clean + execshellcmd(cmdexc, self.logFile, None) + return True + + def runtest(self): + print("Running mgbench..") + cmdexc = "cd " + self.thistestpath + ";" + "./" + self.binary + ";" + envtoset = os.environ.copy() + self.runlog = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, self.runlog, envtoset) + + def clean(self): + print("Cleaning mgbench..") + cmdcd = "cd " + self.thistestpath + ";" + cmdrm = "rm -f " + self.binary + ";" + cmdexc = cmdcd + cmdrm + if self.runlog != None: + self.runlog.close() + execshellcmd(cmdexc, None, None) + + def parse_result(self, test): + return MgbenchParser(self.runlog).parse(test) diff --git a/src/hiptestsuite/applications/mgbench/mgbench_build_nvidia.py b/src/hiptestsuite/applications/mgbench/mgbench_build_nvidia.py new file mode 100644 index 0000000..174f33f --- /dev/null +++ b/src/hiptestsuite/applications/mgbench/mgbench_build_nvidia.py @@ -0,0 +1,86 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os +import tempfile +from hiptestsuite.common.hip_shell import * +from hiptestsuite.applications.mgbench.mgbench_parser_common import MgbenchParser + +class BuildRunNvidia(): + def __init__(self, app_root, thistestpath, logFile, mgtestfile, binary): + self.app_root = app_root + self.thistestpath = thistestpath + self.logFile = logFile + self.mgtestfile = mgtestfile + self.binary = binary + self.runlog = None + + def getenvironmentvariables(self): + envtoset = os.environ.copy() + envtoset["HIP_PLATFORM"] = "nvidia" + envtoset["HIP_COMPILER"] = "nvcc" + envtoset["HIP_RUNTIME"] = "cuda" + return envtoset + + def buildtest(self): + if not os.path.exists("/opt/rocm"): + print("ROCm not installed. Exiting!") + return False + # In this function put the build steps for test cases + # which differ across platforms (amd/nvidia/intel) + # Build dependencies + print("Building mgbench..") + depfolder = os.path.join(self.app_root, "deps/gflags/") + cmdcd = "cd " + depfolder + ";" + cmdbuild = "cmake .; make clean; make;" + cmdexcdep = cmdcd + cmdbuild + execshellcmd(cmdexcdep, self.logFile, None) + if not os.path.isfile(depfolder + "lib/libgflags.a"): + print("Dependency (gflags) build failed") + return False + env = self.getenvironmentvariables() + mgtestfile_hipified = "hip_" + self.mgtestfile + cmdcd = "cd " + self.thistestpath + ";" + cmd_hipify = "/opt/rocm/bin/hipify-perl " + self.mgtestfile + ">> " + mgtestfile_hipified + ";" + cmd_build = "/opt/rocm/bin/hipcc " + mgtestfile_hipified +\ + " -lgflags -L../../deps/gflags/lib/ -I ../../deps/gflags/include/ -o " + self.binary + ";" + cmd_clean = "rm -f " + mgtestfile_hipified + ";" + cmdexc = cmdcd + cmd_hipify + cmd_build + cmd_clean + execshellcmd(cmdexc, self.logFile, env) + return True + + def runtest(self): + print("Running mgbench..") + env = self.getenvironmentvariables() + cmdexc = "cd " + self.thistestpath + ";" + "./" + self.binary + ";" + self.runlog = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logFile, self.runlog, env) + + def clean(self): + print("Cleaning mgbench..") + cmdcd = "cd " + self.thistestpath + ";" + cmdrm = "rm -f " + self.binary + ";" + cmdexc = cmdcd + cmdrm + if self.runlog != None: + self.runlog.close() + execshellcmd(cmdexc, None, None) + + def parse_result(self, test): + return MgbenchParser(self.runlog).parse(test) diff --git a/src/hiptestsuite/applications/mgbench/mgbench_parser_common.py b/src/hiptestsuite/applications/mgbench/mgbench_parser_common.py new file mode 100644 index 0000000..13c556a --- /dev/null +++ b/src/hiptestsuite/applications/mgbench/mgbench_parser_common.py @@ -0,0 +1,53 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os +import re + +class MgbenchParser(): + def __init__(self, results): + results.seek(0) + logbytes = results.read() + self.results = logbytes + + def parse(self, test): + numgpus = 0 + gpumatch = re.search("GPUs: *\d+\s", self.results) + if gpumatch: + gpunum = re.split(":", gpumatch.group(0)) + numgpus = int(gpunum[1]) + else: + return False + + result1 = self.results.count("Exchanging between") + result2 = self.results.count("Copying from") + passed = False + if "fullduplex" == test: + if numgpus > 1: + if result1 > 0: + passed = True + else: + if result1 == 0: + passed = True + else: + if result2 > 0: + passed = True + + return passed diff --git a/src/hiptestsuite/common/__init__.py b/src/hiptestsuite/common/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/hiptestsuite/common/hip_get_packages.py b/src/hiptestsuite/common/hip_get_packages.py new file mode 100644 index 0000000..7f3ac0e --- /dev/null +++ b/src/hiptestsuite/common/hip_get_packages.py @@ -0,0 +1,188 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.common.hip_shell import execshellcmd + +import os + +# Common class to clone/pull dependent Packages +class HipPackages(): + def __init__(self): + self.cwdAbs = os.getcwd() + self.conformancePath = os.path.join(self.cwdAbs, "src/hiptestsuite/conformance/") + # HIP & HIPAMD + self.hippath = os.path.join(self.conformancePath, "HIP/") + self.hipamdpath = os.path.join(self.conformancePath, "HIPAMD/") + # HIP-Examples + self.appPath = os.path.join(self.cwdAbs,"src/hiptestsuite/applications/hip_examples/") + self.examplepath = os.path.join(self.appPath, "HIP-Examples/") + self.mixbenchpath = os.path.join(self.appPath, "mixbench/") + self.gpustrmpath = os.path.join(self.appPath, "GPU-STREAM/") + # Vdi and OpenCL + self.rocclrpath = os.path.join(self.conformancePath, "ROCclr/") + self.openclpath = os.path.join(self.conformancePath, "ROCm-OpenCL-Runtime/") + # mgbench + self.mgbenchrootpath = os.path.join(self.cwdAbs, "src/hiptestsuite/applications/mgbench/") + self.mgbenchapppath = os.path.join(self.mgbenchrootpath, "mgbench/") + # CUDA-grep + self.cudagreprootpath = os.path.join(self.cwdAbs, "src/hiptestsuite/applications/cuda_grep/") + self.cudagrepapppath = os.path.join(self.cudagreprootpath, "CUDA-grep/") + # cuda_memtest + self.cudamemrootpath = os.path.join(self.cwdAbs, "src/hiptestsuite/applications/cuda_memtest/") + self.cudamemapppath = os.path.join(self.cudamemrootpath, "cuda_memtest/") + # Quicksilver + self.qsrootpath = os.path.join(self.cwdAbs, "src/hiptestsuite/applications/hpc_apps/quicksilver/") + self.qsapppath = os.path.join(self.qsrootpath, "Quicksilver/") + # Gridtools + self.gridtoolsrootpath = os.path.join(self.cwdAbs, "src/hiptestsuite/applications/hpc_apps/gridtools/GridTools/") + self.gridtoolsapppath = os.path.join(self.gridtoolsrootpath, "gridtools/") + self.gtbenchrootpath = os.path.join(self.cwdAbs, "src/hiptestsuite/applications/hpc_apps/gridtools/GridTools/") + self.gtbenchapppath = os.path.join(self.gtbenchrootpath, "gtbench/") + # Kokkos + self.kokkosrootpath = os.path.join(self.cwdAbs, "src/hiptestsuite/applications/hpc_apps/kokkos/") + self.kokkosapppath = os.path.join(self.kokkosrootpath, "kokkos/") + # Laghos + self.mfemrootpath = os.path.join(self.cwdAbs, "src/hiptestsuite/applications/hpc_apps/laghos/") + self.mfemapppath = os.path.join(self.mfemrootpath, "mfem/") + self.laghosrootpath = os.path.join(self.cwdAbs, "src/hiptestsuite/applications/hpc_apps/laghos/") + self.laghosapppath = os.path.join(self.laghosrootpath, "Laghos/") + # CP2K + self.cp2krootpath = os.path.join(self.cwdAbs, "src/hiptestsuite/applications/hpc_apps/cp2k/") + self.cp2kapppath = os.path.join(self.cp2krootpath, "cp2k/") + + def pull_repo(self, logFile, repo, branch, commitId, reponame): + repo_root_path = "" + repo_location = "" + repo_dir = "" + if reponame == "gpu-stream": + repo_root_path = self.gpustrmpath + repo_location = self.appPath + repo_dir = "GPU-STREAM" + elif reponame == "mixbench": + repo_root_path = self.mixbenchpath + repo_location = self.appPath + repo_dir = "mixbench" + elif reponame == "hip_examples": + repo_root_path = self.examplepath + repo_location = self.appPath + repo_dir = "HIP-Examples" + elif reponame == "HIP": + repo_root_path = self.hippath + repo_location = self.conformancePath + repo_dir = "HIP" + elif reponame == "hipamd": + repo_root_path = self.hipamdpath + repo_location = self.conformancePath + repo_dir = "HIPAMD" + elif reponame == "rocclr": + repo_root_path = self.rocclrpath + repo_location = self.conformancePath + repo_dir = "ROCclr" + elif reponame == "opencl": + repo_root_path = self.openclpath + repo_location = self.conformancePath + repo_dir = "ROCm-OpenCL-Runtime" + elif reponame == "mgbench": + repo_root_path = self.mgbenchapppath + repo_location = self.mgbenchrootpath + repo_dir = "mgbench" + elif reponame == "cudagrep": + repo_root_path = self.cudagrepapppath + repo_location = self.cudagreprootpath + repo_dir = "CUDA-grep" + elif reponame == "cudamemtest": + repo_root_path = self.cudamemapppath + repo_location = self.cudamemrootpath + repo_dir = "cuda_memtest" + elif reponame == "quicksilver": + repo_root_path = self.qsapppath + repo_location = self.qsrootpath + repo_dir = "Quicksilver" + elif reponame == "gridtools": + repo_root_path = self.gridtoolsapppath + repo_location = self.gridtoolsrootpath + repo_dir = "gridtools" + elif reponame == "gtbench": + repo_root_path = self.gtbenchapppath + repo_location = self.gtbenchrootpath + repo_dir = "gtbench" + elif reponame == "kokkos": + repo_root_path = self.kokkosapppath + repo_location = self.kokkosrootpath + repo_dir = "kokkos" + elif reponame == "mfem": + repo_root_path = self.mfemapppath + repo_location = self.mfemrootpath + repo_dir = "mfem" + elif reponame == "Laghos": + repo_root_path = self.laghosapppath + repo_location = self.laghosrootpath + repo_dir = "Laghos" + elif reponame == "cp2k": + repo_root_path = self.cp2kapppath + repo_location = self.cp2krootpath + repo_dir = reponame + + if os.path.isdir(repo_root_path) and os.path.isdir(repo_root_path + "/.git"): + print(reponame + " already exist") + # Check if branch and commitId of local repo matches with input branch and commitId + # if not then update local repo + cmd = "cd " + os.path.join(repo_location, repo_dir) + ";" + cmd += "git branch" + currentbranch = execshellcmd(cmd, logFile, None) + currentbranch = currentbranch.replace("* ", "") + cmd = "cd " + os.path.join(repo_location, repo_dir) + ";" + cmd += "git rev-parse HEAD" + currentcommitid = execshellcmd(cmd, logFile, None) + # Check if the local repo is upto date with input configuration + if ((branch == "") or (branch in currentbranch)) and\ + ((commitId == "") or (commitId in currentcommitid)): + print("This repo is up to date with config") + return True + else: + print(reponame + " does not exist") + + # Update the repo + print("Updating: " + reponame) + cmdcd = "cd " + repo_location + ";" + cmdcd += "rm -Rf " + repo_dir + "/;" + cmdPull = "" + print("Cloning " + reponame + " to " + repo_location) + if branch == "": #No branch name provided. Clone from main branch. + cmdPull = "git clone " + repo + else: + print("From branch: " + branch) + cmdPull = "git clone -b " + branch + " " + repo + cmdexc = cmdcd + cmdPull + # Clone the latest version from repo + execshellcmd(cmdexc, logFile, None) + isRepoPresent = os.path.isdir(repo_root_path) + if not isRepoPresent: + return False + else: + print(reponame + " cloned successfully") + # At this point check ensure git head is pointing to commitId + if commitId != "": + print("Resetting to commit: " + commitId) + cmdcd = "cd " + repo_root_path + ";" + cmdexc = cmdcd + "git reset --hard " + commitId + ";" + execshellcmd(cmdexc, logFile, None) + print("Updating: " + reponame + " completed") + return True diff --git a/src/hiptestsuite/common/hip_shell.py b/src/hiptestsuite/common/hip_shell.py new file mode 100644 index 0000000..a106322 --- /dev/null +++ b/src/hiptestsuite/common/hip_shell.py @@ -0,0 +1,55 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import subprocess +import re + +def execshellcmd(cmdexc, logfile, myenv): + proc = subprocess.Popen(cmdexc, shell=True, env=myenv, + stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, + bufsize=0) + proc.wait() + stdoutstr = proc.stdout.read().decode('utf-8', errors='ignore') + if logfile != None: + logfile.write(stdoutstr) + return stdoutstr + +def execshellcmd_largedump(cmdexc, logfile, runlog, myenv): + runlog.seek(0) + proc = subprocess.Popen(cmdexc, shell=True, env=myenv, + stdin=subprocess.PIPE, stdout=runlog, stderr=subprocess.STDOUT, + bufsize=0, universal_newlines=False) + proc.wait() + runlog.seek(0) + if logfile != None: + for line in runlog: + logfile.write(line) + runlog.seek(0) + return proc.returncode + +def get_gpuarch(logFile): + # Get GPU Architecture + cmdexc = "/opt/rocm/bin/mygpu" + gpuarchsh = execshellcmd(cmdexc, logFile, None) + if not re.match("gfx\d+", gpuarchsh): + print("GPU Architecture unknown") + return None + gpuarch = gpuarchsh.strip() + return gpuarch diff --git a/src/hiptestsuite/config_processor.py b/src/hiptestsuite/config_processor.py new file mode 100644 index 0000000..46a22de --- /dev/null +++ b/src/hiptestsuite/config_processor.py @@ -0,0 +1,30 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.AMD import AMDObject + + +class ConfigProcessor(AMDObject): + def __init__(self): + AMDObject.__init__(self) + self.config = None + + def loadConfig(self): + pass diff --git a/src/hiptestsuite/conformance/__init__.py b/src/hiptestsuite/conformance/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/hiptestsuite/conformance/conformance_test_classifier.py b/src/hiptestsuite/conformance/conformance_test_classifier.py new file mode 100644 index 0000000..8b805e4 --- /dev/null +++ b/src/hiptestsuite/conformance/conformance_test_classifier.py @@ -0,0 +1,31 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.test_classifier import TestClassifier + +from typing import Union + + +class CONFORMANCE(TestClassifier): + def __init__(self): + TestClassifier.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + TestClassifier.add_matched_with_names(self, {"conformance": matched_with_names}) diff --git a/src/hiptestsuite/conformance/hip_dtest.py b/src/hiptestsuite/conformance/hip_dtest.py new file mode 100644 index 0000000..9b585b1 --- /dev/null +++ b/src/hiptestsuite/conformance/hip_dtest.py @@ -0,0 +1,172 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.TesterRepository import Tester +from hiptestsuite.Test import HIPTestData, TestResult, Test, HIPBuildData, HIP_PLATFORM +from typing import Union, List +from hiptestsuite.conformance.conformance_test_classifier import CONFORMANCE +from hiptestsuite.test_classifier import TestClassifier +from hiptestsuite.conformance.hip_dtest_build_amd import BuildRunAmd +from hiptestsuite.conformance.hip_dtest_build_nvidia import BuildRunNvidia +from hiptestsuite.common.hip_get_packages import HipPackages +from hiptestsuite.common.hip_shell import * + +import os + +# Common class to clone, set up, build and run test +class PrepareTest(): + def __init__(self): + self.hiprepo = "" # Default + self.hipbranch = "" + self.hipbranch = "" + self.hipcommitId = "" + self.logfd = None + self.downloadresult = True + self.buildresult = True + self.buildobj = None + + # Get the GIT repo information from configuration input + def setrepoinfo(self, test_data: HIPBuildData): + if test_data.repos["hip"].repo_url != None: + self.hiprepo = test_data.repos["hip"].repo_url + else: + return False + if test_data.repos["hip"].branch != None: + self.hipbranch = test_data.repos["hip"].branch + if test_data.repos["hip"].commit_id != None: + self.hipcommitId = test_data.repos["hip"].commit_id + return True + + # Download the relevant packages from GIT for this test + def downloadTest(self, log, platform): + ret = HipPackages().pull_repo(log, self.hiprepo, self.hipbranch,\ + self.hipcommitId, "HIP") + return ret + + # Build Packages + def build_package(self, logFile, platform): + status = True + if platform == HIP_PLATFORM.nvidia: + self.buildobj = BuildRunNvidia(logFile) + elif platform == HIP_PLATFORM.amd: + self.buildobj = BuildRunAmd(logFile) + else: + print("Invalid Platform") + return False + + return self.buildobj.build_package() + + # Run test + def runtest(self, logFile, verbosity, testcase): + status = "Failed" + if self.buildobj != None: + status = self.buildobj.runtest(logFile, verbosity, testcase) + return status + + # Get ctest info + def get_ctest_list(self): + ret = None + if self.buildobj != None: + ret = self.buildobj.get_all_ctest() + return ret + + +# Test HIP Dtest/ +class Hipconformance(Tester, PrepareTest): + def __init__(self): + Tester.__init__(self) + PrepareTest.__init__(self) + + def getTests(self, get_tests_data: HIPBuildData) -> List[Test]: + # Set repo info + testlist = [] + if get_tests_data.quick: + return testlist + self.logfd = open(get_tests_data.log_location + "/Hipconformance.log", 'w+') + status = self.setrepoinfo(get_tests_data) + if not status: + self.downloadresult = False + return testlist + + # Download repos + self.downloadresult = self.downloadTest(self.logfd, get_tests_data.HIP_PLATFORM) + if not self.downloadresult: + cmd = "echo \"Rocm packages download failed!\";" + execshellcmd(cmd, self.logfd, None) + return testlist + + print("Building HIP package ...") + # Build Rocclr and Hip + self.buildresult = self.build_package(self.logfd, get_tests_data.HIP_PLATFORM) + if not self.buildresult: + cmd = "echo \"Rocm packages build failed!!\";" + execshellcmd(cmd, self.logfd, None) + print("Rocm packages build failed!") + return testlist + + print("Building HIP package completed") + # Fetch all the ctests + dtestnamelist = self.get_ctest_list() + + for testname in dtestnamelist: + test = Test() + test.test_name = testname + category = CONFORMANCE() + category.add_matched_with_names() + test.classifiers = [category] + test.tester = self + testlist.append(test) + return testlist + + def get_test_classifiers(self) -> Union[None, List[TestClassifier]]: + category = CONFORMANCE() + category.add_matched_with_names() + return [category] + + def test(self, test_data: HIPTestData): + if not self.downloadresult: + test_data.test_result = TestResult.ERROR + cmd = "echo \"HIP Catch2 Build FAILED!\";" + execshellcmd(cmd, self.logfd, None) + return + + if not self.buildresult: + test_data.test_result = TestResult.ERROR + cmd = "echo \"HIP Catch2 Build FAILED!\";" + execshellcmd(cmd, self.logfd, None) + return + # Build test + print("Running test: " + test_data.test.test_name + "..........") + testcase = test_data.test.test_name + status = None + + with open(test_data.log_location + "/test.log", 'wb+') as testLogger: + status = self.runtest(testLogger, test_data.CONFORMANCE_VERBOSE, testcase) + + if status == "PASSED": + test_data.test_result = TestResult.PASS + elif status == "SKIP": + test_data.test_result = TestResult.SKIP + elif status == "FAILED": + test_data.test_result = TestResult.FAIL + + def clean(self): + if self.logfd != None: + self.logfd.close() diff --git a/src/hiptestsuite/conformance/hip_dtest_build_amd.py b/src/hiptestsuite/conformance/hip_dtest_build_amd.py new file mode 100644 index 0000000..680b7a6 --- /dev/null +++ b/src/hiptestsuite/conformance/hip_dtest_build_amd.py @@ -0,0 +1,68 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os, glob +import tempfile +import json +import re +from hiptestsuite.common.hip_shell import execshellcmd_largedump, execshellcmd +from hiptestsuite.conformance.hip_dtest_build_common import BuildRunCommon + +class BuildRunAmd(BuildRunCommon): + ''' + In this class insert the build and execution steps specific + for testsuite platform. + ''' + def __init__(self, logfile): + BuildRunCommon.__init__(self, logfile) + self.envtoset = os.environ.copy() + + # Build HIP Catch2 for AMD platform + def build_package(self): + buildSuccess = True + if not os.path.exists("/opt/rocm"): + print("ROCm not installed. Exiting!") + return False + isCatchBuilt = self.validate_hipcatch_build() + if not isCatchBuilt: + # Build HIP + print("Catch2 test not built. Building Catch2 ..") + cmd = "cd " + self.hippath + ";" + cmd += "mkdir build; cd build;" + cmd += "cmake -DHIP_PATH=/opt/rocm/hip -DHIP_PLATFORM=amd ../tests/catch;" + cmd += "make -j build_tests;" + cmdexc = cmd + runlogdump = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logfile, runlogdump, self.envtoset) + runlogdump.close() + + # Validate if HIP build is successful + isCatchBuilt = self.validate_hipcatch_build() + if not isCatchBuilt: + print("HIP Catch2 Build Failed!") + return False + else: + print("HIP Catch2 already Built") + return True + + # Execute test cases + def runtest(self, log, verbosity, testcase): + return BuildRunCommon.runtest(self, log, verbosity, testcase, self.envtoset) + diff --git a/src/hiptestsuite/conformance/hip_dtest_build_common.py b/src/hiptestsuite/conformance/hip_dtest_build_common.py new file mode 100644 index 0000000..94a1af8 --- /dev/null +++ b/src/hiptestsuite/conformance/hip_dtest_build_common.py @@ -0,0 +1,97 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os +import tempfile +import re +from hiptestsuite.common.hip_shell import execshellcmd_largedump, execshellcmd + +class BuildRunCommon(): + ''' + In this class insert the build and execution steps for test cases + which are identical across different platforms (amd/nvidia/intel). + ''' + def __init__(self, logfile): + self.logfile = logfile + self.hippath = os.path.join(os.getcwd(), "src/hiptestsuite/conformance/HIP") + self.ctest_file = os.path.join(self.hippath, "build/ctest.txt") + self.builddir = os.path.join(self.hippath, "build") + self.expected_catch_binaries = ["ABMTests","MultiProcTests","UnitTests"] + + # Validate if HIP build is successful + def validate_hipcatch_build(self): + status = True + catch_binary_path = os.path.join(self.hippath, "build/hipTestMain") + for catchbin in self.expected_catch_binaries: + if not os.path.isfile(\ + os.path.join(catch_binary_path, catchbin)): + print("Did not find catch2 binary " + catchbin) + status &= False + return status + + # Fetches all available dtests using ctest + def get_all_ctest(self): + if not os.path.isfile(self.ctest_file): + cmdexc = "cd " + self.builddir + ";" + cmdexc += "ctest -N;" + with open(self.ctest_file, "w+") as ctestlog: + execshellcmd_largedump(cmdexc, self.logfile, ctestlog, None) + + testlist = [] + with open(self.ctest_file, "r") as ctestlog: + for test in ctestlog: + if re.search("Test *#\d*:", test) != None: + dtest = re.sub("Test *#\d*: ", "", test) + dtest = re.sub("/", ".", dtest) + dtest = dtest.lstrip() + dtest = dtest.rstrip() + testlist.append(dtest) + return testlist + + # Parse the test result + def parsetest(self, log): + log.seek(0) + logbytes = log.read() + logutf = logbytes.decode('utf-8', errors='ignore') + status = None + if re.search("100% tests passed", logutf) != None: + status = "PASSED" + else: + status = "FAILED" + return status + + # Execute the test case + def runtest(self, log, verbosity, testcase, envtoset): + if verbosity == 0: + cmdtest = "ctest -R " + "\"" + testcase + "\"" + elif verbosity == 1: + cmdtest = "ctest -R " + "\"" + testcase + "\"" + " --verbose" + else: + cmdtest = "ctest -R " + "\"" + testcase + "\"" + + print("Executing command = " + cmdtest) + # run test + cmd = "cd " + self.builddir + ";" + cmd += cmdtest + ";" + runlogdump = tempfile.TemporaryFile("wb+") + execshellcmd_largedump(cmd, log, runlogdump, envtoset) + status = self.parsetest(runlogdump) + runlogdump.close() + return status diff --git a/src/hiptestsuite/conformance/hip_dtest_build_nvidia.py b/src/hiptestsuite/conformance/hip_dtest_build_nvidia.py new file mode 100644 index 0000000..3b4711f --- /dev/null +++ b/src/hiptestsuite/conformance/hip_dtest_build_nvidia.py @@ -0,0 +1,74 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import os, glob +import tempfile +import json +import re +from hiptestsuite.common.hip_shell import execshellcmd_largedump, execshellcmd +from hiptestsuite.conformance.hip_dtest_build_common import BuildRunCommon + +class BuildRunNvidia(BuildRunCommon): + ''' + In this class insert the build and execution steps specific + for NVIDIA platform. + ''' + def __init__(self, logfile): + BuildRunCommon.__init__(self, logfile) + self.envtoset = os.environ.copy() + + def setenv(self): + env = "export HIP_PLATFORM=nvidia;" + env += "export HIP_COMPILER=nvcc;" + env += "export HIP_RUNTIME=cuda;" + return env + + # Build HIP Catch2 for NVIDIA platform + def build_package(self): + buildSuccess = True + if not os.path.exists("/opt/rocm"): + print("ROCm not installed. Exiting!") + return False + isCatchBuilt = self.validate_hipcatch_build() + if not isCatchBuilt: + # Build HIP + print("Catch2 test not built. Building Catch2 ..") + cmd = self.setenv() + cmd += "cd " + self.hippath + ";" + cmd += "mkdir build; cd build;" + cmd += "cmake -DHIP_COMPILER=nvcc -DHIP_PLATFORM=nvidia -DHIP_RUNTIME=cuda -DHIP_PATH=/opt/rocm/hip ../tests/catch;" + cmd += "make -j build_tests;" + cmdexc = cmd + runlogdump = tempfile.TemporaryFile("w+") + execshellcmd_largedump(cmdexc, self.logfile, runlogdump, self.envtoset) + runlogdump.close() + + # Validate if HIP build is successful + isCatchBuilt = self.validate_hipcatch_build() + if not isCatchBuilt: + print("HIP Catch2 Build Failed!") + return False + else: + print("HIP Catch2 already Built") + return True + + # Execute test cases + def runtest(self, log, verbosity, testcase): + return BuildRunCommon.runtest(self, log, verbosity, testcase, self.envtoset) diff --git a/src/hiptestsuite/list_tests.py b/src/hiptestsuite/list_tests.py new file mode 100644 index 0000000..0c09f15 --- /dev/null +++ b/src/hiptestsuite/list_tests.py @@ -0,0 +1,144 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import tempfile +import typing +from typing import Union, List + +from hiptestsuite.TesterRepository import TesterRepository, GetTests +from hiptestsuite.Test import Test, Quick +from hiptestsuite.test_classifier import TestClassifier + + +def list_tests(quick: bool, cfg, tester_repository=None): + if not quick: + print("Generating tests, please wait...") + print(); print(); print() + pretty_table_installed = False + try: + from prettytable import PrettyTable + pretty_table_installed = True + except Exception as err: + print("For better UI experience, please pip3 install -r requirements.txt") + + if tester_repository is None: + tester_repository: TesterRepository = TesterRepository() + tester_repository.addAllTesters() + + get_tests = GetTests(tester_repository=tester_repository) + get_tests.config = cfg + get_tests.loadConfig() + + with tempfile.TemporaryDirectory() as tmpdirname: + tests = get_tests.get_tests(log_location=tmpdirname, quick=quick) + + field_names = ["Classifiers", "Test"] + if pretty_table_installed: + test_info_table = PrettyTable() + test_info_table.field_names = field_names + else: + print(" | ".join(field_names)) + + for test in tests: + if not pretty_table_installed: + classifiers_s = get_classifiers_s(test) + print(classifiers_s, end=" | ") + print(test.test_name) + else: + classifiers_s = get_classifiers_s(test) + test_info_table.add_row([classifiers_s, test.test_name]) + + if quick: + quick_getTests_testers = list() + all_testers = tester_repository.getTesters() + + for tester in all_testers: + get_tests_t = typing.get_type_hints(tester.getTests) + get_tests_data_t = None + if get_tests_t: + if "get_tests_data" in get_tests_t: + get_tests_data_t = get_tests_t["get_tests_data"] + + if get_tests_data_t and issubclass(get_tests_data_t, Quick): + quick_getTests_testers.append(tester) + + for tester in quick_getTests_testers: + if not pretty_table_installed: + classifiers_s = get_classifiers_s_from_classifier(tester.get_test_classifiers()) + print(classifiers_s, end=" | ") + print("*") + else: + classifiers_s = get_classifiers_s_from_classifier(tester.get_test_classifiers()) + test_info_table.add_row([classifiers_s, "*"]) + + if pretty_table_installed: + print(test_info_table.get_string(title="Test Information")) + + if quick: + print("Note: Run python3 run.py -lst for expanding * tests") + print(""" + 1. If you want to run all tests for amd platform, python3 run.py + If you want to select specific test(s) e.g. for + ----------------------- + | Classifiers | Test | + |---------------------| + | c1:c2:c3 | test1 | + | c4 | test2 | + | c1:c2 | test3 | + | c1:c2:c3 | test4 | + ----------------------- + c1:c2:c3 means c3 is subset of c2 is subset of c1 + e.g. selecting c2 automatically executes tests under c2 and c3 + 2. test1 => python3 run.py -t test1 + 3. tests with python regex test.* => -t test.* + 4. all c3 => -t c3 (i.e. test1, test4) + 5. all c1 => -t c1 (i.e. test1, test3, test4) + 6. all c1:c2 => -t c1:c2 (i.e. test1, test3, test4) + 7. all c1:c2:c3 => -t c1:c2:c3 (i.e. test1, test4) + 8. test2 and test3 => -t test2 test3 + 9. c4 and test4 => -t c4 test4 (i.e. test2, test4) + """) + + +def get_classifiers_s_from_classifier(classifiers: List[TestClassifier]): + classifiers_s = '' + if classifiers: + for ix, classifier in enumerate(classifiers): + matched_with_names = classifier.matched_with_names + classifiers_s += ':'.join(get_one_sequence(matched_with_names)) + if ix != len(classifiers) - 1: + classifiers_s += ', ' + return classifiers_s + + +def get_classifiers_s(test: Test): + return get_classifiers_s_from_classifier(test.classifiers) + + +def get_one_sequence(d: dict): + k = list(d.keys())[0] + v = list(d.values())[0] + + if v is None: + return [k] + else: + m = get_one_sequence(v) + m.insert(0, k) + return m diff --git a/src/hiptestsuite/match_fun_args_call.py b/src/hiptestsuite/match_fun_args_call.py new file mode 100644 index 0000000..ac8db70 --- /dev/null +++ b/src/hiptestsuite/match_fun_args_call.py @@ -0,0 +1,32 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +import typing +from typing import Callable + + +def match_fun_args_call(fun: Callable, args: dict): + fun_args = typing.get_type_hints(fun) + call_args = dict() + for fun_arg_name, fun_arg_t in fun_args.items(): + if fun_arg_name != 'return': + call_args[fun_arg_name] = args[fun_arg_name] + + return fun(**call_args) diff --git a/src/hiptestsuite/stress/__init__.py b/src/hiptestsuite/stress/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/hiptestsuite/targets.py b/src/hiptestsuite/targets.py new file mode 100644 index 0000000..16fc502 --- /dev/null +++ b/src/hiptestsuite/targets.py @@ -0,0 +1,164 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.AMD import AMDObject + + +class Target: + def __init__(self): + pass + + +class AMDTarget(AMDObject, Target): + def __init__(self): + AMDObject.__init__(self) + Target.__init__(self) + + +class IntelTarget(AMDObject, Target): + def __init__(self): + AMDObject.__init__(self) + Target.__init__(self) + + +class NvidiaTarget(AMDObject, Target): + def __init__(self): + AMDObject.__init__(self) + Target.__init__(self) + + +class gfx700(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx701(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx702(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx703(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx704(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx705(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx801(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx802(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx803(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx805(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx810(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx900(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx902(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx904(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx906(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx908(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx909(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx90a(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx1010(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx1011(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx1012(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx1030(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx1031(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) + + +class gfx1032(AMDTarget): + def __init__(self): + AMDTarget.__init__(self) diff --git a/src/hiptestsuite/test_classifier.py b/src/hiptestsuite/test_classifier.py new file mode 100644 index 0000000..4998766 --- /dev/null +++ b/src/hiptestsuite/test_classifier.py @@ -0,0 +1,34 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.AMD import AMDObject + +from typing import Union + + +class TestClassifier(AMDObject): + def __init__(self): + self.matched_with_names = dict() + AMDObject.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + if matched_with_names is None: + matched_with_names = dict() + self.matched_with_names.update(matched_with_names) diff --git a/src/hiptestsuite/test_selector.py b/src/hiptestsuite/test_selector.py new file mode 100644 index 0000000..cc7b61d --- /dev/null +++ b/src/hiptestsuite/test_selector.py @@ -0,0 +1,204 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.AMD import AMDObject +from hiptestsuite.TesterRepository import TesterRepository, Test, GetTests +from typing import Union, List, Dict + +import re, sys, os + + +class TestSelector(GetTests): + def __init__(self, tester_repository: TesterRepository): + AMDObject.__init__(self) + GetTests.__init__(self, tester_repository=tester_repository) + + def to_select_this_test(self, test: Test, test_name_regexes, exclude_module_paths=None): + test_name = test.test_name + also_matched_with_test_names = test.also_matched_with_test_names + applicable_for_target = test.applicable_for_target + classifiers = test.classifiers + + select_this_test = False + class_abs_path = os.path.abspath(sys.modules[test.tester.__class__.__module__].__file__) + + if test_name_regexes is None: + select_this_test = True + for exclude_module_path in exclude_module_paths: + if exclude_module_path in class_abs_path: + print("Warning: Test " + test_name + " is excluded, please run selectively") + select_this_test = False + break + else: + for test_name_regex in test_name_regexes: + test_name_regex: Union[str, List[str]] = test_name_regex + + if type(test_name_regex) == str: + asked_classifiers: List[str] = test_name_regex.split(':') + else: # type(test_name_regex) == list: + asked_classifiers: List[str] = test_name_regex + + in_dicts: List[Dict] = list() + ret_is_sequence_in_dicts_obj = ret_is_sequence_in_dicts() + if classifiers: + for classifier in classifiers: + in_dicts.append(classifier.matched_with_names) + is_sequence_in_dicts(sequence=asked_classifiers, in_dicts=in_dicts, + ret_is_sequence_in_dicts_obj=ret_is_sequence_in_dicts_obj) + + if not classifiers or ret_is_sequence_in_dicts_obj.initial_matching: + if classifiers and ret_is_sequence_in_dicts_obj.last_also_matching: + select_this_test = True + else: + if re.findall(asked_classifiers[-1].lower(), test_name.lower()): + select_this_test = True + if also_matched_with_test_names: + for also_matched_with_test_name in also_matched_with_test_names: + if re.findall(asked_classifiers[-1].lower(), also_matched_with_test_name.lower()): + select_this_test = True + return select_this_test + + def check_quicktestlist(self, test_to_find, quicktestlist): + testfound = False + for test in quicktestlist: + if re.search(test_to_find, test.test_name): + testfound = True + return testfound + + def check_classifierlist(self, test_to_find, classifierlist): + testfound = False + for thisclassifier in classifierlist: + if re.search(test_to_find, thisclassifier): + testfound = True + return testfound + + def check_quicktestlist_sufficient(self, usertestlist, quicktestlist, classifierlist): + testcount = 0 + for test in usertestlist: + if self.check_classifierlist(test, classifierlist): + # We know this test is non conformance + testcount = testcount + 1 + elif self.check_quicktestlist(test, quicktestlist): + testcount = testcount + 1 + isSufficient = True + if len(usertestlist) != testcount: + # quicktestlist is not sufficient + isSufficient = False + return isSufficient + + def get_all_classifierkeys(self, testclassifierdict, classifierlist): + for key in testclassifierdict: + classifierlist.append(key) + if testclassifierdict[key] != None: + if isinstance(testclassifierdict[key], dict): + self.get_all_classifierkeys(testclassifierdict[key], classifierlist) + + def get_all_classifiers(self, testclassifier, classifierlist): + for myclassifier in testclassifier: + self.get_all_classifierkeys(myclassifier.matched_with_names, classifierlist) + + def select_tests(self, log_location: str, exclude_module_paths) -> List[Test]: + config = self.config + tests = list() + run_tests = config.run_tests + if type(run_tests) == str: + test_name_regexes = [run_tests] + elif type(run_tests) == list: + test_name_regexes = run_tests + else: + test_name_regexes = None + + if test_name_regexes is None: + for test_of_tester in self.get_tests(log_location=log_location, quick=False): + select_this_test = self.to_select_this_test(test=test_of_tester, test_name_regexes=test_name_regexes, exclude_module_paths=exclude_module_paths) + if select_this_test: + tests.append(test_of_tester) + else: + for test_of_tester in self.get_tests(log_location=log_location, quick=True): + select_this_test = self.to_select_this_test(test=test_of_tester, test_name_regexes=test_name_regexes) + if select_this_test: + tests.append(test_of_tester) + + classifierlist = list() + for thistest in tests: + tmpclassifierlist = list() + self.get_all_classifiers(thistest.classifiers, tmpclassifierlist) + thistestclassification = "" + for elem in tmpclassifierlist: + thistestclassification = thistestclassification + elem + ":" + classifierlist.append(thistestclassification) + # Check at this point if quick search is sufficient + if False == self.check_quicktestlist_sufficient(test_name_regexes, tests, classifierlist): + tests.clear() + if not tests: + for test_of_tester in self.get_tests(log_location=log_location, quick=False): + select_this_test = self.to_select_this_test(test=test_of_tester, test_name_regexes=test_name_regexes) + if select_this_test: + tests.append(test_of_tester) + + return tests + + +class ret_is_sequence_in_dicts: + def __init__(self): + self.initial_matching = False + self.last_also_matching = False + + +def next_item_process(item, sequence: List, till_matched: List): + if till_matched == sequence: + return + if sequence[len(till_matched)] == item: + till_matched.append(item) + else: + till_matched.clear() + + +def is_sequence_in_dict(sequence: List, in_dict: Dict, ret_is_sequence_in_dict_obj: ret_is_sequence_in_dicts, till_matched: List): + for k, v in in_dict.items(): + next_item_process(item=k, sequence=sequence, till_matched=till_matched) + if sequence[:-1] == till_matched: + ret_is_sequence_in_dict_obj.initial_matching = True + if sequence == till_matched: + ret_is_sequence_in_dict_obj.last_also_matching = True + ret_is_sequence_in_dict_obj.initial_matching = True + else: + if type(v) == dict: + is_sequence_in_dict(sequence=sequence, in_dict=v, ret_is_sequence_in_dict_obj=ret_is_sequence_in_dict_obj, till_matched=till_matched) + + +def is_sequence_in_dicts(sequence: List, in_dicts: List[Dict], ret_is_sequence_in_dicts_obj: ret_is_sequence_in_dicts): + initial_matching = False + last_also_matching = False + for in_dict in in_dicts: + ret_is_sequence_in_dict_obj = ret_is_sequence_in_dicts() + is_sequence_in_dict(sequence=sequence, in_dict=in_dict, ret_is_sequence_in_dict_obj=ret_is_sequence_in_dict_obj, till_matched=list()) + seq_initial_matching = ret_is_sequence_in_dict_obj.initial_matching + seq_last_also_matching = ret_is_sequence_in_dict_obj.last_also_matching + + if not initial_matching: + initial_matching = seq_initial_matching + if not last_also_matching: + last_also_matching = seq_last_also_matching + if seq_initial_matching and seq_last_also_matching: + break + + ret_is_sequence_in_dicts_obj.initial_matching = initial_matching + ret_is_sequence_in_dicts_obj.last_also_matching = last_also_matching diff --git a/src/hiptestsuite/thirdparty/__init__.py b/src/hiptestsuite/thirdparty/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/hiptestsuite/thirdparty/sampleapp/__init__.py b/src/hiptestsuite/thirdparty/sampleapp/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/hiptestsuite/thirdparty/sampleapp/sampleapp_thirdparty.py b/src/hiptestsuite/thirdparty/sampleapp/sampleapp_thirdparty.py new file mode 100644 index 0000000..63ad77a --- /dev/null +++ b/src/hiptestsuite/thirdparty/sampleapp/sampleapp_thirdparty.py @@ -0,0 +1,32 @@ +from hiptestsuite.TesterRepository import Tester, Test, TestData +from hiptestsuite.Test import HIPTestData, TestResult, HIP_PLATFORM +from typing import Union, List +from hiptestsuite.thirdparty.thirdparty_classifier import THIRDPARTY_CLASSIFIER + +class SAMPLEAPP(THIRDPARTY_CLASSIFIER): + def __init__(self): + THIRDPARTY_CLASSIFIER.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + THIRDPARTY_CLASSIFIER.add_matched_with_names(self, {"sampleapp": matched_with_names}) + +# Sample App for 3rd party +class sampleapp_test(Tester): + def __init__(self): + Tester.__init__(self) + + def getTests(self) -> List[Test]: + test = Test() + test.test_name = self.__class__.__name__ + classifier = SAMPLEAPP() + classifier.add_matched_with_names() + test.classifiers = [classifier] + test.tester = self + return [test] + + def clean(self): + pass + + def test(self, test_data: HIPTestData): + print("=============== Running Sample Third Party App ===============") + test_data.test_result = TestResult.PASS diff --git a/src/hiptestsuite/thirdparty/thirdparty_classifier.py b/src/hiptestsuite/thirdparty/thirdparty_classifier.py new file mode 100644 index 0000000..e79c6e0 --- /dev/null +++ b/src/hiptestsuite/thirdparty/thirdparty_classifier.py @@ -0,0 +1,31 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +from hiptestsuite.test_classifier import TestClassifier + +from typing import Union + + +class THIRDPARTY_CLASSIFIER(TestClassifier): + def __init__(self): + TestClassifier.__init__(self) + + def add_matched_with_names(self, matched_with_names: Union[None, dict] = None): + TestClassifier.add_matched_with_names(self, {"thirdparty": matched_with_names}) diff --git a/src/hiptestsuite/version.py b/src/hiptestsuite/version.py new file mode 100644 index 0000000..f3227cc --- /dev/null +++ b/src/hiptestsuite/version.py @@ -0,0 +1,23 @@ +# Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in +# all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +# THE SOFTWARE. + +MAJOR = 1 +MINOR = 0 +PATCH = 0