Skip to content

add script for batch processing #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kaiseu wants to merge 3 commits into
base: master
Choose a base branch
from
Open

add script for batch processing #16

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a44a65c
add script for batch processing
Mar 26, 2019
1a1b6ab
update generated file name
Jul 5, 2019
0b9d943
update batch process
Jul 5, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions PAT-post-processing/batch-process.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
## script for post processing PAT data in batch
## ${Base_dir} contains all the PAT source data that needs to be processed, change this dir to your own
Base_dir=/home/test1

################################################################################
## DO NOT NEED TO EDIT BELOW PARTS!!!
################################################################################
PAT_POST_HOME=$( cd $( dirname ${BASH_SOURCE[0]} ) && pwd )

cd ${Base_dir}
for item in `ls -d */` ## only dirs under ${Base_dir}
do
echo "Processing ${item} ..."
New_dir=${Base_dir}/${item}/instruments
if [ -d ${New_dir} ]; then
sed -i "s|<source>.*</source>|<source>${New_dir}</source>|g" ${PAT_POST_HOME}/config.xml
${PAT_POST_HOME}/pat-post-process.py ${PAT_POST_HOME}/config.xml
echo "${item} done!"
else
echo "WARNING: ${item} is not a PAT file, will ignore!"
fi
done
## change back to original value
sed -i "s|<source>.*</source>|<source>/foo/bar/instruments/</source>|g" ${PAT_POST_HOME}/config.xml
cd -



10 changes: 8 additions & 2 deletions PAT-post-processing/pat-post-process.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -272,12 +272,18 @@ def generate_output(cluster):
en_memory_csv = root[2].find('csv-memory').text
result_path = root[3].text
name_node = root[4].text

path = root[3].text
if path[-1] == "/":
pat_name = path.split("/")[-3]
else:
pat_name = path.split("/")[-2]

if en_pdf == 'yes':
print "----Rendering pdf", time.ctime(), "----"

# global pdf file that will contain all charts
pp = PdfPages(result_path + '/PAT-Result.pdf')
pp = PdfPages(result_path + '/' + pat_name + '.pdf')

# print average cpu utilization graph to pdf
if en_avg_cpu == 'yes' or en_avg_cpu == 'Yes':
Expand Down Expand Up @@ -362,7 +368,7 @@ def generate_output(cluster):

if en_xl == 'yes':
print "----Generating Excel", time.ctime(), "----"
wb = xlsxwriter.Workbook(result_path + '/PAT-Result.xlsm')
wb = xlsxwriter.Workbook(result_path + '/' + pat_name + '.xlsm')
print "----Generating CSV", time.ctime(), "----"
csv_path_cpu = result_path + "/CPU.csv"
csv_path_disk = result_path + "/DISK.csv"
Expand Down