Regression testing scripts

energyplus
testing
retag add tags

I need to run full regression test suite using compiled EnergyPlus builds. It seems EnergyPlusDevSupport repository had scripts (RunComparison.py etc) needed for running regression tests but that repository doesn't exist anymore? Or maybe its contents moved to some other repository. CMakeLists.txt has BUILDTESTING and ENABLEREGRESSION_TESTING set to ON but needs following details which I am struggling to provide.

set( REGRESSION_BASELINE_SHA "" CACHE STRING "SHA of baseline comparison" )``
set( COMMIT_SHA "" CACHE STRING "Commit sha of this build" )
set( REGRESSION_SCRIPT_PATH "" CACHE PATH "Path to regression scripts" )
set( REGRESSION_BASELINE_PATH "" CACHE PATH "Path to energyplus baseline folder to compare against" )

Any help is appreciated. Thanks.

2.4k

Chandan Sharma

asked 2015-04-14 21:35:05 -0500

4.4k

__AmirRoth__

updated 2015-07-11 17:26:15 -0500

edit flag offensive 0 remove flag close merge delete

Comments

add a comment see more comments

2 Answers

Sort by

oldest

newest

most voted

COMMIT_SHA and REGRESSION_BASELINE_SHA are mostly not necessary - you can set them to anything you want to, for the purposes of running the regression tests locally.

Set up your baseline
1. Compile the version of EnergyPlus that you want to build against (baseline)
2. Run the unit tests. This generates the necessary output
Clone the tools necessary for running the regression from https://github.com/NREL/EnergyPlusDev...
Set up the version of EnergyPlus you are testing
1. Be sure to configure in a directory separate from the one you build your baseline in
2. Set REGRESSION_SCRIPT_PATHto the folder you cloned https://github.com/NREL/EnergyPlusDev... into
3. Set REGRESSION_BASELINE_PATH to the build folder you compiled the 'baseline' in
4. Set ENABLE_REGRESSION_TESTING to TRUE
5. Compile as normal
Run regressions
1. At this point make test should run all tests including the regression comparisons
2. Or for parallelization ctest -j<numjobs>

lefticus

answered 2015-07-24 13:53:58 -0500

edit flag offensive 0 remove flag delete link

Comments

Thanks. The GitHub link of EnergyPlusDevSupport repository is saying 404 Page not found. Am I missing something here?

Chandan Sharma (2015-07-28 08:07:41 -0500) edit

Oh Snap! The repo requires authentication. @Edwin can we do anything about that?

Kyle Benne (2015-07-28 11:41:50 -0500) edit

That repo is where defect files are maintained which could contain proprietary info. We can't open it up. We can, however, move the test scripts. They could even go right in NREL/EnergyPlus as far as I am concerned. @lefticus what will this affect on the CI end of things?

Edwin (2015-07-28 12:00:48 -0500) edit

I did provide the scripts via private email, so no hurry on moving these items.

rraustad (2015-07-28 12:34:36 -0500) edit

Thank you all for the help.

Chandan Sharma (2015-08-24 20:28:07 -0500) edit

add a comment see more comments

In CMake there is a check box for ENABLE REGRESSION TESTING. Also there is still the local machine regression testing as we performed in the past and still perform to test for differences in the results (csv diff's).

13.8k

rraustad

answered 2015-07-24 12:08:35 -0500, updated 2015-07-24 12:09:55 -0500

edit flag offensive 0 remove flag delete link

Comments

Local machine regression scripts are what I was actually looking for at that time. Thought they were also on the StarTeam and probably moved to GitHub. Where can find them?

Chandan Sharma (2015-07-28 08:07:43 -0500) edit

These are now stored in a private archive repository.

rraustad (2015-07-28 09:49:53 -0500) edit

Thank you Rich.

Chandan Sharma (2015-07-28 10:18:17 -0500) edit

add a comment see more comments

Regression testing scripts edit

Comments

2 Answers

Comments

Comments

Regression testing scripts