.. _test_analysis_tools:

Analysis Tools for Testing
==========================

The `TestTools` class is a set of unit tests for validating the consistency of baseline results across different project directories. The tests ensure that the columns in the CSV files generated by the `buildstockbatch` and `run_analysis` processes match and that the values in these columns are comparable within an acceptable tolerance.

These tests are designed to ensure that the output of the baseline analysis process is consistent both in terms of the structure of the CSV files (columns) and the numerical results (values). By performing these checks, we can ensure that the analysis workflow is functioning correctly and that the data is reliable for further analysis.

The `test_baseline_columns` and `test_baseline_results` tests are located in the `TestTools` class. This class is run as part of the standard testing process for baseline result validation. It is designed to ensure that no discrepancies occur between the expected and actual output during the baseline analysis phase of the project.

The tests are designed to check two main aspects of the baseline analysis output:

1. baseline columns consistency,
2. baseline results consistency.

Each test is executed for two project directories, `national` and `testing`, and ensures that the results produced by both the `buildstockbatch` and `run_analysis` are consistent in terms of both the structure and numerical values.

**Dependencies**

- **Minitest** : This test framework is used to define and run the tests.

- **CSV** : The CSV library is used to read and manipulate the result files.

- **HPXMLtoOpenStudio** : The helper files for the test setup and any necessary utilities are included via the `minitest_helper` in the `HPXMLtoOpenStudio/resources` directory.

`test_baseline_columns` Function
--------------------------------

This test verifies that the CSV files generated by the `buildstockbatch` and `run_analysis` processes have the same column structure.

Test Description:

- **Input:** 

  - `buildstockbatch` results: Located at `buildstockbatch/project_<project>/project_baseline/results_csvs/results_up00.csv`
  - `run_analysis` results: Located at `run_analysis/project_<project>/project_baseline/results-Baseline.csv`
  
- **Process:**

  1. For each project (`national` and `testing`), the CSV files from both the `buildstockbatch` and `run_analysis` directories are read into memory.
  2. The headers (columns) of each CSV file are compared.
  3. Any extra columns in `buildstockbatch` that are not present in `run_analysis` (excluding specific columns such as `apply_upgrade` columns) are reported.
  4. Similarly, any extra columns in `run_analysis` that are not present in `buildstockbatch` are reported.
  5. An assertion is made to check that there are no additional columns in either of the files after excluding the specific columns related to `apply_upgrade`.
  
- **Expected Outcome:**

  - The test passes if the column headers in the two files match, except for the known differences related to `apply_upgrade` columns.

- **Assertion:** 

  - The test asserts that there should be no extra columns in either the `buildstockbatch` or `run_analysis` files, after excluding the `apply_upgrade` columns.

`test_baseline_results` Function
--------------------------------

This test verifies that the numerical results in the baseline CSV files from `buildstockbatch` and `run_analysis` are consistent, within a specified tolerance.

Test Description:

- **Input:**

  - `buildstockbatch` results: Located at `buildstockbatch/project_<project>/project_baseline/results_csvs/results_up00.csv`
  - `run_analysis` results: Located at `run_analysis/project_<project>/project_baseline/results-Baseline.csv`
  
- **Process:**

  1. For each project (`national` and `testing`), the specified column (`report_simulation_output.energy_use_total_m_btu`) from both the `buildstockbatch` and `run_analysis` CSV files is extracted.
  2. The numerical values in the column are summed for both files.
  3. An assertion is made to check that the number of rows in the specified column matches between the two files.
  4. The sums of the values are compared using `assert_in_epsilon`, which allows for small differences (up to 0.01).
  
- **Expected Outcome:**

  - The test passes if the sums of the values in the specified column are nearly identical (within 0.01 tolerance).
  
- **Assertion:**

  - The test asserts that the sums of the `report_simulation_output.energy_use_total_m_btu` values from both files are within a tolerance of 0.01, ensuring that there are no significant discrepancies between the results.