Run Analysis Tests
==================

Overview
--------

The `test_run_analysis.rb` file provides a full integration test suite for the ResStock `workflow/run_analysis.rb` script. It is responsible for verifying the correct behavior of the ResStock analysis pipeline under a variety of configurations and inputs, ensuring that:

- Valid YAML configurations execute as expected and generate proper output artifacts.
- Common misconfigurations are caught and return helpful error messages.
- The measure execution order, file outputs, and CLI behavior conform to expectations.
- Corner cases such as invalid upgrade names, unsupported sampling modes, and incorrect weather file paths are handled gracefully.

The tests are implemented using Ruby's `Minitest::Test` framework and are located in the `resstock/test/` directory. This test file is a key component in maintaining the integrity of the analysis pipeline across code changes and configuration updates.

**Dependencies**

- `Minitest`
- `OpenStudio CLI`
- Local YAML configuration files
- `HPXMLtoOpenStudio`, `buildstock.rb`, and associated resources

Functions
---------

The class defines a number of helper and test methods that serve distinct purposes:

**Setup Method**

- ``before_setup``:
  Initializes variables such as CLI path, command string, and buildstock output paths. Also ensures output directories from prior runs are cleared.

**Helper Methods**

- ``_test_measure_order(osw)``:
  Verifies that the sequence of OpenStudio measures in an OSW file matches the expected order.
- ``_assert_and_puts(output, msg, expect = true)``:
  Asserts that expected messages appear (or do not appear) in stdout or logs, and prints output on failure.
- ``_verify_outputs(cli_output_log)``:
  Validates `cli_output.log` for known, expected warning messages and fails on any unexpected warnings.
- ``_expected_warning_message(message, txt)``:
  Returns true if a given message is an expected warning.

Test Scenarios
--------------

The file covers a wide range of test scenarios including error conditions, configuration variants, and output verification.

**Version Check**

- ``test_version``: Confirms that ResStock, OpenStudio, and HPXML version strings are reported correctly.

**Invalid or Missing YAML Configurations**

- ``test_no_yml_argument``: No `-y` argument results in a helpful usage error.
- ``test_errors_wrong_path``: YAML path does not exist.
- ``test_errors_bad_value``: Malformed input values in YAML cause errors.
- ``test_errors_missing_key``: YAML is missing required sections like `build_existing_model`.
- ``test_errors_weather_files``: Missing weather files directory triggers configuration error.
- ``test_errors_invalid_upgrade_name``: Upgrade name misspelled or not found in valid list.
- ``test_errors_precomputed_outdated_missing_parameter``: Parameter mismatch between `buildstock.csv` and `options_lookup.tsv`.
- ``test_errors_precomputed_outdated_extra_parameter``: Additional unexpected parameter detected in project files.

**Unsupported Sampling Modes**

- ``test_errors_downselect_resample``: `resample` sampling type is not yet supported.
- ``test_errors_downsampler``: Invalid or unknown sampler type.
- ``test_errors_already_exists``: Output directory already exists before test execution.

**Selective Pipeline Execution**

- ``test_measures_only``: Executes only the measure portion of the pipeline (`-m`).
- ``test_sampling_only``: Executes only the sampling portion (`-s`).
- ``test_building_id``: Runs analysis for a specific building ID (`-i`).
- ``test_upgrade_name``: Runs analysis for multiple specified upgrades.
- ``test_threads_and_keep_run_folders``: Uses threading and preserves run directories (`-n`, `-k`).

**Precomputed and Sample Weights**

- ``test_precomputed``: Uses precomputed `buildstock.csv` with sample weights.
- ``test_precomputed_sample_weight``: Confirms weight values are correctly applied and balanced.

**Full End-to-End Runs**

- ``test_testing_baseline``: Executes the ResStock workflow using the `project_testing/testing_baseline.yml` file. Confirms expected outputs including `.osw`, `.csv`, `.xml`, and timeseries results.
- ``test_national_baseline``: Same as above, but for `project_national/national_baseline.yml`.

**Relative Paths**

- ``test_relative_weather_files_path``: Ensures weather files can be pulled using relative paths.