Roll forward (dotnet/core-setup#5691)
authorVitek Karas <vitek.karas@microsoft.com>
Fri, 19 Apr 2019 21:35:58 +0000 (14:35 -0700)
committerGitHub <noreply@github.com>
Fri, 19 Apr 2019 21:35:58 +0000 (14:35 -0700)
This is the new implementation of the framework resolution with the new roll forward setting.
The spec for this is [framework version resolution](https://github.com/dotnet/core-setup/blob/master/Documentation/design-docs/framework-version-resolution.md).

Major parts:
- introduce `rollForward` setting (CLI, env, config)
- convert the existing `rollForwardOnNoCandidateFx` to the new `rollForward` internally
- change framework reference compatibility algorithm to use the new `rollFoward` settings.
- basically rewrite the framework resolution algorithm with the new settings and fix an existing bug with incorrect retry logic
- add test infra for the new setting
- add tests for the new setting

Commit migrated from https://github.com/dotnet/core-setup/commit/ef496a091be8d8b43ab8c4ed3622ca89bce251cb

35 files changed:
docs/installer/design-docs/framework-version-resolution.md
src/installer/corehost/cli/fx_definition.cpp
src/installer/corehost/cli/fx_definition.h
src/installer/corehost/cli/fx_reference.cpp
src/installer/corehost/cli/fx_reference.h
src/installer/corehost/cli/fxr/CMakeLists.txt
src/installer/corehost/cli/fxr/fx_muxer.cpp
src/installer/corehost/cli/fxr/fx_resolver.cpp
src/installer/corehost/cli/fxr/fx_resolver.h
src/installer/corehost/cli/fxr/fx_resolver.messages.cpp
src/installer/corehost/cli/hostpolicy/CMakeLists.txt
src/installer/corehost/cli/hostpolicy/hostpolicy.cpp
src/installer/corehost/cli/roll_forward_option.cpp [new file with mode: 0644]
src/installer/corehost/cli/roll_forward_option.h [new file with mode: 0644]
src/installer/corehost/cli/roll_fwd_on_no_candidate_fx_option.h
src/installer/corehost/cli/runtime_config.cpp
src/installer/corehost/cli/runtime_config.h
src/installer/corehost/common/utils.cpp
src/installer/test/HostActivationTests/Constants.cs
src/installer/test/HostActivationTests/FrameworkResolution/ApplyPatchesSettings.cs
src/installer/test/HostActivationTests/FrameworkResolution/DotNetCliExtensions.cs
src/installer/test/HostActivationTests/FrameworkResolution/FrameworkResolutionBase.Settings.cs
src/installer/test/HostActivationTests/FrameworkResolution/FrameworkResolutionBase.cs
src/installer/test/HostActivationTests/FrameworkResolution/FrameworkResolutionCommandResultExtensions.cs
src/installer/test/HostActivationTests/FrameworkResolution/FxVersionCLI.cs
src/installer/test/HostActivationTests/FrameworkResolution/MultipleHives.cs
src/installer/test/HostActivationTests/FrameworkResolution/RollForwardMultipleFrameworks.cs [new file with mode: 0644]
src/installer/test/HostActivationTests/FrameworkResolution/RollForwardOnNoCandidateFx.cs
src/installer/test/HostActivationTests/FrameworkResolution/RollForwardOnNoCandidateFxMultipleFrameworks.cs
src/installer/test/HostActivationTests/FrameworkResolution/RollForwardOnNoCandidateFxSettings.cs
src/installer/test/HostActivationTests/FrameworkResolution/RollForwardPreReleaseOnly.cs [new file with mode: 0644]
src/installer/test/HostActivationTests/FrameworkResolution/RollForwardReleaseAndPreRelease.cs [new file with mode: 0644]
src/installer/test/HostActivationTests/FrameworkResolution/RollForwardReleaseOnly.cs [new file with mode: 0644]
src/installer/test/HostActivationTests/FrameworkResolution/RollForwardSettings.cs [new file with mode: 0644]
src/installer/test/HostActivationTests/RuntimeConfig.cs

index 6f0849e..59a2b5c 100644 (file)
@@ -269,7 +269,7 @@ To maintain backward compatibility, each framework reference will also have to c
 Terminology
 - `framework reference`: consists of framework `name`, `version`, `rollForward` and optionally `applyPatches`.
 - `config fx references`: `framework references` for a single `.runtimeconfig.json`.
-- `newest fx references`: dictionary of `framework references` keyed off of framework `name` that contains the highest `version` requested and most constrained `rollForward` and `applyPatches`. It is used to perform "soft roll-forwards" to compatible references of the same framework name without reading the disk or performing excessive re-try.
+- `effective fx references`: dictionary of `framework references` keyed off of framework `name` that contains the highest `version` requested and merged `rollForward` and `applyPatches`. It is used to track the most up to date effective framework reference without reading the disk, it prevents excessive re-tries of the resolution.
 - `resolved frameworks`: a list of frameworks that have been resolved, meaning a compatible framework was found on disk.
 
 Steps
@@ -277,31 +277,31 @@ Steps
    * Parse the application's `.runtimeconfig.json` `runtimeOptions.frameworks` section.
    * Insert each `framework reference` into the `config fx references`.
 2. For each `framework reference` in `config fx references`:
-3. --> If the framework `name` is not currently in the `newest fx references` list Then add it.
+3. --> If the framework `name` is not currently in the `effective fx references` list Then add it.
    * By doing this for all `framework references` here, before the next loop, we minimize the number of re-try attempts.
 4. For each `framework reference` in `config fx references`:
 5. --> If the framework's `name` is not in `resolved frameworks` Then resolve the `framework reference` to the actual framework on disk:
-   * If the framework `name` already exists in the `newest fx references` resolve the currently processed `framework reference` with the one from the `newest fx references` (see above for the algorithm). 
-   *Term "soft roll-forward" is used for this in the code*
-     * The resolution will always pick the higher `version` and will consolidate the `rollForward` and `applyPatches` settings to the most constrained.
-     * The resolution may fail if it's not possible to roll forward from one `framework reference` to the other.
-     * Update the `newest fx references` with the resolved `framework reference` (note that this may be a combination of version and settings from the two `framework references` being considered).
-   * Probe for the framework on disk using the whole `framework reference` (which by now is the `framework reference` in `newest fx references`)
-   *Term "hard resolve" is used for this in the code*
-     * This follows the roll-forward rules as describe above.
+   * If the framework `name` already exists in the `effective fx references` reconcile the currently processed `framework reference` with the one from the `effective fx references` (see above for the algorithm). 
+   *Term "reconcile framework references" is used for this in the code, this used to be called "soft-roll-forward" as well.*
+     * The reconciliation will always pick the higher `version` and will merge the `rollForward` and `applyPatches` settings.
+     * The reconciliation may fail if it's not possible to roll forward from one `framework reference` to the other.
+     * Update the `effective fx references` with the reconciled `framework reference` (note that this may be a combination of version and settings from the two `framework references` being considered).
+   * Resolve the `framework reference` (which by now is the one from `effective fx references`) against the frameworks available on the disk
+   *Sometimes this is referred to as "hard-roll-forward".*
+     * This follows the roll-forward framework selection rules as describe above.
    * If success add it to `resolved frameworks`
      * Parse the `.runtimeconfig.json` of the resolved framework and create a new `config fx references`. Make a recursive call back to Step 2 with these new `config fx references`.
      * Continue with the next `framework reference` (Step 4).
-6. --> Else perform a "soft roll-forward" to the `framework reference` in `newest fx references`.
+6. --> Else perform reconcile the `framework reference` with the one from `effective fx references`.
    * We may fail here if not compatible.
-   * If the roll-forward results in a different `framework reference` than the one in `newest fx references`
-     * Update the `framework reference` in `newest fx references`
-     * Re-start the algorithm (goto Step 1) with new/clear state except for `newest fx references` so we attempt to use the newer `framework reference` next time.
-   * Else (no need to change the `newest fx references`) - use the already resolved framework and continue with the next `framework reference` (Step 4).
+   * If the reconciliation results in a different `framework reference` than the one in `effective fx references`
+     * Update the `framework reference` in `effective fx references`
+     * Re-start the algorithm (goto Step 1) with new/clear state except for `effective fx references` so we attempt to use the newer `framework reference` next time.
+   * Else (no need to change the `effective fx references`) - use the already resolved framework and continue with the next `framework reference` (Step 4).
 
 Notes on this algorithm:
 * This algorithm for resolving the various framework references assumes the **No Downgrading** best practice explained below in order to prevent loading a newer version of a framework than necessary.
-* Probing for the framework on disk never changes the `newest fx references`. This means that the `newest fx references` contains the latest effective framework reference for each framework without considering what frameworks are actually available. This is very important to avoid ordering issues. (See the **Fixing ordering issues** in the sections below.)
+* Probing for the framework on disk never changes the `effective fx references`. This means that the `effective fx references` contains the latest effective framework reference for each framework without considering what frameworks are actually available. This is very important to avoid ordering issues. (See the **Fixing ordering issues** in the sections below.)
 
 
 ### Best practices for a `.runtimeconfig.json`
index da0f9e0..2fdd3a7 100644 (file)
@@ -27,11 +27,10 @@ fx_definition_t::fx_definition_t(
 void fx_definition_t::parse_runtime_config(
     const pal::string_t& path,
     const pal::string_t& dev_path,
-    const fx_reference_t& fx_ref,
-    const fx_reference_t& override_settings
+    const runtime_config_t::settings_t& override_settings
 )
 {
-    m_runtime_config.parse(path, dev_path, fx_ref, override_settings);
+    m_runtime_config.parse(path, dev_path, override_settings);
 }
 
 void fx_definition_t::parse_deps()
index a918ea5..8420816 100644 (file)
@@ -24,7 +24,7 @@ public:
     const pal::string_t& get_found_version() const { return m_found_version; }
     const pal::string_t& get_dir() const { return m_dir; }
     const runtime_config_t& get_runtime_config() const { return m_runtime_config; }
-    void parse_runtime_config(const pal::string_t& path, const pal::string_t& dev_path, const fx_reference_t& fx_ref, const fx_reference_t& override_settings);
+    void parse_runtime_config(const pal::string_t& path, const pal::string_t& dev_path, const runtime_config_t::settings_t& override_settings);
 
     const pal::string_t& get_deps_file() const { return m_deps_file; }
     void set_deps_file(const pal::string_t value) { m_deps_file = value; }
index d0ce95f..c06fdda 100644 (file)
@@ -7,99 +7,92 @@
 #include "fx_reference.h"
 #include "roll_fwd_on_no_candidate_fx_option.h"
 
-bool fx_reference_t::is_roll_forward_compatible(const fx_ver_t& other) const
+bool fx_reference_t::is_compatible_with_higher_version(const fx_ver_t& higher_version) const
 {
-    // We expect the version to be <
-    assert(get_fx_version_number() < other);
+    assert(get_fx_version_number() <= higher_version);
 
-    if (get_fx_version_number() == other)
+    if (get_fx_version_number() == higher_version)
     {
         return true;
     }
 
-    if (get_use_exact_version())
-    {
-        return false;
-    }
-
     // Verify major roll forward
-    if (get_fx_version_number().get_major() != other.get_major()
-        && roll_fwd_on_no_candidate_fx != roll_fwd_on_no_candidate_fx_option::major)
+    if (get_fx_version_number().get_major() != higher_version.get_major()
+        && roll_forward < roll_forward_option::Major)
     {
         return false;
     }
 
     // Verify minor roll forward
-    if (get_fx_version_number().get_minor() != other.get_minor()
-        && roll_fwd_on_no_candidate_fx != roll_fwd_on_no_candidate_fx_option::major
-        && roll_fwd_on_no_candidate_fx != roll_fwd_on_no_candidate_fx_option::minor)
+    if (get_fx_version_number().get_minor() != higher_version.get_minor()
+        && roll_forward < roll_forward_option::Minor)
     {
         return false;
     }
 
     // Verify patch roll forward
-    // We do not distinguish here whether a previous framework reference found a patch version based on:
-    //  - initial reference matching a patch version,
-    //  - or roll_fwd_on_no_candidate_fx_option=major\minor finding a compatible patch version as initial framework,
-    //  - or applyPatches=true finding a newer patch version
-    if (get_fx_version_number().get_patch() != other.get_patch()
-        && patch_roll_fwd == false
-        && roll_fwd_on_no_candidate_fx != roll_fwd_on_no_candidate_fx_option::major
-        && roll_fwd_on_no_candidate_fx != roll_fwd_on_no_candidate_fx_option::minor)
+    if (get_fx_version_number().get_patch() != higher_version.get_patch()
+        && roll_forward == roll_forward_option::LatestPatch
+        && apply_patches == false)
     {
         return false;
     }
 
-    // Release cannot roll forward to pre-release
-    if (!get_fx_version_number().is_prerelease() && other.is_prerelease())
+    // In here it means that either everything but pre-release part is the same, or the difference is OK
+    // The roll-forward rules don't affect pre-release roll forward except when
+    //  - rollForward is Disable - in which case no roll forward should occur, and the versions must exactly match
+    //  - rollForward is LatestPatch and applyPatches=false - which would normally mean exactly the same as Disable, but
+    //    for backward compat reasons this is a special case. In this case applyPatches is ignored for pre-release versions.
+    //    So even if pre-release are different, the versions are compatible.
+    if (roll_forward == roll_forward_option::Disable)
     {
+        // We know the versions are different since we compared 100% equality above, so they're not compatible.
+        // In here the versions could differ in patch or pre-release, in both cases they're not compatible.
         return false;
     }
 
+    // Concerning pre-release versions
+    //  - Pre-release is allowed to roll to any version (release or pre-release)
+    //  - Release should prefer rolling to release, but is allowed to roll to pre-release if no compatible release is available
+    // This function only compares framework references, it doesn't resolve framework reference to the available framework on disk.
+    // As such it can't implement the "release should prefer release" as that requires the knowledge of all available versions.
+
     return true;
 }
 
-void fx_reference_t::apply_settings_from(const fx_reference_t& from)
+void fx_reference_t::merge_roll_forward_settings_from(const fx_reference_t& from)
 {
-    if (from.get_fx_version().length() > 0)
-    {
-        set_fx_version(from.get_fx_version());
-    }
+    // In general lower value roll_forward should win, with one exception
+    // If Major and LatestMinor is merged, the result should be Minor for now.
+    //   - this is the conservative approach where the most strict behavior is taken
+    //       Major means "closest available version and allow roll over major"
+    //       LatestMinor means "latest available version and allow roll over minor"
+    //       So the most restrictive merge would be "closest available version and allow roll over minor"
+    //       which is exactly what Minor does.
+    // All other combinations are already following the same logic.
 
-    const roll_fwd_on_no_candidate_fx_option* from_rollfwd = from.get_roll_fwd_on_no_candidate_fx();
-    if (from_rollfwd != nullptr)
+    if (from.get_roll_forward() < get_roll_forward())
     {
-        set_roll_fwd_on_no_candidate_fx(*from_rollfwd);
-    }
+        roll_forward_option effective_roll_forward = from.get_roll_forward();
+        if (from.get_roll_forward() == roll_forward_option::LatestMinor && get_roll_forward() == roll_forward_option::Major)
+        {
+            effective_roll_forward = roll_forward_option::Minor;
+        }
 
-    const bool* from_patch = from.get_patch_roll_fwd();
-    if (from_patch != nullptr)
+        set_roll_forward(effective_roll_forward);
+    }
+    else if (from.get_roll_forward() == roll_forward_option::Major && get_roll_forward() == roll_forward_option::LatestMinor)
     {
-        set_patch_roll_fwd(*from_patch);
+        set_roll_forward(roll_forward_option::Minor);
     }
-}
 
-void fx_reference_t::merge_roll_forward_settings_from(const fx_reference_t& from)
-{
-    const roll_fwd_on_no_candidate_fx_option* from_rollfwd = from.get_roll_fwd_on_no_candidate_fx();
-    if (from_rollfwd != nullptr)
+    if (get_apply_patches() == true && from.get_apply_patches() == false)
     {
-        const roll_fwd_on_no_candidate_fx_option* to_rollfwd = get_roll_fwd_on_no_candidate_fx();
-        if (to_rollfwd == nullptr ||
-            *from_rollfwd < *to_rollfwd)
-        {
-            set_roll_fwd_on_no_candidate_fx(*from_rollfwd);
-        }
+        set_apply_patches(false);
     }
 
-    const bool* from_patch = from.get_patch_roll_fwd();
-    if (from_patch != nullptr)
+    if (from.get_prefer_release() && !get_prefer_release())
     {
-        const bool* to_patch = get_patch_roll_fwd();
-        if (to_patch == nullptr ||
-            *from_patch == false)
-        {
-            set_patch_roll_fwd(*from_patch);
-        }
+        set_prefer_release(true);
     }
 }
index 097cfdc..69de7f9 100644 (file)
@@ -8,17 +8,15 @@
 #include <list>
 #include "pal.h"
 #include "fx_ver.h"
-#include "roll_fwd_on_no_candidate_fx_option.h"
+#include "roll_forward_option.h"
 
 class fx_reference_t
 {
 public:
     fx_reference_t()
-        : has_patch_roll_fwd(false)
-        , patch_roll_fwd(false)
-        , has_roll_fwd_on_no_candidate_fx(false)
-        , roll_fwd_on_no_candidate_fx((roll_fwd_on_no_candidate_fx_option)0)
-        , use_exact_version(false)
+        : apply_patches(true)
+        , roll_forward(roll_forward_option::Minor)
+        , prefer_release(false)
         , fx_name(_X(""))
         , fx_version(_X(""))
         , fx_version_number()
@@ -49,52 +47,62 @@ public:
         return fx_version_number;
     }
 
-    const bool* get_patch_roll_fwd() const
+    const bool get_apply_patches() const
     {
-        return (has_patch_roll_fwd ? &patch_roll_fwd : nullptr);
+        return apply_patches;
     }
-    void set_patch_roll_fwd(bool value)
+    void set_apply_patches(bool value)
     {
-        has_patch_roll_fwd = true;
-        patch_roll_fwd = value;
+        apply_patches = value;
     }
 
-    const bool get_use_exact_version() const
+    const roll_forward_option get_roll_forward() const
     {
-        return use_exact_version;
+        return roll_forward;
     }
-    void set_use_exact_version(bool value)
+    void set_roll_forward(roll_forward_option value)
     {
-        use_exact_version = value;
+        roll_forward = value;
     }
 
-    const roll_fwd_on_no_candidate_fx_option* get_roll_fwd_on_no_candidate_fx() const
+    const bool get_prefer_release() const
     {
-        return (has_roll_fwd_on_no_candidate_fx ? &roll_fwd_on_no_candidate_fx : nullptr);
+        return prefer_release;
     }
-    void set_roll_fwd_on_no_candidate_fx(roll_fwd_on_no_candidate_fx_option value)
+    void set_prefer_release(bool value)
     {
-        has_roll_fwd_on_no_candidate_fx = true;
-        roll_fwd_on_no_candidate_fx = value;
+        prefer_release = value;
     }
 
-    // Is the current version compatible with another instance with roll-forward semantics.
-    bool is_roll_forward_compatible(const fx_ver_t& other) const;
+    // Is the current version compatible with the specified equal or higher version.
+    bool is_compatible_with_higher_version(const fx_ver_t& higher_version) const;
 
-    // Copy over any non-null values
-    void apply_settings_from(const fx_reference_t& from);
-
-    // Apply the most restrictive settings
+    // Merge roll forward settings for two framework references
     void merge_roll_forward_settings_from(const fx_reference_t& from);
 
+    bool operator==(const fx_reference_t& other)
+    {
+        return
+            fx_name == other.fx_name &&
+            fx_version == other.fx_version &&
+            apply_patches == other.apply_patches &&
+            roll_forward == other.roll_forward &&
+            prefer_release == other.prefer_release;
+    }
+
+    bool operator!=(const fx_reference_t& other)
+    {
+        return !(*this == other);
+    }
+
 private:
-    bool has_patch_roll_fwd;
-    bool patch_roll_fwd;
+    bool apply_patches;
 
-    bool has_roll_fwd_on_no_candidate_fx;
-    roll_fwd_on_no_candidate_fx_option roll_fwd_on_no_candidate_fx;
+    roll_forward_option roll_forward;
 
-    bool use_exact_version;
+    // This indicates that when resolving the framework reference the search should prefer release version
+    // and only resolve to pre-release if there's no matching release version available.
+    bool prefer_release;
 
     pal::string_t fx_name;
 
index 92c9075..f01169a 100644 (file)
@@ -15,6 +15,7 @@ set(SOURCES
     ../deps_format.cpp
     ../deps_entry.cpp
     ../host_startup_info.cpp
+    ../roll_forward_option.cpp
     ../runtime_config.cpp
     ../json/casablanca/src/json/json.cpp
     ../json/casablanca/src/json/json_parsing.cpp
@@ -43,6 +44,8 @@ set(HEADERS
     ../json/casablanca/include/cpprest/json.h
     ../fx_definition.h
     ../fx_reference.h
+    ../roll_forward_option.h
+    ../roll_fwd_on_no_candidate_fx_option.h
     ../version.h
     ./corehost_init.h
     ./fx_ver.h
index 01e63d0..4172aa4 100644 (file)
@@ -23,6 +23,7 @@
 #include "runtime_config.h"
 #include "sdk_info.h"
 #include "sdk_resolver.h"
+#include "roll_fwd_on_no_candidate_fx_option.h"
 
 using corehost_main_fn = int(*) (const int argc, const pal::char_t* argv[]);
 using corehost_get_delegate_fn = int(*)(coreclr_delegate_type type, void** delegate);
@@ -268,6 +269,7 @@ std::vector<host_option> fx_muxer_t::get_known_opts(bool exec_mode, host_mode_t
     {
         // If mode=host_mode_t::apphost, these are only used when the app is framework-dependent.
         known_opts.push_back({ _X("--fx-version"), _X("<version>"), _X("Version of the installed Shared Framework to use to run the application.")});
+        known_opts.push_back({ _X("--roll-forward"), _X("<value>"), _X("Roll forward to framework version (LatestPatch, Minor, LatestMinor, Major, LatestMajor, Disable)")});
         known_opts.push_back({ _X("--roll-forward-on-no-candidate-fx"), _X("<n>"), _X("Roll forward on no candidate framework (0=off, 1=roll minor, 2=roll major & minor).")});
         known_opts.push_back({ _X("--additional-deps"), _X("<path>"), _X("Path to additional deps.json file.")});
     }
@@ -365,7 +367,7 @@ namespace
         fx_definition_t& app,
         const pal::string_t& app_candidate,
         pal::string_t& runtime_config,
-        const fx_reference_t& override_settings)
+        const runtime_config_t::settings_t& override_settings)
     {
         if (!runtime_config.empty() && !pal::realpath(&runtime_config))
         {
@@ -386,7 +388,7 @@ namespace
             get_runtime_config_paths_from_arg(runtime_config, &config_file, &dev_config_file);
         }
 
-        app.parse_runtime_config(config_file, dev_config_file, fx_reference_t(), override_settings);
+        app.parse_runtime_config(config_file, dev_config_file, override_settings);
         if (!app.get_runtime_config().is_valid())
         {
             trace::error(_X("Invalid runtimeconfig.json [%s] [%s]"), app.get_runtime_config().get_path().c_str(), app.get_runtime_config().get_dev_path().c_str());
@@ -461,6 +463,7 @@ namespace
         std::unique_ptr<corehost_init_t> &init)
     {
         pal::string_t opts_fx_version = _X("--fx-version");
+        pal::string_t opts_roll_forward = _X("--roll-forward");
         pal::string_t opts_roll_fwd_on_no_candidate_fx = _X("--roll-forward-on-no-candidate-fx");
         pal::string_t opts_deps_file = _X("--depsfile");
         pal::string_t opts_probe_path = _X("--additionalprobingpath");
@@ -476,19 +479,45 @@ namespace
             return StatusCode::InvalidArgFailure;
         }
 
-        fx_reference_t override_settings;
-
-        // 'Roll forward on no candidate fx' is set to 1 (roll_fwd_on_no_candidate_fx_option::minor) by default. It can be changed through:
-        // 1. Command line argument (--roll-forward-on-no-candidate-fx).
-        // 2. Runtimeconfig json file ('rollForwardOnNoCandidateFx' property in "framework" section).
-        // 3. Runtimeconfig json file ('rollForwardOnNoCandidateFx' property in a referencing "frameworks" section).
-        // 4. DOTNET_ROLL_FORWARD_ON_NO_CANDIDATE_FX env var.
-        // The conflicts will be resolved by following the priority rank described above (from 1 to 4).
+        runtime_config_t::settings_t override_settings;
+
+        // `Roll forward` is set to Minor (2) (roll_forward_option::Minor) by default. 
+        // For backward compatibility there are two settings:
+        //  - rollForward (the new one) which has more possible values
+        //  - rollForwardOnNoCandidateFx (the old one) with only 0-Off, 1-Minor, 2-Major
+        // It can be changed through:
+        // 1. Command line argument --roll-forward or --roll-forward-on-no-candidate-fx
+        // 2. DOTNET_ROLL_FORWARD env var.
+        // 3. Runtimeconfig json file ('rollForward' or 'rollForwardOnNoCandidateFx' property in "framework" section).
+        // 4. Runtimeconfig json file ('rollForward' or 'rollForwardOnNoCandidateFx' property in a "runtimeOptions" section).
+        // 5. DOTNET_ROLL_FORWARD_ON_NO_CANDIDATE_FX env var.
+        // The conflicts will be resolved by following the priority rank described above (from 1 to 5, lower number wins over higher number).
         // The env var condition is verified in the config file processing
+
+        pal::string_t roll_forward = get_last_known_arg(opts, opts_roll_forward, _X(""));
+        if (roll_forward.length() > 0)
+        {
+            auto val = roll_forward_option_from_string(roll_forward);
+            if (val == roll_forward_option::__Last)
+            {
+                trace::error(_X("Invalid value for command line argument '%s'"), opts_roll_forward.c_str());
+                return StatusCode::InvalidArgFailure;
+            }
+
+            override_settings.set_roll_forward(val);
+        }
+
         pal::string_t roll_fwd_on_no_candidate_fx = get_last_known_arg(opts, opts_roll_fwd_on_no_candidate_fx, _X(""));
         if (roll_fwd_on_no_candidate_fx.length() > 0)
         {
-            override_settings.set_roll_fwd_on_no_candidate_fx(static_cast<roll_fwd_on_no_candidate_fx_option>(pal::xtoi(roll_fwd_on_no_candidate_fx.c_str())));
+            if (override_settings.has_roll_forward)
+            {
+                trace::error(_X("It's invalid to use both '%s' and '%s' command line options."), opts_roll_forward.c_str(), opts_roll_fwd_on_no_candidate_fx.c_str());
+                return StatusCode::InvalidArgFailure;
+            }
+
+            auto val = static_cast<roll_fwd_on_no_candidate_fx_option>(pal::xtoi(roll_fwd_on_no_candidate_fx.c_str()));
+            override_settings.set_roll_forward(roll_fwd_on_no_candidate_fx_to_roll_forward(val));
         }
 
         // Read config
@@ -720,7 +749,7 @@ namespace
         auto app = new fx_definition_t();
         fx_definitions.push_back(std::unique_ptr<fx_definition_t>(app));
 
-        fx_reference_t override_settings;
+        runtime_config_t::settings_t override_settings;
         int rc = read_config(*app, host_info.app_path, runtime_config_path, override_settings);
         if (rc != StatusCode::Success)
             return rc;
index 90cf5b8..3e3b2ec 100644 (file)
@@ -10,142 +10,198 @@ namespace
 {
     const int Max_Framework_Resolve_Retries = 100;
 
-    fx_ver_t resolve_framework_version(
+    static_assert(roll_forward_option::LatestPatch > roll_forward_option::Disable, "Code assumes ordering of roll-forward options from least restrictive to most restrictive");
+    static_assert(roll_forward_option::Minor > roll_forward_option::LatestPatch, "Code assumes ordering of roll-forward options from least restrictive to most restrictive");
+    static_assert(roll_forward_option::LatestMinor > roll_forward_option::Minor, "Code assumes ordering of roll-forward options from least restrictive to most restrictive");
+    static_assert(roll_forward_option::Major > roll_forward_option::LatestMinor, "Code assumes ordering of roll-forward options from least restrictive to most restrictive");
+    static_assert(roll_forward_option::LatestMajor > roll_forward_option::Major, "Code assumes ordering of roll-forward options from least restrictive to most restrictive");
+
+    fx_ver_t search_for_best_framework_major_or_minor_match(
         const std::vector<fx_ver_t>& version_list,
-        const pal::string_t& fx_ver,
-        const fx_ver_t& specified,
-        bool patch_roll_fwd,
-        roll_fwd_on_no_candidate_fx_option roll_fwd_on_no_candidate_fx)
+        const fx_reference_t& fx_ref,
+        bool release_only)
     {
-        trace::verbose(_X("Attempting FX roll forward starting from [%s]"), fx_ver.c_str());
+        fx_ver_t best_match_version;
 
-        fx_ver_t most_compatible = specified;
-        if (!specified.is_prerelease())
+        if (fx_ref.get_roll_forward() > roll_forward_option::LatestPatch)
         {
-            if (roll_fwd_on_no_candidate_fx != roll_fwd_on_no_candidate_fx_option::disabled)
-            {
-                fx_ver_t next_lowest;
+            bool search_for_latest = fx_ref.get_roll_forward() == roll_forward_option::LatestMinor || fx_ref.get_roll_forward() == roll_forward_option::LatestMajor;
 
-                // Look for the least production version
-                trace::verbose(_X("'Roll forward on no candidate fx' enabled with value [%d]. Looking for the least production greater than or equal to [%s]"),
-                    roll_fwd_on_no_candidate_fx, fx_ver.c_str());
+            trace::verbose(
+                _X("'Roll forward' enabled with value [%d]. Looking for the %s %s greater than or equal version to [%s]"),
+                fx_ref.get_roll_forward(),
+                search_for_latest ? _X("latest") : _X("least"),
+                release_only ? _X("release") : _X("release/pre-release"),
+                fx_ref.get_fx_version().c_str());
 
-                for (const auto& ver : version_list)
+            for (const auto& ver : version_list)
+            {
+                if ((!release_only || !ver.is_prerelease()) && ver >= fx_ref.get_fx_version_number())
                 {
-                    if (!ver.is_prerelease() && ver >= specified)
+                    if (fx_ref.get_roll_forward() <= roll_forward_option::LatestMinor)
                     {
-                        if (roll_fwd_on_no_candidate_fx == roll_fwd_on_no_candidate_fx_option::minor)
+                        if (ver.get_major() != fx_ref.get_fx_version_number().get_major())
                         {
-                            // We only want to roll forward on minor
-                            if (ver.get_major() != specified.get_major())
-                            {
-                                continue;
-                            }
+                            continue;
                         }
-                        next_lowest = (next_lowest == fx_ver_t()) ? ver : std::min(next_lowest, ver);
-                    }
-                }
-
-                if (next_lowest == fx_ver_t())
-                {
-                    // Look for the least preview version
-                    trace::verbose(_X("No production greater than or equal to [%s] found. Looking for the least preview greater than [%s]"),
-                        fx_ver.c_str(), fx_ver.c_str());
 
-                    for (const auto& ver : version_list)
-                    {
-                        if (ver.is_prerelease() && ver >= specified)
+                        if (fx_ref.get_roll_forward() <= roll_forward_option::LatestPatch)
                         {
-                            if (roll_fwd_on_no_candidate_fx == roll_fwd_on_no_candidate_fx_option::minor)
+                            if (ver.get_minor() != fx_ref.get_fx_version_number().get_minor())
                             {
-                                // We only want to roll forward on minor
-                                if (ver.get_major() != specified.get_major())
-                                {
-                                    continue;
-                                }
+                                continue;
                             }
-                            next_lowest = (next_lowest == fx_ver_t()) ? ver : std::min(next_lowest, ver);
                         }
                     }
-                }
 
-                if (next_lowest == fx_ver_t())
-                {
-                    trace::verbose(_X("No preview greater than or equal to [%s] found."), fx_ver.c_str());
-                }
-                else
-                {
-                    trace::verbose(_X("Found version [%s]"), next_lowest.as_str().c_str());
-                    most_compatible = next_lowest;
+                    best_match_version = (best_match_version == fx_ver_t())
+                        ? ver
+                        : (search_for_latest ? std::max(best_match_version, ver) : std::min(best_match_version, ver));
                 }
             }
 
-            if (patch_roll_fwd)
+            if (best_match_version == fx_ver_t())
             {
-                trace::verbose(_X("Applying patch roll forward from [%s]"), most_compatible.as_str().c_str());
-                for (const auto& ver : version_list)
-                {
-                    trace::verbose(_X("Inspecting version... [%s]"), ver.as_str().c_str());
-
-                    if (most_compatible.is_prerelease() == ver.is_prerelease() && // prevent production from rolling forward to preview on patch
-                        ver.get_major() == most_compatible.get_major() &&
-                        ver.get_minor() == most_compatible.get_minor())
-                    {
-                        // Pick the greatest that differs only in patch.
-                        most_compatible = std::max(ver, most_compatible);
-                    }
-                }
+                trace::verbose(_X("No match greater than or equal to [%s] found."), fx_ref.get_fx_version().c_str());
+            }
+            else
+            {
+                trace::verbose(_X("Found version [%s]"), best_match_version.as_str().c_str());
             }
         }
-        else
+
+        return best_match_version;
+    }
+
+    fx_ver_t search_for_latest_patch(
+        const std::vector<fx_ver_t>& version_list,
+        const fx_reference_t& fx_ref,
+        const fx_ver_t& start_with_version,
+        bool release_only)
+    {
+        fx_ver_t best_match_version = start_with_version;
+
+        // For LatestMinor and LatestMajor the above search should already find the latest patch (it looks for latest version as a whole).
+        // For Disable, there's no roll forward (in fact we should not even get here).
+        // For Major and Minor, we need to look for latest patch as the above would have found the lowest patch (as it looks for lowest version as a whole).
+        // For LatestPatch we haven't found any version yet, so simply find the latest patch.
+        //   For backward compatibility reasons we also need to consider the apply_patches setting though.
+        //   For backward compatibility reasons the apply_patches for pre-release framework reference only applies to the patch portion of the version,
+        //     the pre-release portion of the version ignores apply_patches and we should roll to the latest (100% backward would roll to closest, but for consistency
+        //     in the new behavior we will roll to latest).
+        if ((fx_ref.get_roll_forward() == roll_forward_option::LatestPatch || 
+             fx_ref.get_roll_forward() == roll_forward_option::Minor || 
+             fx_ref.get_roll_forward() == roll_forward_option::Major)
+            && (fx_ref.get_apply_patches() || fx_ref.get_fx_version_number().is_prerelease()))
         {
-            // pre-release has its own roll forward rules and ignores roll_fwd_on_no_candidate_fx and patch_roll_fwd
+            fx_ver_t apply_patch_from_version = start_with_version;
+            if (apply_patch_from_version == fx_ver_t())
+            {
+                apply_patch_from_version = fx_ref.get_fx_version_number();
+            }
+
+            trace::verbose(
+                _X("Applying patch roll forward from [%s] on %s"),
+                apply_patch_from_version.as_str().c_str(),
+                release_only ? _X("release only") : _X("release/pre-release"));
+
             for (const auto& ver : version_list)
             {
                 trace::verbose(_X("Inspecting version... [%s]"), ver.as_str().c_str());
 
-                if (ver.is_prerelease() && // prevent roll forward to production.
-                    ver.get_major() == specified.get_major() &&
-                    ver.get_minor() == specified.get_minor() &&
-                    ver.get_patch() == specified.get_patch() &&
-                    ver > specified)
+                if ((!release_only || !ver.is_prerelease()) &&
+                    (fx_ref.get_apply_patches() || ver.get_patch() == apply_patch_from_version.get_patch()) &&
+                    ver >= apply_patch_from_version &&
+                    ver.get_major() == apply_patch_from_version.get_major() &&
+                    ver.get_minor() == apply_patch_from_version.get_minor())
                 {
-                    // Pick the smallest prerelease that is greater than specified.
-                    most_compatible = (most_compatible == specified) ? ver : std::min(ver, most_compatible);
+                    // Pick the greatest that differs only in patch.
+                    best_match_version = std::max(ver, best_match_version);
                 }
             }
         }
 
-        return most_compatible;
+        return best_match_version;
+    }
+
+    fx_ver_t search_for_best_framework_match(
+        const std::vector<fx_ver_t>& version_list,
+        const fx_reference_t& fx_ref,
+        bool release_only)
+    {
+        // Roll forward to the best major.minor.* version
+        fx_ver_t best_match_version = search_for_best_framework_major_or_minor_match(version_list, fx_ref, release_only);
+
+        // Roll forward to the latest patch for a given major.minor
+        best_match_version = search_for_latest_patch(version_list, fx_ref, best_match_version, release_only);
+
+        return best_match_version;
+    }
+
+    fx_ver_t resolve_framework_reference_from_version_list(
+        const std::vector<fx_ver_t>& version_list,
+        const fx_reference_t& fx_ref)
+    {
+        trace::verbose(
+            _X("Attempting FX roll forward starting from version='[%s]', apply_patches=%d, roll_forward=%s, prefer_release=%d"),
+            fx_ref.get_fx_version().c_str(),
+            fx_ref.get_apply_patches(),
+            roll_forward_option_to_string(fx_ref.get_roll_forward()).c_str(),
+            fx_ref.get_prefer_release());
+
+        // If the framework reference prefers release, then search for release versions only first.
+        if (fx_ref.get_prefer_release())
+        {
+            fx_ver_t best_match_release_only = search_for_best_framework_match(
+                version_list,
+                fx_ref,
+                /*release_only*/ true);
+
+            if (best_match_release_only != fx_ver_t())
+            {
+                return best_match_release_only;
+            }
+        }
+
+        // If release-only didn't find anything, or the framework reference has no preference to release,
+        // do a full search on all versions.
+        fx_ver_t best_match = search_for_best_framework_match(
+            version_list,
+            fx_ref,
+            /*release_only*/ false);
+
+        if (best_match == fx_ver_t())
+        {
+            // This is not strictly necessary, we just need to return version which doesn't exist.
+            // But it's cleaner to return the desider reference then invalid -1.-1.-1 version.
+            best_match = fx_ref.get_fx_version_number();
+            trace::verbose(_X("Framework reference didn't resolve to any available version."));
+        }
+        else
+        {
+            trace::verbose(_X("Framework reference resolved to version '%s'."), best_match.as_str().c_str());
+        }
+
+        return best_match;
     }
 
-    fx_definition_t* resolve_fx(
+    fx_definition_t* resolve_framework_reference(
         const fx_reference_t & fx_ref,
         const pal::string_t & oldest_requested_version,
         const pal::string_t & dotnet_dir)
     {
+#if DEBUG
         assert(!fx_ref.get_fx_name().empty());
         assert(!fx_ref.get_fx_version().empty());
-        assert(fx_ref.get_patch_roll_fwd() != nullptr);
-        assert(fx_ref.get_roll_fwd_on_no_candidate_fx() != nullptr);
+
+        fx_ver_t _debug_ver;
+        assert(fx_ver_t::parse(fx_ref.get_fx_version(), &_debug_ver, false));
+        assert(_debug_ver == fx_ref.get_fx_version_number());
+#endif // DEBUG
 
         trace::verbose(_X("--- Resolving FX directory, name '%s' version '%s'"),
             fx_ref.get_fx_name().c_str(), fx_ref.get_fx_version().c_str());
 
-        const auto fx_ver = fx_ref.get_fx_version();
-        fx_ver_t specified;
-        if (!fx_ver_t::parse(fx_ver, &specified, false))
-        {
-            trace::error(_X("The specified framework version '%s' could not be parsed"), fx_ver.c_str());
-            return nullptr;
-        }
-
-        // Multi-level SharedFX lookup will look for the most appropriate version in several locations
-        // by following the priority rank below:
-        // .exe directory
-        //  Global .NET directory
-        // If it is not activated, then only .exe directory will be considered
-
         std::vector<pal::string_t> hive_dir;
         get_framework_and_sdk_locations(dotnet_dir, &hive_dir);
 
@@ -161,33 +217,26 @@ namespace
             append_path(&fx_dir, _X("shared"));
             append_path(&fx_dir, fx_ref.get_fx_name().c_str());
 
-            bool do_roll_forward = false;
-            if (!fx_ref.get_use_exact_version())
+            // Roll forward is disabled when:
+            //   roll_forward is set to Disable
+            //   roll_forward is set to LatestPatch AND
+            //     apply_patches is false AND
+            //     release framework reference (this is for backward compat with pre-release rolling over pre-release portion of version ignoring apply_patches)
+            //   use exact version is set (this is when --fx-version was used on the command line)
+            if ((fx_ref.get_roll_forward() == roll_forward_option::Disable) ||
+                ((fx_ref.get_roll_forward() == roll_forward_option::LatestPatch) && (!fx_ref.get_apply_patches() && !fx_ref.get_fx_version_number().is_prerelease())))
             {
-                if (!specified.is_prerelease())
-                {
-                    // If production and no roll forward use given version.
-                    do_roll_forward = (*(fx_ref.get_patch_roll_fwd())) || (*(fx_ref.get_roll_fwd_on_no_candidate_fx()) != roll_fwd_on_no_candidate_fx_option::disabled);
-                }
-                else
-                {
-                    // Prerelease, but roll forward only if version doesn't exist.
-                    pal::string_t ver_dir = fx_dir;
-                    append_path(&ver_dir, fx_ver.c_str());
-                    do_roll_forward = !pal::directory_exists(ver_dir);
-                }
-            }
-
-            if (!do_roll_forward)
-            {
-                trace::verbose(_X("Did not roll forward because patch_roll_fwd=%d, roll_fwd_on_no_candidate_fx=%d, use_exact_version=%d chose [%s]"),
-                    *(fx_ref.get_patch_roll_fwd()), *(fx_ref.get_roll_fwd_on_no_candidate_fx()), fx_ref.get_use_exact_version(), fx_ver.c_str());
+                trace::verbose(
+                    _X("Did not roll forward because apply_patches=%d, roll_forward=%s chose [%s]"),
+                    fx_ref.get_apply_patches(),
+                    roll_forward_option_to_string(fx_ref.get_roll_forward()).c_str(),
+                    fx_ref.get_fx_version().c_str());
 
-                append_path(&fx_dir, fx_ver.c_str());
+                append_path(&fx_dir, fx_ref.get_fx_version().c_str());
                 if (pal::directory_exists(fx_dir))
                 {
                     selected_fx_dir = fx_dir;
-                    selected_fx_version = fx_ver;
+                    selected_fx_version = fx_ref.get_fx_version();
                     break;
                 }
             }
@@ -206,7 +255,7 @@ namespace
                     }
                 }
 
-                fx_ver_t resolved_ver = resolve_framework_version(version_list, fx_ver, specified, *(fx_ref.get_patch_roll_fwd()), *(fx_ref.get_roll_fwd_on_no_candidate_fx()));
+                fx_ver_t resolved_ver = resolve_framework_reference_from_version_list(version_list, fx_ref);
 
                 pal::string_t resolved_ver_str = resolved_ver.as_str();
                 append_path(&fx_dir, resolved_ver_str.c_str());
@@ -219,7 +268,7 @@ namespace
                         std::vector<fx_ver_t> version_list;
                         version_list.push_back(resolved_ver);
                         version_list.push_back(selected_ver);
-                        resolved_ver = resolve_framework_version(version_list, fx_ver, specified, *(fx_ref.get_patch_roll_fwd()), *(fx_ref.get_roll_fwd_on_no_candidate_fx()));
+                        resolved_ver = resolve_framework_reference_from_version_list(version_list, fx_ref);
                     }
 
                     if (resolved_ver != selected_ver)
@@ -245,72 +294,73 @@ namespace
     }
 }
 
-StatusCode fx_resolver_t::soft_roll_forward_helper(
-    const fx_reference_t & higher_fx_ref,
-    const fx_reference_t & lower_fx_ref,
-    bool newest_is_hard_roll_forward)
+StatusCode fx_resolver_t::reconcile_fx_references_helper(
+    const fx_reference_t& lower_fx_ref,
+    const fx_reference_t& higher_fx_ref,
+    /*out*/ fx_reference_t& effective_fx_ref)
 {
-    const pal::string_t& fx_name = higher_fx_ref.get_fx_name();
-    fx_reference_t updated_newest = higher_fx_ref;
-
-    if (lower_fx_ref.get_fx_version_number() == higher_fx_ref.get_fx_version_number())
+    if (!lower_fx_ref.is_compatible_with_higher_version(higher_fx_ref.get_fx_version_number()))
     {
-        updated_newest.merge_roll_forward_settings_from(lower_fx_ref);
-        m_newest_references[fx_name] = updated_newest;
-        return StatusCode::Success;
+        // Error condition - not compatible with the other reference
+        display_incompatible_framework_error(higher_fx_ref.get_fx_version(), lower_fx_ref);
+        return StatusCode::FrameworkCompatFailure;
     }
 
-    if (lower_fx_ref.is_roll_forward_compatible(higher_fx_ref.get_fx_version_number()))
-    {
-        updated_newest.merge_roll_forward_settings_from(lower_fx_ref);
-        m_newest_references[fx_name] = updated_newest;
+    effective_fx_ref = fx_reference_t(higher_fx_ref); // copy
+    effective_fx_ref.merge_roll_forward_settings_from(lower_fx_ref);
 
-        if (newest_is_hard_roll_forward)
-        {
-            display_retry_framework_trace(lower_fx_ref, higher_fx_ref);
-            return StatusCode::FrameworkCompatRetry;
-        }
+    display_compatible_framework_trace(higher_fx_ref.get_fx_version(), lower_fx_ref);
+    return StatusCode::Success;
+}
 
-        display_compatible_framework_trace(higher_fx_ref.get_fx_version(), lower_fx_ref);
-        return StatusCode::Success;
+// Reconciles two framework references into a new effective framework reference
+// This process is sometimes also called "soft roll forward" (soft as in no IO)
+// - fx_ref_a - one of the framework references to reconcile
+// - fx_ref_b - the other framework reference to reconcile
+// - effective_fx_ref - the resulting effective framework reference
+//
+// The function will
+//   - Validate that the two references are compatible, of not it returns appropriate error code
+//   - Pick the higher version from the two references and use that in the effective reference
+//   - Merge roll forward settings and use the result in the effective reference
+StatusCode fx_resolver_t::reconcile_fx_references(
+    const fx_reference_t& fx_ref_a,
+    const fx_reference_t& fx_ref_b,
+    /*out*/ fx_reference_t& effective_fx_ref)
+{
+    // The function is split into the helper because the various tracing messages
+    // make more sense if they're always written with higher/lower versions ordered in particular way.
+    if (fx_ref_a.get_fx_version_number() >= fx_ref_b.get_fx_version_number())
+    {
+        return reconcile_fx_references_helper(fx_ref_b, fx_ref_a, effective_fx_ref);
+    }
+    else
+    {
+        return reconcile_fx_references_helper(fx_ref_a, fx_ref_b, effective_fx_ref);
     }
-
-    // Error condition - not compatible with the other reference
-    display_incompatible_framework_error(higher_fx_ref.get_fx_version(), lower_fx_ref);
-    return StatusCode::FrameworkCompatFailure;
 }
 
-// Performs a "soft" roll-forward meaning we don't read any physical framework folders
-// and just check if the lower_fx_ref reference is compatible with the higher_fx_ref reference
-// with respect to roll-forward/apply-patches.
-//  - fx_ref
-//      The reference to resolve (the one we're processing).
-//      Passed by-value to avoid side-effects with mutable newest_references and oldest_references.
-//  - newest_is_hard_roll_forward
-//      true if there is a reference to the framework specified by fx_ref in the newest_references
-//      and that reference is pointing to a physically resolved framework on the disk (so the version in it
-//      actually exists on disk).
-//      This is used to restart the framework resolution if the soft-roll-forward results in updating
-//      the m_newest_references for the processed framework with a higher version. In that case
-//      it is necessary to throw away the results of the previous disk resolution and resolve the new
-//      current reference against the frameworks available on disk.
-//      If the current reference on the other hand has not been resolved against the disk yet
-//      then we can safely move it forward to a higher version. It will be resolved against the disk eventually
-//      but there's no work to throw away/retry yet.
-StatusCode fx_resolver_t::soft_roll_forward(
-    const fx_reference_t fx_ref,
-    bool newest_is_hard_roll_forward)
+void fx_resolver_t::update_newest_references(
+    const runtime_config_t& config)
 {
-    /*byval*/ fx_reference_t current_ref = m_newest_references[fx_ref.get_fx_name()];
-
-    // Perform soft "in-memory" roll-forwards
-    if (fx_ref.get_fx_version_number() >= current_ref.get_fx_version_number())
+    // Loop through each reference and update the list of newest references before we resolve_framework_reference.
+    for (const fx_reference_t& fx_ref : config.get_frameworks())
     {
-        return soft_roll_forward_helper(fx_ref, current_ref, newest_is_hard_roll_forward);
+        const pal::string_t& fx_name = fx_ref.get_fx_name();
+        auto temp_ref = m_effective_fx_references.find(fx_name);
+        if (temp_ref == m_effective_fx_references.end())
+        {
+            m_effective_fx_references.insert({ fx_name, fx_ref });
+            m_oldest_fx_references.insert({ fx_name, fx_ref });
+        }
+        else
+        {
+            if (fx_ref.get_fx_version_number() < m_oldest_fx_references[fx_name].get_fx_version_number())
+            {
+                m_oldest_fx_references[fx_name] = fx_ref;
+            }
+        }
     }
-
-    assert(fx_ref.get_fx_version_number() < current_ref.get_fx_version_number());
-    return soft_roll_forward_helper(current_ref, fx_ref, false);
 }
 
 // Processes one framework's runtime configuration.
@@ -338,29 +388,12 @@ StatusCode fx_resolver_t::soft_roll_forward(
 //     InvalidConfigFile - reading of a runtime config for some of the processed frameworks has failed.
 StatusCode fx_resolver_t::read_framework(
     const host_startup_info_t & host_info,
-    const fx_reference_t & override_settings,
+    const runtime_config_t::settings_t& override_settings,
     const runtime_config_t & config,
     fx_definition_vector_t & fx_definitions)
 {
-    // Loop through each reference and update the list of newest references before we resolve_fx.
     // This reconciles duplicate references to minimize the number of resolve retries.
-    for (const fx_reference_t& fx_ref : config.get_frameworks())
-    {
-        const pal::string_t& fx_name = fx_ref.get_fx_name();
-        auto temp_ref = m_newest_references.find(fx_name);
-        if (temp_ref == m_newest_references.end())
-        {
-            m_newest_references.insert({ fx_name, fx_ref });
-            m_oldest_references.insert({ fx_name, fx_ref });
-        }
-        else
-        {
-            if (fx_ref.get_fx_version_number() < m_oldest_references[fx_name].get_fx_version_number())
-            {
-                m_oldest_references[fx_name] = fx_ref;
-            }
-        }
-    }
+    update_newest_references(config);
 
     StatusCode rc = StatusCode::Success;
 
@@ -368,6 +401,8 @@ StatusCode fx_resolver_t::read_framework(
     for (const fx_reference_t& fx_ref : config.get_frameworks())
     {
         const pal::string_t& fx_name = fx_ref.get_fx_name();
+        const fx_reference_t& current_effective_fx_ref = m_effective_fx_references[fx_name];
+        fx_reference_t new_effective_fx_ref;
 
         auto existing_framework = std::find_if(
             fx_definitions.begin(),
@@ -376,27 +411,35 @@ StatusCode fx_resolver_t::read_framework(
 
         if (existing_framework == fx_definitions.end())
         {
-            // Perform a "soft" roll-forward meaning we don't read any physical framework folders yet
-            // Since we didn't find the framework in the resolved list yet, it's a pure soft roll-forward
-            // it's OK to update the newest reference as we haven't processed it yet.
-            rc = soft_roll_forward(fx_ref, /*newest_is_hard_roll_forward*/ false);
+            // Reconcile the framework reference with the most up to date so far we have for the framework.
+            // This does not read any physical framework folders yet.
+            // Since we didn't find the framework in the resolved list yet, it's OK to update the effective reference 
+            // as we haven't processed it yet.
+            rc = reconcile_fx_references(fx_ref, current_effective_fx_ref, new_effective_fx_ref);
             if (rc)
             {
                 break; // Error case
             }
 
-            fx_reference_t& newest_ref = m_newest_references[fx_name];
+            m_effective_fx_references[fx_name] = new_effective_fx_ref;
 
-            // Resolve the framwork against the the existing physical framework folders
-            fx_definition_t* fx = resolve_fx(newest_ref, m_oldest_references[fx_name].get_fx_version(), host_info.dotnet_root);
+            // Resolve the effective framework reference against the the existing physical framework folders
+            fx_definition_t* fx = resolve_framework_reference(new_effective_fx_ref, m_oldest_fx_references[fx_name].get_fx_version(), host_info.dotnet_root);
             if (fx == nullptr)
             {
-                display_missing_framework_error(fx_name, newest_ref.get_fx_version(), pal::string_t(), host_info.dotnet_root);
+                display_missing_framework_error(fx_name, new_effective_fx_ref.get_fx_version(), pal::string_t(), host_info.dotnet_root);
                 return FrameworkMissingFailure;
             }
 
-            // Update the newest version based on the hard version found
-            newest_ref.set_fx_version(fx->get_found_version());
+            // Do NOT update the effective reference to have the same version as the resolved framework.
+            // This could prevent correct resolution in some cases.
+            // For example if the resolution starts with reference "2.1.0 LatestMajor" the resolution could
+            // return "3.0.0". If later on we find another reference "2.1.0 Minor", while the two references are compatible
+            // we would not be able to resolve it, since we would compare "2.1.0 Minor" with "3.0.0 LatestMajor" which are
+            // not compatible.
+            // So instead leave the effective reference as is. If the above situation occurs, the reference reconciliation
+            // will change the effective reference from "2.1.0 LatestMajor" to "2.1.0 Minor" and restart the framework resolution process.
+            // So during the second run we will resolve for example "2.2.0" which will be compatible with both framework references.
 
             fx_definitions.push_back(std::unique_ptr<fx_definition_t>(fx));
 
@@ -404,7 +447,7 @@ StatusCode fx_resolver_t::read_framework(
             pal::string_t config_file;
             pal::string_t dev_config_file;
             get_runtime_config_paths(fx->get_dir(), fx_name, &config_file, &dev_config_file);
-            fx->parse_runtime_config(config_file, dev_config_file, newest_ref, override_settings);
+            fx->parse_runtime_config(config_file, dev_config_file, override_settings);
 
             runtime_config_t new_config = fx->get_runtime_config();
             if (!new_config.is_valid())
@@ -421,14 +464,22 @@ StatusCode fx_resolver_t::read_framework(
         }
         else
         {
-            // Perform a "soft" roll-forward meaning we don't read any physical framework folders yet
+            // Reconcile the framework reference with the most up to date so far we have for the framework.
             // Note that since we found the framework in the already resolved frameworks
-            // pass a flag which marks the newest resolved framework reference as "hard roll-forward"
-            // meaning that if we need to update it, we need to restart the entire process.
-            rc = soft_roll_forward(fx_ref, /*newest_is_hard_roll_forward*/ true);
+            // any update to the effective framework reference needs to restart the resolution process
+            // so that we re-resolve the framework against disk.
+            rc = reconcile_fx_references(fx_ref, current_effective_fx_ref, new_effective_fx_ref);
             if (rc)
             {
-                break; // Error or retry case
+                break; // Error case
+            }
+
+            if (new_effective_fx_ref != current_effective_fx_ref)
+            {
+                display_retry_framework_trace(current_effective_fx_ref, fx_ref);
+
+                m_effective_fx_references[fx_name] = new_effective_fx_ref;
+                return StatusCode::FrameworkCompatRetry;
             }
 
             // Success but move it to the back (without calling dtors) so that lower-level frameworks come last including Microsoft.NetCore.App
@@ -445,7 +496,7 @@ fx_resolver_t::fx_resolver_t()
 
 StatusCode fx_resolver_t::resolve_frameworks_for_app(
     const host_startup_info_t & host_info,
-    const fx_reference_t & override_settings,
+    const runtime_config_t::settings_t& override_settings,
     const runtime_config_t & app_config,
     fx_definition_vector_t & fx_definitions)
 {
@@ -464,7 +515,7 @@ StatusCode fx_resolver_t::resolve_frameworks_for_app(
 
     if (rc == StatusCode::Success)
     {
-        display_summary_of_frameworks(fx_definitions, resolver.m_newest_references);
+        display_summary_of_frameworks(fx_definitions, resolver.m_effective_fx_references);
     }
 
     return rc;
index 6b2f92b..01b9c75 100644 (file)
@@ -17,26 +17,30 @@ class fx_resolver_t
 public:
     static StatusCode resolve_frameworks_for_app(
         const host_startup_info_t& host_info,
-        const fx_reference_t& override_settings,
+        const runtime_config_t::settings_t& override_settings,
         const runtime_config_t& app_config,
         fx_definition_vector_t& fx_definitions);
 
 private:
     fx_resolver_t();
 
-    StatusCode soft_roll_forward_helper(
-        const fx_reference_t& newer,
-        const fx_reference_t& older,
-        bool older_is_hard_roll_forward);
-    StatusCode soft_roll_forward(
-        const fx_reference_t existing_ref,
-        bool current_is_hard_roll_forward);
+    void update_newest_references(
+        const runtime_config_t& config);
     StatusCode read_framework(
         const host_startup_info_t& host_info,
-        const fx_reference_t& override_settings,
+        const runtime_config_t::settings_t& override_settings,
         const runtime_config_t& config,
         fx_definition_vector_t& fx_definitions);
 
+    static StatusCode reconcile_fx_references_helper(
+        const fx_reference_t& lower_fx_ref,
+        const fx_reference_t& higher_fx_ref,
+        /*out*/ fx_reference_t& effective_fx_ref);
+    static StatusCode reconcile_fx_references(
+        const fx_reference_t& fx_ref_a,
+        const fx_reference_t& fx_ref_b,
+        /*out*/ fx_reference_t& effective_fx_ref);
+
     static void display_missing_framework_error(
         const pal::string_t& fx_name,
         const pal::string_t& fx_version,
@@ -55,16 +59,16 @@ private:
         const fx_definition_vector_t& fx_definitions,
         const fx_name_to_fx_reference_map_t& newest_references);
 
-    // Map of FX Name -> FX Reference of the most up-to-date resolved references so far. This map is keeping the state
+    // Map of FX Name -> FX Reference of the most up-to-date effective references so far. This map is keeping the state
     // of the resolution algorithm. For each framework it holds the highest version referenced and the merged
-    // roll-forward settings. If the reference has been resolved against the frameworks on disk, this map will hold
-    // the version of the actually resolved version on the disk (so called hard-roll-forward).
-    fx_name_to_fx_reference_map_t m_newest_references;
+    // roll-forward settings. If the reference has been resolved against the frameworks on disk, this map will still hold
+    // the effective reference used to resolve the framework, not the actual found version.
+    fx_name_to_fx_reference_map_t m_effective_fx_references;
 
     // Map of FX Name -> FX Reference of the oldest reference found for the framework yet. This map is only used
     // to fill the "oldest reference" for each resolved framework in the end. It does not affect the behavior
     // of the algorithm.
-    fx_name_to_fx_reference_map_t m_oldest_references;
+    fx_name_to_fx_reference_map_t m_oldest_fx_references;
 };
 
 #endif // __FX_RESOLVER_H__
\ No newline at end of file
index d2136ca..1fae140 100644 (file)
@@ -13,14 +13,11 @@ void fx_resolver_t::display_incompatible_framework_error(
     const pal::string_t& higher,
     const fx_reference_t& lower)
 {
-    assert(lower.get_patch_roll_fwd() != nullptr);
-    assert(lower.get_roll_fwd_on_no_candidate_fx() != nullptr);
-
-    trace::error(_X("The specified framework '%s', version '%s', patch_roll_fwd=%d, roll_fwd_on_no_candidate_fx=%d cannot roll-forward to the previously referenced version '%s'."),
+    trace::error(_X("The specified framework '%s', version '%s', apply_patches=%d, roll_forward=%s cannot roll-forward to the previously referenced version '%s'."),
         lower.get_fx_name().c_str(),
         lower.get_fx_version().c_str(),
-        *lower.get_patch_roll_fwd(),
-        *lower.get_roll_fwd_on_no_candidate_fx(),
+        lower.get_apply_patches(),
+        roll_forward_option_to_string(lower.get_roll_forward()).c_str(),
         higher.c_str());
 }
 
@@ -30,14 +27,11 @@ void fx_resolver_t::display_compatible_framework_trace(
 {
     if (trace::is_enabled())
     {
-        assert(lower.get_patch_roll_fwd() != nullptr);
-        assert(lower.get_roll_fwd_on_no_candidate_fx() != nullptr);
-
-        trace::verbose(_X("--- The specified framework '%s', version '%s', patch_roll_fwd=%d, roll_fwd_on_no_candidate_fx=%d is compatible with the previously referenced version '%s'."),
+        trace::verbose(_X("--- The specified framework '%s', version '%s', apply_patches=%d, roll_forward=%s is compatible with the previously referenced version '%s'."),
             lower.get_fx_name().c_str(),
             lower.get_fx_version().c_str(),
-            *lower.get_patch_roll_fwd(),
-            *lower.get_roll_fwd_on_no_candidate_fx(),
+            lower.get_apply_patches(),
+            roll_forward_option_to_string(lower.get_roll_forward()).c_str(),
             higher.c_str());
     }
 }
@@ -48,15 +42,12 @@ void fx_resolver_t::display_retry_framework_trace(
 {
     if (trace::is_enabled())
     {
-        assert(fx_new.get_patch_roll_fwd() != nullptr);
-        assert(fx_new.get_roll_fwd_on_no_candidate_fx() != nullptr);
-
-        trace::verbose(_X("--- Restarting all framework resolution because the previously resolved framework '%s', version '%s' must be re-resolved with the new version '%s', patch_roll_fwd=%d, roll_fwd_on_no_candidate_fx=%d ."),
+        trace::verbose(_X("--- Restarting all framework resolution because the previously resolved framework '%s', version '%s' must be re-resolved with the new version '%s', apply_patches=%d, roll_forward=%s ."),
             fx_existing.get_fx_name().c_str(),
             fx_existing.get_fx_version().c_str(),
             fx_new.get_fx_version().c_str(),
-            *fx_new.get_patch_roll_fwd(),
-            *fx_new.get_roll_fwd_on_no_candidate_fx());
+            fx_new.get_apply_patches(),
+            roll_forward_option_to_string(fx_new.get_roll_forward()).c_str());
     }
 }
 
@@ -79,16 +70,14 @@ void fx_resolver_t::display_summary_of_frameworks(
             {
                 auto newest_ref = newest_references.find(fx->get_name());
                 assert(newest_ref != newest_references.end());
-                assert(newest_ref->second.get_fx_version() == fx->get_found_version());
-                assert(newest_ref->second.get_patch_roll_fwd() != nullptr);
-                assert(newest_ref->second.get_roll_fwd_on_no_candidate_fx() != nullptr);
 
-                trace::verbose(_X("     framework:'%s', lowest requested version='%s', found version='%s', patch_roll_fwd=%d, roll_fwd_on_no_candidate_fx=%d, folder=%s"),
+                trace::verbose(_X("     framework:'%s', lowest requested version='%s', found version='%s', effective reference version='%s' apply_patches=%d, roll_forward=%s, folder=%s"),
                     fx->get_name().c_str(),
                     fx->get_requested_version().c_str(),
                     fx->get_found_version().c_str(),
-                    *newest_ref->second.get_patch_roll_fwd(),
-                    *newest_ref->second.get_roll_fwd_on_no_candidate_fx(),
+                    newest_ref->second.get_fx_version().c_str(),
+                    newest_ref->second.get_apply_patches(),
+                    roll_forward_option_to_string(newest_ref->second.get_roll_forward()).c_str(),
                     fx->get_dir().c_str());
             }
         }
index 88e61f0..6baf0a3 100644 (file)
@@ -20,6 +20,7 @@ set(SOURCES
     ./hostpolicy.cpp
     ./hostpolicy_context.cpp
     ./hostpolicy_init.cpp
+    ../roll_forward_option.cpp
     ../runtime_config.cpp
     ../json/casablanca/src/json/json.cpp
     ../json/casablanca/src/json/json_parsing.cpp
index 5aa0d7e..ceed011 100644 (file)
@@ -505,7 +505,7 @@ SHARED_API int corehost_resolve_component_dependencies(
     // Call parse_runtime_config since it initializes the defaults for various settings
     // but we don't have any .runtimeconfig.json for the component, so pass in empty paths.
     // Empty paths is a valid case and the method will simply skip parsing anything.
-    app->parse_runtime_config(pal::string_t(), pal::string_t(), fx_reference_t(), fx_reference_t());
+    app->parse_runtime_config(pal::string_t(), pal::string_t(), runtime_config_t::settings_t());
     if (!app->get_runtime_config().is_valid())
     {
         // This should really never happen, but fail gracefully if it does anyway.
diff --git a/src/installer/corehost/cli/roll_forward_option.cpp b/src/installer/corehost/cli/roll_forward_option.cpp
new file mode 100644 (file)
index 0000000..e774ed1
--- /dev/null
@@ -0,0 +1,62 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+// See the LICENSE file in the project root for more information.
+
+#include "pal.h"
+#include "trace.h"
+#include "roll_forward_option.h"
+#include "roll_fwd_on_no_candidate_fx_option.h"
+
+roll_forward_option roll_fwd_on_no_candidate_fx_to_roll_forward(roll_fwd_on_no_candidate_fx_option roll_fwd_on_no_candidate_fx)
+{
+    switch (roll_fwd_on_no_candidate_fx)
+    {
+    case roll_fwd_on_no_candidate_fx_option::disabled:
+        return roll_forward_option::LatestPatch;
+    case roll_fwd_on_no_candidate_fx_option::minor:
+        return roll_forward_option::Minor;
+    case roll_fwd_on_no_candidate_fx_option::major:
+        return roll_forward_option::Major;
+    default:
+        assert(false);
+        return roll_forward_option::Disable;
+    }
+}
+
+namespace
+{
+    const pal::char_t* OptionNameMapping[] =
+    {
+        _X("Disable"),
+        _X("LatestPatch"),
+        _X("Minor"),
+        _X("LatestMinor"),
+        _X("Major"),
+        _X("LatestMajor")
+    };
+
+    static_assert((sizeof(OptionNameMapping) / sizeof(*OptionNameMapping)) == static_cast<size_t>(roll_forward_option::__Last), "Invalid option count");
+}
+
+pal::string_t roll_forward_option_to_string(roll_forward_option roll_forward)
+{
+    int idx = static_cast<int>(roll_forward);
+    assert(0 <= idx && idx < static_cast<int>(roll_forward_option::__Last));
+
+    return OptionNameMapping[idx];
+}
+
+roll_forward_option roll_forward_option_from_string(const pal::string_t& value)
+{
+    for (int idx = 0; idx < static_cast<int>(roll_forward_option::__Last); idx++)
+    {
+        if (pal::strcasecmp(OptionNameMapping[idx], value.c_str()) == 0)
+        {
+            return static_cast<roll_forward_option>(idx);
+        }
+    }
+
+    trace::error(_X("Unrecognized roll forward setting value '%s'."), value.c_str());
+    return roll_forward_option::__Last;
+}
+
diff --git a/src/installer/corehost/cli/roll_forward_option.h b/src/installer/corehost/cli/roll_forward_option.h
new file mode 100644 (file)
index 0000000..e7de0ed
--- /dev/null
@@ -0,0 +1,32 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+// See the LICENSE file in the project root for more information.
+
+#ifndef __ROLL_FORWARD_OPTION_H_
+#define __ROLL_FORWARD_OPTION_H_
+
+// Specifies the roll forward option value
+// High-level notes on roll forward algorithm
+//  - Try to use the version which the app asked for. If not possible try to use the closest higher version (unless modified via settings)
+//  - Always pick the latest patch for servicing/security
+//  - Allow customization of the behavior via rollForward setting
+//  - Backward compatible with deprecated settings rollForwardOnNoCandidateFx and applyPatches
+enum class roll_forward_option
+{
+    // The order is in increasing level of relaxation
+    // Lower values are more restrictive than higher values
+
+    Disable = 0,     // No roll-forward is allowed - only exact match
+    LatestPatch = 1, // Roll forward to latest patch.
+    Minor = 2,       // Roll forward to closest minor but same major and then highest patch
+    LatestMinor = 3, // Roll forward to highest minor.patch but same major
+    Major = 4,       // Roll forward to closest major.minor and then highest patch
+    LatestMajor = 5, // Roll forward to highest major.minor.patch
+
+    __Last           // Sentinel value
+};
+
+pal::string_t roll_forward_option_to_string(roll_forward_option roll_forward);
+roll_forward_option roll_forward_option_from_string(const pal::string_t& value);
+
+#endif // __ROLL_FORWARD_OPTION_H_
index d282fe8..b9afcdb 100644 (file)
@@ -14,4 +14,7 @@ enum class roll_fwd_on_no_candidate_fx_option
     major           // also inludes minor and patch
 };
 
+enum class roll_forward_option;
+roll_forward_option roll_fwd_on_no_candidate_fx_to_roll_forward(roll_fwd_on_no_candidate_fx_option roll_fwd_on_no_candidate_fx);
+
 #endif // __ROLL_FWD_ON_NO_CANDIDATE_FX_OPTION_H_
index ed79ac0..dd08b0d 100644 (file)
@@ -7,40 +7,61 @@
 #include "utils.h"
 #include "cpprest/json.h"
 #include "runtime_config.h"
+#include "roll_fwd_on_no_candidate_fx_option.h"
 #include <cassert>
 
-
 // The semantics of applying the runtimeconfig.json values follows, in the following steps from
 // first to last, where last always wins. These steps are also annotated in the code here.
-// 1) Apply the environment settings
+// 0) Start with the default values
+// 1) Apply the environment settings for DOTNET_ROLL_FORWARD_ON_NO_CANDIDATE_FX
 // 2) Apply the values in the current "runtimeOptions" section
 // 3) Apply the values in the referenced "frameworks" section
-// 4) Apply the overrides (from command line or other)
+// 4) Apply the environment settings for DOTNET_ROLL_FORWARD
+// 5) Apply the overrides (from command line or other)
 
 runtime_config_t::runtime_config_t()
-    : m_is_framework_dependent(false)
+    : m_default_settings()
+    , m_override_settings()
+    , m_specified_settings(none)
+    , m_is_framework_dependent(false)
     , m_valid(false)
+    , m_roll_forward_to_prerelease(false)
+{
+    pal::string_t roll_forward_to_prerelease_env;
+    if (pal::getenv(_X("DOTNET_ROLL_FORWARD_TO_PRERELEASE"), &roll_forward_to_prerelease_env))
+    {
+        auto roll_forward_to_prerelease_val = pal::xtoi(roll_forward_to_prerelease_env.c_str());
+        m_roll_forward_to_prerelease = (roll_forward_to_prerelease_val == 1);
+    }
+}
+
+runtime_config_t::settings_t::settings_t()
+    : has_apply_patches(false)
+    , apply_patches(true)
+    , has_roll_forward(false)
+    , roll_forward(roll_forward_option::Minor)
 {
 }
 
-void runtime_config_t::parse(const pal::string_t& path, const pal::string_t& dev_path, const fx_reference_t& fx_ref, const fx_reference_t& override_settings)
+void runtime_config_t::parse(const pal::string_t& path, const pal::string_t& dev_path, const settings_t& override_settings)
 {
     m_path = path;
     m_dev_path = dev_path;
-    m_fx_ref = fx_ref;
-    m_fx_overrides = override_settings;
+    m_override_settings = override_settings;
 
-    // Step #1: set the defaults from the environment
-    m_fx_defaults.set_patch_roll_fwd(true);
+    // Step #0: start with the default values
+    m_default_settings.set_apply_patches(true);
+    roll_forward_option roll_forward = roll_forward_option::Minor;
 
-    roll_fwd_on_no_candidate_fx_option roll_fwd_option = roll_fwd_on_no_candidate_fx_option::minor;
-    pal::string_t env_no_candidate;
-    if (pal::getenv(_X("DOTNET_ROLL_FORWARD_ON_NO_CANDIDATE_FX"), &env_no_candidate))
+    // Step #1: set the defaults from the environment DOTNET_ROLL_FORWARD_ON_NO_CANDIDATE_FX (apply patches has no env. variable)
+    pal::string_t env_roll_forward_on_no_candidate_fx;
+    if (pal::getenv(_X("DOTNET_ROLL_FORWARD_ON_NO_CANDIDATE_FX"), &env_roll_forward_on_no_candidate_fx))
     {
-        roll_fwd_option = static_cast<roll_fwd_on_no_candidate_fx_option>(pal::xtoi(env_no_candidate.c_str()));
+        auto val = static_cast<roll_fwd_on_no_candidate_fx_option>(pal::xtoi(env_roll_forward_on_no_candidate_fx.c_str()));
+        roll_forward = roll_fwd_on_no_candidate_fx_to_roll_forward(val);
     }
 
-    m_fx_defaults.set_roll_fwd_on_no_candidate_fx(roll_fwd_option);
+    m_default_settings.set_roll_forward(roll_forward);
 
     // Parse the file
     m_valid = ensure_parsed();
@@ -89,17 +110,42 @@ bool runtime_config_t::parse_opts(const json_value& opts)
     }
 
     // Step #2: set the defaults from the "runtimeOptions"
-    auto patch_roll_fwd = opts_obj.find(_X("applyPatches"));
-    if (patch_roll_fwd != opts_obj.end())
+    auto roll_forward = opts_obj.find(_X("rollForward"));
+    if (roll_forward != opts_obj.end())
     {
-        m_fx_defaults.set_patch_roll_fwd(patch_roll_fwd->second.as_bool());
+        auto val = roll_forward_option_from_string(roll_forward->second.as_string());
+        if (val == roll_forward_option::__Last)
+        {
+            trace::error(_X("Invalid value for property 'rollForward'."));
+            return false;
+        }
+        m_default_settings.set_roll_forward(val);
+
+        if (!mark_specified_setting(specified_roll_forward))
+        {
+            return false;
+        }
+    }
+
+    auto apply_patches = opts_obj.find(_X("applyPatches"));
+    if (apply_patches != opts_obj.end())
+    {
+        m_default_settings.set_apply_patches(apply_patches->second.as_bool());
+        if (!mark_specified_setting(specified_roll_forward_on_no_candidate_fx_or_apply_patched))
+        {
+            return false;
+        }
     }
 
     auto roll_fwd_on_no_candidate_fx = opts_obj.find(_X("rollForwardOnNoCandidateFx"));
     if (roll_fwd_on_no_candidate_fx != opts_obj.end())
     {
         auto val = static_cast<roll_fwd_on_no_candidate_fx_option>(roll_fwd_on_no_candidate_fx->second.as_integer());
-        m_fx_defaults.set_roll_fwd_on_no_candidate_fx(val);
+        m_default_settings.set_roll_forward(roll_fwd_on_no_candidate_fx_to_roll_forward(val));
+        if (!mark_specified_setting(specified_roll_forward_on_no_candidate_fx_or_apply_patched))
+        {
+            return false;
+        }
     }
 
     auto tfm = opts_obj.find(_X("tfm"));
@@ -140,9 +186,25 @@ bool runtime_config_t::parse_opts(const json_value& opts)
     return rc;
 }
 
+namespace
+{
+    void apply_settings_to_fx_reference(const runtime_config_t::settings_t& settings, fx_reference_t& fx_ref)
+    {
+        if (settings.has_roll_forward)
+        {
+            fx_ref.set_roll_forward(settings.roll_forward);
+        }
+
+        if (settings.has_apply_patches)
+        {
+            fx_ref.set_apply_patches(settings.apply_patches);
+        }
+    }
+}
+
 bool runtime_config_t::parse_framework(const json_object& fx_obj, fx_reference_t& fx_out)
 {
-    fx_out.apply_settings_from(m_fx_defaults);
+    apply_settings_to_fx_reference(m_default_settings, fx_out);
 
     auto fx_name= fx_obj.find(_X("name"));
     if (fx_name != fx_obj.end())
@@ -154,21 +216,68 @@ bool runtime_config_t::parse_framework(const json_object& fx_obj, fx_reference_t
     if (fx_ver != fx_obj.end())
     {
         fx_out.set_fx_version(fx_ver->second.as_string());
+
+        // Release version should prefer release versions, unless the rollForwardToPrerelease is set
+        // in which case no preference should be applied.
+        if (!fx_out.get_fx_version_number().is_prerelease() && !m_roll_forward_to_prerelease)
+        {
+            fx_out.set_prefer_release(true);
+        }
+    }
+
+    auto roll_forward = fx_obj.find(_X("rollForward"));
+    if (roll_forward != fx_obj.end())
+    {
+        auto val = roll_forward_option_from_string(roll_forward->second.as_string());
+        if (val == roll_forward_option::__Last)
+        {
+            trace::error(_X("Invalid value for property 'rollForward'."));
+            return false;
+        }
+        fx_out.set_roll_forward(val);
+        if (!mark_specified_setting(specified_roll_forward))
+        {
+            return false;
+        }
     }
 
-    auto patch_roll_fwd = fx_obj.find(_X("applyPatches"));
-    if (patch_roll_fwd != fx_obj.end())
+    auto apply_patches = fx_obj.find(_X("applyPatches"));
+    if (apply_patches != fx_obj.end())
     {
-        fx_out.set_patch_roll_fwd(patch_roll_fwd->second.as_bool());
+        fx_out.set_apply_patches(apply_patches->second.as_bool());
+        if (!mark_specified_setting(specified_roll_forward_on_no_candidate_fx_or_apply_patched))
+        {
+            return false;
+        }
     }
 
     auto roll_fwd_on_no_candidate_fx = fx_obj.find(_X("rollForwardOnNoCandidateFx"));
     if (roll_fwd_on_no_candidate_fx != fx_obj.end())
     {
-        fx_out.set_roll_fwd_on_no_candidate_fx(static_cast<roll_fwd_on_no_candidate_fx_option>(roll_fwd_on_no_candidate_fx->second.as_integer()));
+        auto val = static_cast<roll_fwd_on_no_candidate_fx_option>(roll_fwd_on_no_candidate_fx->second.as_integer());
+        fx_out.set_roll_forward(roll_fwd_on_no_candidate_fx_to_roll_forward(val));
+        if (!mark_specified_setting(specified_roll_forward_on_no_candidate_fx_or_apply_patched))
+        {
+            return false;
+        }
+    }
+
+    // Step #4: apply environment for DOTNET_ROLL_FORWARD
+    pal::string_t env_roll_forward;
+    if (pal::getenv(_X("DOTNET_ROLL_FORWARD"), &env_roll_forward))
+    {
+        auto val = roll_forward_option_from_string(env_roll_forward);
+        if (val == roll_forward_option::__Last)
+        {
+            trace::error(_X("Invalid value for environment variable 'DOTNET_ROLL_FORWARD'."));
+            return false;
+        }
+
+        fx_out.set_roll_forward(val);
     }
 
-    fx_out.apply_settings_from(m_fx_overrides);
+    // Step #5: apply overrides (command line and such)
+    apply_settings_to_fx_reference(m_override_settings, fx_out);
 
     return true;
 }
@@ -339,7 +448,19 @@ void runtime_config_t::set_fx_version(pal::string_t version)
     assert(m_frameworks.size() > 0);
 
     m_frameworks[0].set_fx_version(version);
-    m_frameworks[0].set_patch_roll_fwd(false);
-    m_frameworks[0].set_roll_fwd_on_no_candidate_fx(roll_fwd_on_no_candidate_fx_option::disabled);
-    m_frameworks[0].set_use_exact_version(true);
+    m_frameworks[0].set_apply_patches(false);
+    m_frameworks[0].set_roll_forward(roll_forward_option::Disable);
+}
+
+bool runtime_config_t::mark_specified_setting(specified_setting setting)
+{
+    // If there's any flag set but the one we're trying to set, it's invalid
+    if (m_specified_settings & ~setting)
+    {
+        trace::error(_X("It's invalid to use both `rollForward` and one of `rollForwardOnNoCandidateFx` or `applyPatches` in the same runtime config."));
+        return false;
+    }
+
+    m_specified_settings = (specified_setting)(m_specified_settings | setting);
+    return true;
 }
index 10a5cc4..53540f2 100644 (file)
@@ -17,8 +17,22 @@ typedef web::json::object json_object;
 class runtime_config_t
 {
 public:
+    struct settings_t
+    {
+        settings_t();
+
+        bool has_apply_patches;
+        bool apply_patches;
+        void set_apply_patches(bool value) { has_apply_patches = true; apply_patches = value; }
+
+        bool has_roll_forward;
+        roll_forward_option roll_forward;
+        void set_roll_forward(roll_forward_option value) { has_roll_forward = true; roll_forward = value; }
+    };
+
+public:
     runtime_config_t();
-    void parse(const pal::string_t& path, const pal::string_t& dev_path, const fx_reference_t& fx_ref, const fx_reference_t& override_settings);
+    void parse(const pal::string_t& path, const pal::string_t& dev_path, const settings_t& override_settings);
     bool is_valid() const { return m_valid; }
     const pal::string_t& get_path() const { return m_path; }
     const pal::string_t& get_dev_path() const { return m_dev_path; }
@@ -36,23 +50,36 @@ private:
 
     std::unordered_map<pal::string_t, pal::string_t> m_properties;
     fx_reference_vector_t m_frameworks;
-    fx_reference_t m_fx_defaults;   // the default settings (Steps #1 and #2)
-    fx_reference_t m_fx_ref;        // the settings from the referenced "frameworks" section (Step #3)
-    fx_reference_t m_fx_overrides;  // the settings that can't be changed (Step #4)
+    settings_t m_default_settings;   // the default settings (Steps #0 and #1)
+    settings_t m_override_settings;  // the settings that can't be changed (Step #5)
     std::vector<std::string> m_prop_keys;
     std::vector<std::string> m_prop_values;
     std::list<pal::string_t> m_probe_paths;
 
     pal::string_t m_tfm;
 
+    // This is used to detect cases where rollForward is used together with the obsoleted
+    // rollForwardOnNoCandidateFx/applyPatches.
+    // Flags
+    enum specified_setting
+    {
+        none = 0x0,
+        specified_roll_forward = 0x1,
+        specified_roll_forward_on_no_candidate_fx_or_apply_patched = 0x2
+    } m_specified_settings;
+
     pal::string_t m_dev_path;
     pal::string_t m_path;
     bool m_is_framework_dependent;
     bool m_valid;
 
-private:
+    // Cached value of DOTNET_ROLL_FORWARD_TO_PRERELEASE to avoid testing env. variables too often.
+    // If set to true, all versions (including pre-release) are considered even if starting from a release framework reference.
+    bool m_roll_forward_to_prerelease;
+
     bool parse_framework(const json_object& fx_obj, fx_reference_t& fx_out);
     bool read_framework_array(web::json::array frameworks);
-    static void copy_framework_settings_to(const fx_reference_t& from, fx_reference_t& to);
+    
+    bool mark_specified_setting(specified_setting setting);
 };
 #endif // __RUNTIME_CONFIG_H__
index c5ca16e..2a1c5cf 100644 (file)
@@ -342,6 +342,12 @@ void get_framework_and_sdk_locations(const pal::string_t& dotnet_dir, std::vecto
 {
     bool multilevel_lookup = multilevel_lookup_enabled();
 
+    // Multi-level lookup will look for the most appropriate version in several locations
+    // by following the priority rank below:
+    //  .exe directory
+    //  Global .NET directories
+    // If it is not activated, then only .exe directory will be considered
+
     pal::string_t dotnet_dir_temp;
     if (!dotnet_dir.empty())
     {
index c8feff0..5fe7b9a 100644 (file)
@@ -1,6 +1,6 @@
-using System;
-using System.Collections.Generic;
-using System.Text;
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+// See the LICENSE file in the project root for more information.
 
 namespace Microsoft.DotNet.CoreSetup.Test.HostActivation
 {
@@ -18,11 +18,30 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation
             public const string EnvironmentVariable = "DOTNET_ROLL_FORWARD_ON_NO_CANDIDATE_FX";
         }
 
+        public static class RollForwardSetting
+        {
+            public const string RuntimeConfigPropertyName = "rollForward";
+            public const string CommandLineArgument = "--roll-forward";
+            public const string EnvironmentVariable = "DOTNET_ROLL_FORWARD";
+
+            public const string LatestPatch = "LatestPatch";
+            public const string Minor = "Minor";
+            public const string Major = "Major";
+            public const string LatestMinor = "LatestMinor";
+            public const string LatestMajor = "LatestMajor";
+            public const string Disable = "Disable";
+        }
+
         public static class FxVersion
         {
             public const string CommandLineArgument = "--fx-version";
         }
 
+        public static class RollForwardToPreRelease
+        {
+            public const string EnvironmentVariable = "DOTNET_ROLL_FORWARD_TO_PRERELEASE";
+        }
+
         public static class TestOnlyEnvironmentVariables
         {
             public const string RegistryPath = "_DOTNET_TEST_REGISTRY_PATH";
index 60d0d71..51bf7b4 100644 (file)
@@ -4,7 +4,6 @@
 
 using Microsoft.DotNet.Cli.Build;
 using Microsoft.DotNet.Cli.Build.Framework;
-using System;
 using Xunit;
 
 namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
@@ -22,60 +21,55 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
             SharedState = sharedState;
         }
 
+        // Verifies that the default is true
         [Fact]
         public void Default()
         {
             RunTest(
-                runtimeConfig => runtimeConfig
-                    .WithFramework(MicrosoftNETCoreApp, "5.1.2"),
-                result => result.Should().Pass()
-                    .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3"));
-        }
-
-        [Fact]
-        public void RuntimeConfigOnly()
-        {
-            RunTest(
-                runtimeConfig => runtimeConfig
-                    .WithApplyPatches(false)
-                    .WithFramework(MicrosoftNETCoreApp, "5.1.2"),
-                result => result.Should().Pass()
-                    .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.2"));
+                new TestSettings()
+                    .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
+                        .WithFramework(MicrosoftNETCoreApp, "5.1.2")))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3");
         }
 
-        [Fact]
-        public void FrameworkReferenceOnly()
+        // Verifies that it works in all supported locations
+        [Theory]
+        [InlineData(SettingLocation.RuntimeOptions)]
+        [InlineData(SettingLocation.FrameworkReference)]
+        public void AllLocations(SettingLocation location)
         {
             RunTest(
-                runtimeConfig => runtimeConfig
-                    .WithFramework(new RuntimeConfig.Framework(MicrosoftNETCoreApp, "5.1.2")
-                        .WithApplyPatches(false)),
-                result => result.Should().Pass()
-                    .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.2"));
+                new TestSettings()
+                    .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
+                        .WithFramework(MicrosoftNETCoreApp, "5.1.2"))
+                    .With(ApplyPatchesSetting(location, false)))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.2");
         }
 
+        // Verifies that framework reference setting wins over runtime options one
         [Fact]
         public void Priority()
         {
             RunTest(
-                runtimeConfig => runtimeConfig
-                    .WithApplyPatches(true)
-                    .WithFramework(new RuntimeConfig.Framework(MicrosoftNETCoreApp, "5.1.2")
-                        .WithApplyPatches(false)),
-                result => result.Should().Pass()
-                    .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.2"));
+                new TestSettings()
+                    .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
+                        .WithApplyPatches(true)
+                        .WithFramework(new RuntimeConfig.Framework(MicrosoftNETCoreApp, "5.1.2")
+                            .WithApplyPatches(false))))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.2");
 
             RunTest(
-                runtimeConfig => runtimeConfig
-                    .WithApplyPatches(false)
-                    .WithFramework(new RuntimeConfig.Framework(MicrosoftNETCoreApp, "5.1.2")
-                        .WithApplyPatches(true)),
-                result => result.Should().Pass()
-                    .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3"));
+                new TestSettings()
+                    .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
+                        .WithApplyPatches(false)
+                        .WithFramework(new RuntimeConfig.Framework(MicrosoftNETCoreApp, "5.1.2")
+                            .WithApplyPatches(true))))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3");
         }
 
+        // Verifies that it works on inner framework references in runtime options
         [Fact]
-        public void InnerFrameworkReference_RuntimeConfig()
+        public void InnerFrameworkReference_RuntimeOptions()
         {
             using (var dotnetCustomizer = SharedState.DotNetWithFrameworks.Customize())
             {
@@ -83,14 +77,15 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
                     runtimeConfig.WithApplyPatches(false));
 
                 RunTest(
-                    runtimeConfig => runtimeConfig
-                        .WithFramework(MiddleWare, "2.1.0"),
-                    result => result.Should().Pass()
-                        .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.2")
-                        .And.HaveResolvedFramework(MiddleWare, "2.1.2"));
+                    new TestSettings()
+                        .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
+                            .WithFramework(MiddleWare, "2.1.0")))
+                    .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.2")
+                    .And.HaveResolvedFramework(MiddleWare, "2.1.2");
             }
         }
 
+        // Verifies that it works on inner framework references in framework reference
         [Fact]
         public void InnerFrameworkReference_Framework()
         {
@@ -100,14 +95,15 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
                     runtimeConfig.GetFramework(MicrosoftNETCoreApp).WithApplyPatches(false));
 
                 RunTest(
-                    runtimeConfig => runtimeConfig
-                        .WithFramework(MiddleWare, "2.1.0"),
-                    result => result.Should().Pass()
-                        .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.2")
-                        .And.HaveResolvedFramework(MiddleWare, "2.1.2"));
+                    new TestSettings()
+                        .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
+                            .WithFramework(MiddleWare, "2.1.0")))
+                    .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.2")
+                    .And.HaveResolvedFramework(MiddleWare, "2.1.2");
             }
         }
 
+        // Verifies that the setting is not inherited between frameworks
         [Theory]
         [InlineData(SettingLocation.RuntimeOptions)]
         [InlineData(SettingLocation.FrameworkReference)]
@@ -117,32 +113,12 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
                 new TestSettings()
                     .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
                         .WithFramework(MiddleWare, "2.1.2"))
-                    .With(ApplyPatchesSetting(settingLocation, false, MiddleWare)),
-                result => result.Should().Pass()
-                    .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3"));
-        }
-
-        private void RunTest(
-            Func<RuntimeConfig, RuntimeConfig> runtimeConfig,
-            Action<CommandResult> resultAction)
-        {
-            RunTest(
-                SharedState.DotNetWithFrameworks,
-                SharedState.FrameworkReferenceApp,
-                runtimeConfig,
-                resultAction);
+                    .With(ApplyPatchesSetting(settingLocation, false, MiddleWare)))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3");
         }
 
-        private void RunTest(
-            TestSettings testSettings,
-            Action<CommandResult> resultAction)
-        {
-            RunTest(
-                SharedState.DotNetWithFrameworks,
-                SharedState.FrameworkReferenceApp,
-                testSettings,
-                resultAction);
-        }
+        private CommandResult RunTest(TestSettings testSettings) =>
+            RunTest(SharedState.DotNetWithFrameworks, SharedState.FrameworkReferenceApp, testSettings);
 
         public class SharedTestState : SharedTestStateBase
         {
index 8a91ec6..c404d3d 100644 (file)
@@ -10,7 +10,7 @@ using System.Linq;
 
 namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
 {
-    internal static class DotNetCliExtensions
+    public static class DotNetCliExtensions
     {
         public static DotNetCliCustomizer Customize(this DotNetCli dotnet)
         {
@@ -22,7 +22,7 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
             void Backup(string path);
         }
 
-        internal class DotNetCliCustomizer : IDisposable, ITestFileBackup
+        public class DotNetCliCustomizer : IDisposable, ITestFileBackup
         {
             private readonly DotNetCli _dotnet;
             private readonly string _basePath;
@@ -116,7 +116,7 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
             }
         }
 
-        internal class DotNetFramework
+        public class DotNetFramework
         {
             private readonly ITestFileBackup _backup;
             private readonly string _path;
index ecb877b..feec180 100644 (file)
 // See the LICENSE file in the project root for more information.
 
 using System;
+using System.Collections.Generic;
+using System.Linq;
 
 namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
 {
     public partial class FrameworkResolutionBase
     {
+        public class TestSettings
+        {
+            public Func<RuntimeConfig, RuntimeConfig> RuntimeConfigCustomizer { get; set; }
+            public IDictionary<string, string> Environment { get; set; }
+            public IEnumerable<string> CommandLine { get; set; }
+            public Action<DotNetCliExtensions.DotNetCliCustomizer> DotnetCustomizer { get; set; }
+            public string WorkingDirectory { get; set; }
+
+            public TestSettings WithRuntimeConfigCustomizer(Func<RuntimeConfig, RuntimeConfig> customizer)
+            {
+                Func<RuntimeConfig, RuntimeConfig> previousCustomizer = RuntimeConfigCustomizer;
+                if (previousCustomizer == null)
+                {
+                    RuntimeConfigCustomizer = customizer;
+                }
+                else
+                {
+                    RuntimeConfigCustomizer = runtimeConfig => customizer(previousCustomizer(runtimeConfig));
+                }
+
+                return this;
+            }
+
+            public TestSettings WithEnvironment(string key, string value)
+            {
+                Environment = Environment == null ?
+                    new Dictionary<string, string>() { { key, value } } :
+                    new Dictionary<string, string>(Environment.Append(new KeyValuePair<string, string>(key, value)));
+                return this;
+            }
+
+            public TestSettings WithCommandLine(params string[] args)
+            {
+                CommandLine = CommandLine == null ? args : CommandLine.Concat(args);
+                return this;
+            }
+
+            public TestSettings WithWorkingDirectory(string path)
+            {
+                WorkingDirectory = path;
+                return this;
+            }
+
+            public TestSettings WithDotnetCustomizer(Action<DotNetCliExtensions.DotNetCliCustomizer> customizer)
+            {
+                Action<DotNetCliExtensions.DotNetCliCustomizer> previousCustomizer = DotnetCustomizer;
+                if (previousCustomizer == null)
+                {
+                    DotnetCustomizer = customizer;
+                }
+                else
+                {
+                    DotnetCustomizer = dotnet => { previousCustomizer(dotnet); customizer(dotnet); };
+                }
+
+                return this;
+            }
+
+            public TestSettings With(Func<TestSettings, TestSettings> modifier)
+            {
+                return modifier(this);
+            }
+        }
+
         public enum SettingLocation
         {
+            None,
             CommandLine,
             Environment,
             RuntimeOptions,
             FrameworkReference
         }
 
+        public static Func<TestSettings, TestSettings> RollForwardSetting(
+            SettingLocation location,
+            string value,
+            string frameworkReferenceName = MicrosoftNETCoreApp)
+        {
+            if (value == null || location == SettingLocation.None)
+            {
+                return testSettings => testSettings;
+            }
+
+            switch (location)
+            {
+                case SettingLocation.Environment:
+                    return testSettings => testSettings.WithEnvironment(Constants.RollForwardSetting.EnvironmentVariable, value);
+                case SettingLocation.CommandLine:
+                    return testSettings => testSettings.WithCommandLine(Constants.RollForwardSetting.CommandLineArgument, value);
+                case SettingLocation.RuntimeOptions:
+                    return testSettings => testSettings.WithRuntimeConfigCustomizer(rc => rc.WithRollForward(value));
+                case SettingLocation.FrameworkReference:
+                    return testSettings => testSettings.WithRuntimeConfigCustomizer(rc =>
+                    {
+                        rc.GetFramework(frameworkReferenceName).WithRollForward(value);
+                        return rc;
+                    });
+                default:
+                    throw new Exception($"RollForward forward doesn't support setting location {location}.");
+            }
+        }
+
         public static Func<TestSettings, TestSettings> RollForwardOnNoCandidateFxSetting(
             SettingLocation location,
             int? value,
             string frameworkReferenceName = MicrosoftNETCoreApp)
         {
-            if (!value.HasValue)
+            if (!value.HasValue || location == SettingLocation.None)
             {
                 return testSettings => testSettings;
             }
@@ -50,7 +146,7 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
             bool? value,
             string frameworkReferenceName = MicrosoftNETCoreApp)
         {
-            if (!value.HasValue)
+            if (!value.HasValue || location == SettingLocation.None)
             {
                 return testSettings => testSettings;
             }
index 312078f..c0ab424 100644 (file)
@@ -5,7 +5,6 @@
 using Microsoft.DotNet.Cli.Build;
 using Microsoft.DotNet.Cli.Build.Framework;
 using System;
-using System.Collections.Generic;
 using System.IO;
 using System.Linq;
 
@@ -15,105 +14,49 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
     {
         protected const string MicrosoftNETCoreApp = "Microsoft.NETCore.App";
 
-        protected void RunTest(
-            DotNetCli dotnet,
-            TestApp app,
-            Func<RuntimeConfig, RuntimeConfig> runtimeConfig,
-            Action<CommandResult> resultAction,
-            IDictionary<string, string> environment = null,
-            string[] commandLine = null,
-            bool multiLevelLookup = false)
+        public static class ResolvedFramework
         {
-            RunTest(
-                dotnet,
-                app,
-                new TestSettings()
-                {
-                    RuntimeConfigCustomizer = runtimeConfig,
-                    Environment = environment,
-                    CommandLine = commandLine
-                },
-                resultAction,
-                multiLevelLookup);
+            public const string NotFound = "[not found]";
+            public const string FailedToReconcile = "[failed to reconcile]";
         }
 
-        public class TestSettings
+        protected CommandResult RunTest(
+            DotNetCli dotnet,
+            TestApp app,
+            TestSettings settings,
+            Action<CommandResult> resultAction = null,
+            bool multiLevelLookup = false)
         {
-            public Func<RuntimeConfig, RuntimeConfig> RuntimeConfigCustomizer { get; set; }
-            public IDictionary<string, string> Environment { get; set; }
-            public IEnumerable<string> CommandLine { get; set; }
-            public string WorkingDirectory { get; set; }
-
-            public TestSettings WithRuntimeConfigCustomizer(Func<RuntimeConfig, RuntimeConfig> customizer)
+            using (DotNetCliExtensions.DotNetCliCustomizer dotnetCustomizer = settings.DotnetCustomizer == null ? null : dotnet.Customize())
             {
-                Func<RuntimeConfig, RuntimeConfig> previousCustomizer = RuntimeConfigCustomizer;
-                if (previousCustomizer == null)
-                {
-                    RuntimeConfigCustomizer = customizer;
-                }
-                else
+                settings.DotnetCustomizer?.Invoke(dotnetCustomizer);
+
+                if (settings.RuntimeConfigCustomizer != null)
                 {
-                    RuntimeConfigCustomizer = runtimeConfig => customizer(previousCustomizer(runtimeConfig));
+                    settings.RuntimeConfigCustomizer(RuntimeConfig.Path(app.RuntimeConfigJson)).Save();
                 }
 
-                return this;
-            }
-
-            public TestSettings WithEnvironment(string key, string value)
-            {
-                Environment = Environment == null ? 
-                    new Dictionary<string, string>() { { key, value } } : 
-                    new Dictionary<string, string>(Environment.Append(new KeyValuePair<string, string>(key, value)));
-                return this;
-            }
-
-            public TestSettings WithCommandLine(params string[] args)
-            {
-                CommandLine = CommandLine == null ? args : CommandLine.Concat(args);
-                return this;
-            }
-
-            public TestSettings WithWorkingDirectory(string path)
-            {
-                WorkingDirectory = path;
-                return this;
-            }
+                settings.WithCommandLine(app.AppDll);
 
-            public TestSettings With(Func<TestSettings, TestSettings> modifier)
-            {
-                return modifier(this);
-            }
-        }
+                Command command = dotnet.Exec(settings.CommandLine.First(), settings.CommandLine.Skip(1).ToArray());
 
-        protected void RunTest(
-            DotNetCli dotnet,
-            TestApp app,
-            TestSettings settings,
-            Action<CommandResult> resultAction,
-            bool multiLevelLookup = false)
-        {
-            if (settings.RuntimeConfigCustomizer != null)
-            {
-                settings.RuntimeConfigCustomizer(RuntimeConfig.Path(app.RuntimeConfigJson)).Save();
-            }
+                if (settings.WorkingDirectory != null)
+                {
+                    command = command.WorkingDirectory(settings.WorkingDirectory);
+                }
 
-            settings.WithCommandLine(app.AppDll);
+                CommandResult result = command
+                    .EnvironmentVariable("COREHOST_TRACE", "1")
+                    .EnvironmentVariable("DOTNET_MULTILEVEL_LOOKUP", multiLevelLookup ? "1" : "0")
+                    .Environment(settings.Environment)
+                    .CaptureStdOut()
+                    .CaptureStdErr()
+                    .Execute();
 
-            Command command = dotnet.Exec(settings.CommandLine.First(), settings.CommandLine.Skip(1).ToArray());
+                resultAction?.Invoke(result);
 
-            if (settings.WorkingDirectory != null)
-            {
-                command = command.WorkingDirectory(settings.WorkingDirectory);
+                return result;
             }
-
-            CommandResult result = command
-                .EnvironmentVariable("COREHOST_TRACE", "1")
-                .EnvironmentVariable("DOTNET_MULTILEVEL_LOOKUP", multiLevelLookup ? "1" : "0")
-                .Environment(settings.Environment)
-                .CaptureStdOut()
-                .CaptureStdErr()
-                .Execute();
-            resultAction(result);
         }
 
         public class SharedTestStateBase : IDisposable
index 222bd8c..3344068 100644 (file)
@@ -3,6 +3,7 @@
 // See the LICENSE file in the project root for more information.
 
 using FluentAssertions;
+using Microsoft.DotNet.Cli.Build.Framework;
 
 namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
 {
@@ -13,19 +14,100 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
             return assertion.HaveStdOutContaining($"mock frameworks: {name} {version}");
         }
 
+        public static AndConstraint<CommandResultAssertions> ShouldHaveResolvedFramework(this CommandResult result, string resolvedFrameworkName, string resolvedFrameworkVersion)
+        {
+            return result.Should().Pass()
+                .And.HaveResolvedFramework(resolvedFrameworkName, resolvedFrameworkVersion);
+        }
+
+        /// <summary>
+        /// Verifies that the command result either passes with a resolved framework or fails with inability to find compatible framework version.
+        /// </summary>
+        /// <param name="result">The result to verify.</param>
+        /// <param name="resolvedFrameworkName">The name of the framework to verify.</param>
+        /// <param name="resolvedFrameworkVersion">
+        ///     Either null in which case the command result is expected to fail and not find compatible framework version,
+        ///     or the framework versions in which case the command result is expected to succeed and resolve the specified framework version.</param>
+        /// <returns>Constraint</returns>
+        public static AndConstraint<CommandResultAssertions> ShouldHaveResolvedFrameworkOrFailToFind(this CommandResult result, string resolvedFrameworkName, string resolvedFrameworkVersion)
+        {
+            if (resolvedFrameworkName == null || resolvedFrameworkVersion == null || 
+                resolvedFrameworkVersion == FrameworkResolutionBase.ResolvedFramework.NotFound)
+            {
+                return result.ShouldFailToFindCompatibleFrameworkVersion();
+            }
+            else
+            {
+                return result.ShouldHaveResolvedFramework(resolvedFrameworkName, resolvedFrameworkVersion);
+            }
+        }
+
         public static AndConstraint<CommandResultAssertions> DidNotFindCompatibleFrameworkVersion(this CommandResultAssertions assertion)
         {
             return assertion.HaveStdErrContaining("It was not possible to find any compatible framework version");
         }
 
-        public static AndConstraint<CommandResultAssertions> FailedToSoftRollForward(this CommandResultAssertions assertion, string frameworkName, string newVersion, string previousVersion)
+        public static AndConstraint<CommandResultAssertions> ShouldFailToFindCompatibleFrameworkVersion(this CommandResult result)
+        {
+            return result.Should().Fail()
+                .And.DidNotFindCompatibleFrameworkVersion();
+        }
+
+        public static AndConstraint<CommandResultAssertions> FailedToReconcileFrameworkReference(
+            this CommandResultAssertions assertion,
+            string frameworkName,
+            string newVersion,
+            string previousVersion)
         {
-            return assertion.HaveStdErrMatching($".*The specified framework '{frameworkName}', version '{newVersion}', patch_roll_fwd=[0-1], roll_fwd_on_no_candidate_fx=[0-2] cannot roll-forward to the previously referenced version '{previousVersion}'.*");
+            return assertion.HaveStdErrMatching($".*The specified framework '{frameworkName}', version '{newVersion}', apply_patches=[0-1], roll_forward=[^ ]* cannot roll-forward to the previously referenced version '{previousVersion}'.*");
+        }
+
+        public static AndConstraint<CommandResultAssertions> ShouldHaveResolvedFrameworkOrFailedToReconcileFrameworkReference(
+            this CommandResult result,
+            string frameworkName,
+            string resolvedVersion,
+            string lowerVersion,
+            string higherVersion)
+        {
+            if (resolvedVersion == null || resolvedVersion == FrameworkResolutionBase.ResolvedFramework.FailedToReconcile)
+            {
+                return result.Should().Fail().And.FailedToReconcileFrameworkReference(frameworkName, lowerVersion, higherVersion);
+            }
+            else
+            {
+                return result.ShouldHaveResolvedFramework(frameworkName, resolvedVersion);
+            }
+        }
+
+        public static AndConstraint<CommandResultAssertions> ShouldHaveResolvedFrameworkOrFail(
+            this CommandResult result,
+            string frameworkName,
+            string resolvedVersion,
+            string lowerVersion,
+            string higherVersion)
+        {
+            if (resolvedVersion == FrameworkResolutionBase.ResolvedFramework.FailedToReconcile)
+            {
+                return result.Should().Fail().And.FailedToReconcileFrameworkReference(frameworkName, lowerVersion, higherVersion);
+            }
+            else if (resolvedVersion == FrameworkResolutionBase.ResolvedFramework.NotFound)
+            {
+                return result.ShouldFailToFindCompatibleFrameworkVersion();
+            }
+            else
+            {
+                return result.ShouldHaveResolvedFramework(frameworkName, resolvedVersion);
+            }
         }
 
         public static AndConstraint<CommandResultAssertions> RestartedFrameworkResolution(this CommandResultAssertions assertion, string resolvedVersion, string newVersion)
         {
             return assertion.HaveStdErrContaining($"--- Restarting all framework resolution because the previously resolved framework 'Microsoft.NETCore.App', version '{resolvedVersion}' must be re-resolved with the new version '{newVersion}'");
         }
+
+        public static AndConstraint<CommandResultAssertions> DidNotRecognizeRollForwardValue(this CommandResultAssertions assertion, string value)
+        {
+            return assertion.HaveStdErrContaining($"Unrecognized roll forward setting value '{value}'.");
+        }
     }
 }
index 2a3617b..e07ffbb 100644 (file)
@@ -3,6 +3,7 @@
 // See the LICENSE file in the project root for more information.
 
 using Microsoft.DotNet.Cli.Build;
+using Microsoft.DotNet.Cli.Build.Framework;
 using Xunit;
 
 namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
@@ -20,33 +21,36 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
             SharedState = sharedState;
         }
 
-        [Theory]
-        [InlineData("1.0.0", "2.5.5", "2.5.5")]
-        [InlineData("2.0.0", "2.5.5", "2.5.5")]
-        [InlineData("2.5.4", "2.5.5", "2.5.5")]
-        [InlineData("2.5.5", "2.5.5", "2.5.5")]
-        [InlineData("2.5.5", "2.5.4", "2.5.4")]
-        [InlineData("2.5.5", "2.5.3", null)]
-        [InlineData("2.5.5", "2.5.5-preview1", null)]
+        // Validates that --fx-version <fxVersion> overrides framework version specified
+        // in the runtime config in the framework reference <fxRefVer>
+        [Theory] // fxRefVer fxVersion         resolvedFramework
+        [InlineData("1.0.0", "2.5.5",          "2.5.5")]
+        [InlineData("2.0.0", "2.5.5",          "2.5.5")]
+        [InlineData("2.5.4", "2.5.5",          "2.5.5")]
+        [InlineData("2.5.5", "2.5.5",          "2.5.5")]
+        [InlineData("2.5.5", "2.5.4",          "2.5.4")]
+        [InlineData("2.5.5", "2.5.3",          ResolvedFramework.NotFound)]
+        [InlineData("2.5.5", "2.5.5-preview1", ResolvedFramework.NotFound)]
         public void OverridesFrameworkReferences(string frameworkReferenceVersion, string fxVersion, string resolvedFramework)
         {
             RunTest(
                 new TestSettings()
                     .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
                         .WithFramework(MicrosoftNETCoreApp, frameworkReferenceVersion))
-                    .WithCommandLine(Constants.FxVersion.CommandLineArgument, fxVersion),
-                resolvedFramework: resolvedFramework);
+                    .WithCommandLine(Constants.FxVersion.CommandLineArgument, fxVersion))
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
         }
 
-        [Theory]
-        [InlineData(null, null)]
-        [InlineData(0, null)]
-        [InlineData(0, true)]
-        [InlineData(1, null)]
-        [InlineData(1, true)]
-        [InlineData(2, null)]
-        [InlineData(0, false)]
-        public void IgnoresOtherSettings(int? rollForwardOnNoCandidateFx, bool? applyPatches)
+        // Validates that --fx-version ignores any <rollForwardOnNoCandidateFx> or <applyPatches> settings
+        [Theory] // rollForwardOnNoCandidateFx applyPatches
+        [InlineData(null,                      null )]
+        [InlineData(0,                         null )]
+        [InlineData(0,                         true )]
+        [InlineData(1,                         null )]
+        [InlineData(1,                         true )]
+        [InlineData(2,                         null )]
+        [InlineData(0,                         false)]
+        public void IgnoresRollForwardOnNoCandidateFxAndApplyPatchesSettings(int? rollForwardOnNoCandidateFx, bool? applyPatches)
         {
             RunTest(
                 new TestSettings()
@@ -54,10 +58,31 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
                         .WithFramework(MicrosoftNETCoreApp, "2.5.4"))
                     .WithCommandLine(Constants.FxVersion.CommandLineArgument, "2.5.5")
                     .With(RollForwardOnNoCandidateFxSetting(SettingLocation.CommandLine, rollForwardOnNoCandidateFx))
-                    .With(ApplyPatchesSetting(SettingLocation.RuntimeOptions, applyPatches)),
-                resolvedFramework: "2.5.5");
+                    .With(ApplyPatchesSetting(SettingLocation.RuntimeOptions, applyPatches)))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "2.5.5");
         }
 
+        // Validates that --fx-version ignores any rollForward <rollForward> settings
+        [Theory] // rollForward
+        [InlineData(null                                    )]
+        [InlineData(Constants.RollForwardSetting.Disable    )]
+        [InlineData(Constants.RollForwardSetting.LatestPatch)]
+        [InlineData(Constants.RollForwardSetting.Minor      )]
+        [InlineData(Constants.RollForwardSetting.LatestMinor)]
+        [InlineData(Constants.RollForwardSetting.Major      )]
+        [InlineData(Constants.RollForwardSetting.LatestMajor)]
+        public void IgnoresRollForwardSettings(string rollForward)
+        {
+            RunTest(
+                new TestSettings()
+                    .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
+                        .WithFramework(MicrosoftNETCoreApp, "2.5.4"))
+                    .WithCommandLine(Constants.FxVersion.CommandLineArgument, "2.5.5")
+                    .With(RollForwardSetting(SettingLocation.CommandLine, rollForward)))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "2.5.5");
+        }
+
+        // Validates that --fx-version only applies to the first framework reference
         [Fact]
         public void AppliesToFirstFrameworkReference_NETCoreAppFirst()
         {
@@ -66,10 +91,11 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
                     .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
                         .WithFramework(MicrosoftNETCoreApp, "1.0.0")
                         .WithFramework(MiddleWare, "2.1.2"))
-                    .WithCommandLine(Constants.FxVersion.CommandLineArgument, "2.5.5"),
-                resolvedFramework: "2.5.5");
+                    .WithCommandLine(Constants.FxVersion.CommandLineArgument, "2.5.5"))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "2.5.5");
         }
 
+        // Validates that --fx-version only applies to the first framework reference
         [Fact]
         public void AppliesToFirstFrameworkReference_MiddleWareFirst()
         {
@@ -78,32 +104,12 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
                     .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
                         .WithFramework(MiddleWare, "1.0.0")
                         .WithFramework(MicrosoftNETCoreApp, "2.5.0"))
-                    .WithCommandLine(Constants.FxVersion.CommandLineArgument, "2.1.2"),
-                resolvedFramework: "2.5.5");
+                    .WithCommandLine(Constants.FxVersion.CommandLineArgument, "2.1.2"))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "2.5.5");
         }
 
-        private void RunTest(
-            TestSettings testSettings,
-            string resolvedFramework = null)
-        {
-            RunTest(
-                SharedState.DotNetWithFrameworks,
-                SharedState.FrameworkReferenceApp,
-                testSettings,
-                commandResult =>
-                {
-                    if (resolvedFramework != null)
-                    {
-                        commandResult.Should().Pass()
-                            .And.HaveResolvedFramework(MicrosoftNETCoreApp, resolvedFramework);
-                    }
-                    else
-                    {
-                        commandResult.Should().Fail()
-                            .And.DidNotFindCompatibleFrameworkVersion();
-                    }
-                });
-        }
+        private CommandResult RunTest(TestSettings testSettings) =>
+            RunTest(SharedState.DotNetWithFrameworks, SharedState.FrameworkReferenceApp, testSettings);
 
         public class SharedTestState : SharedTestStateBase
         {
index 9a71b49..31c5ae3 100644 (file)
@@ -3,6 +3,7 @@
 // See the LICENSE file in the project root for more information.
 
 using Microsoft.DotNet.Cli.Build;
+using Microsoft.DotNet.Cli.Build.Framework;
 using System;
 using System.Runtime.InteropServices;
 using Xunit;
@@ -23,64 +24,60 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
         [Fact]
         public void FrameworkHiveSelection_GlobalHiveWithBetterMatch()
         {
+            if (!RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
+            {
+                // Multiple hives are only supported on Windows.
+                return;
+            }
+
             RunTest(
                 runtimeConfig => runtimeConfig
-                    .WithFramework(MicrosoftNETCoreApp, "5.0.0"),
-                "5.1.2");
+                    .WithFramework(MicrosoftNETCoreApp, "5.0.0"))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.2");
         }
 
         [Fact]
         public void FrameworkHiveSelection_MainHiveWithBetterMatch()
         {
+            if (!RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
+            {
+                // Multiple hives are only supported on Windows.
+                return;
+            }
+
             RunTest(
                 runtimeConfig => runtimeConfig
-                    .WithFramework(MicrosoftNETCoreApp, "6.0.0"),
-                "6.1.2");
+                    .WithFramework(MicrosoftNETCoreApp, "6.0.0"))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "6.1.2");
         }
 
         [Fact]
         public void FrameworkHiveSelection_CurrentDirectoryIsIgnored()
         {
+            if (!RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
+            {
+                // Multiple hives are only supported on Windows.
+                return;
+            }
+
             RunTest(
                 SharedState.DotNetMainHive,
                 SharedState.FrameworkReferenceApp,
                 new TestSettings()
                     .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
                         .WithFramework(MicrosoftNETCoreApp, "5.0.0"))
-                    .WithWorkingDirectory(SharedState.DotNetCurrentHive.BinPath),
-                result => result.Should().Pass()
-                    .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.2.0"));
+                    .WithWorkingDirectory(SharedState.DotNetCurrentHive.BinPath))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.2.0");
         }
 
-        private void RunTest(
-            Func<RuntimeConfig, RuntimeConfig> runtimeConfig,
-            string resolvedFramework = null)
+        private CommandResult RunTest(Func<RuntimeConfig, RuntimeConfig> runtimeConfig)
         {
-            if (!RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
-            {
-                // Multiple hives are only supported on Windows.
-                return;
-            }
-
-            RunTest(
+            return RunTest(
                 SharedState.DotNetMainHive,
                 SharedState.FrameworkReferenceApp,
                 new TestSettings()
                     .WithRuntimeConfigCustomizer(runtimeConfig)
                     .WithEnvironment(Constants.TestOnlyEnvironmentVariables.GloballyRegisteredPath, SharedState.DotNetGlobalHive.BinPath),
-                commandResult =>
-                {
-                    if (resolvedFramework != null)
-                    {
-                        commandResult.Should().Pass()
-                            .And.HaveResolvedFramework(MicrosoftNETCoreApp, resolvedFramework);
-                    }
-                    else
-                    {
-                        commandResult.Should().Fail()
-                            .And.DidNotFindCompatibleFrameworkVersion();
-                    }
-                },
                 // Must enable multi-level lookup otherwise multiple hives are not enabled
                 multiLevelLookup: true);
         }
diff --git a/src/installer/test/HostActivationTests/FrameworkResolution/RollForwardMultipleFrameworks.cs b/src/installer/test/HostActivationTests/FrameworkResolution/RollForwardMultipleFrameworks.cs
new file mode 100644 (file)
index 0000000..e1aaf86
--- /dev/null
@@ -0,0 +1,704 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+// See the LICENSE file in the project root for more information.
+
+using Microsoft.DotNet.Cli.Build;
+using Microsoft.DotNet.Cli.Build.Framework;
+using System;
+using Xunit;
+
+namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
+{
+    public class RollForwardMultipleFrameworks :
+        FrameworkResolutionBase,
+        IClassFixture<RollForwardMultipleFrameworks.SharedTestState>
+    {
+        private const string MiddleWare = "MiddleWare";
+        private const string AnotherMiddleWare = "AnotherMiddleWare";
+        private const string HighWare = "HighWare";
+
+        private SharedTestState SharedState { get; }
+
+        public RollForwardMultipleFrameworks(SharedTestState sharedState)
+        {
+            SharedState = sharedState;
+        }
+
+        public class SharedTestState : SharedTestStateBase
+        {
+            public TestApp FrameworkReferenceApp { get; }
+
+            public DotNetCli DotNetWithMultipleFrameworks { get; }
+
+            public SharedTestState()
+            {
+                DotNetWithMultipleFrameworks = DotNet("WithOneFramework")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.3")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.4.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.6.0")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("6.0.0")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("6.1.0")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("6.1.1-preview.2")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("6.2.1")
+                    .AddFramework(MiddleWare, "2.1.2", runtimeConfig =>
+                        runtimeConfig.WithFramework(MicrosoftNETCoreApp, "5.1.3"))
+                    .AddFramework(AnotherMiddleWare, "3.0.0", runtimeConfig =>
+                        runtimeConfig.WithFramework(MicrosoftNETCoreApp, "5.1.3"))
+                    .AddFramework(HighWare, "7.3.1", runtimeConfig =>
+                        runtimeConfig
+                            .WithFramework(MicrosoftNETCoreApp, "5.1.3")
+                            .WithFramework(MiddleWare, "2.1.2"))
+                    .Build();
+
+                FrameworkReferenceApp = CreateFrameworkReferenceApp();
+            }
+        }
+
+        // Verify that inner framework reference (<fxRefVersion>, <rollForward>)
+        // is correctly reconciled with app's framework reference 5.1.1 (defaults = RollForward:Minor). App fx reference is higher.
+        [Theory] // fxRefVersion  rollForward                               resolvedFramework
+        [InlineData("5.0.0",      Constants.RollForwardSetting.Disable,     ResolvedFramework.FailedToReconcile)]
+        [InlineData("5.1.1",      Constants.RollForwardSetting.Disable,     "5.1.1")]
+        [InlineData("5.0.0",      Constants.RollForwardSetting.LatestPatch, ResolvedFramework.FailedToReconcile)]
+        [InlineData("5.1.0",      Constants.RollForwardSetting.LatestPatch, "5.1.3")]
+        [InlineData("5.1.1",      Constants.RollForwardSetting.LatestPatch, "5.1.3")]
+        [InlineData("5.0.0",      null,                                     "5.1.3")]
+        [InlineData("5.1.1",      null,                                     "5.1.3")]
+        [InlineData("5.0.0",      Constants.RollForwardSetting.Minor,       "5.1.3")]
+        [InlineData("5.1.1",      Constants.RollForwardSetting.Minor,       "5.1.3")]
+        [InlineData("5.0.0",      Constants.RollForwardSetting.LatestMinor, "5.1.3")] // The app reference which is Minor wins
+        [InlineData("5.1.1",      Constants.RollForwardSetting.LatestMinor, "5.1.3")] // The app reference which is Minor wins
+        [InlineData("1.0.0",      Constants.RollForwardSetting.Minor,       ResolvedFramework.FailedToReconcile)]
+        [InlineData("1.0.0",      Constants.RollForwardSetting.Major,       "5.1.3")] // The app reference which is Minor wins
+        [InlineData("1.0.0",      Constants.RollForwardSetting.LatestMajor, "5.1.3")] // The app reference which is Minor wins
+        public void ReconcileFrameworkReferences_InnerFrameworkReference_ToHigher(
+            string versionReference,
+            string rollForward,
+            string resolvedFramework)
+        {
+            RunTest(
+                runtimeConfig => runtimeConfig
+                    .WithFramework(MiddleWare, "2.1.0")
+                    .WithFramework(MicrosoftNETCoreApp, "5.1.1"),
+                dotnetCustomizer => dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
+                    runtimeConfig.GetFramework(MicrosoftNETCoreApp)
+                        .WithRollForward(rollForward)
+                        .Version = versionReference))
+                .ShouldHaveResolvedFrameworkOrFailedToReconcileFrameworkReference(
+                    MicrosoftNETCoreApp, resolvedFramework, versionReference, "5.1.1");
+        }
+
+        // Verify that inner framework reference (<fxRefVersion>, <rollForward>)
+        // is correctly reconciled with app's framework reference 5.1.1 (defaults = RollForward:Minor). App fx reference is higher.
+        // In this case the direct reference from app is first, so the framework reference from app
+        // is actually resolved against the disk - and the resolved framework is than compared to
+        // the inner framework reference (potentially causing re-resolution).
+        [Theory] // fxRefVersion  rollForward                               resolvedFramework
+        [InlineData("5.0.0",      Constants.RollForwardSetting.Disable,     ResolvedFramework.FailedToReconcile)]
+        [InlineData("5.1.1",      Constants.RollForwardSetting.Disable,     "5.1.1")]
+        [InlineData("5.0.0",      Constants.RollForwardSetting.LatestPatch, ResolvedFramework.FailedToReconcile)]
+        [InlineData("5.1.0",      Constants.RollForwardSetting.LatestPatch, "5.1.3")]
+        [InlineData("5.1.1",      Constants.RollForwardSetting.LatestPatch, "5.1.3")]
+        [InlineData("5.0.0",      null,                                     "5.1.3")]
+        [InlineData("5.1.1",      null,                                     "5.1.3")]
+        [InlineData("5.0.0",      Constants.RollForwardSetting.Minor,       "5.1.3")]
+        [InlineData("5.1.1",      Constants.RollForwardSetting.Minor,       "5.1.3")]
+        [InlineData("5.0.0",      Constants.RollForwardSetting.LatestMinor, "5.1.3")] // The app reference which is Minor wins
+        [InlineData("5.1.1",      Constants.RollForwardSetting.LatestMinor, "5.1.3")] // The app reference which is Minor wins
+        [InlineData("1.0.0",      Constants.RollForwardSetting.Minor,       ResolvedFramework.FailedToReconcile)]
+        [InlineData("1.0.0",      Constants.RollForwardSetting.Major,       "5.1.3")] // The app reference which is Minor wins
+        [InlineData("1.0.0",      Constants.RollForwardSetting.LatestMajor, "5.1.3")] // The app reference which is Minor wins
+        public void ReconcileFrameworkReferences_InnerFrameworkReference_ToHigher_HardResolve(
+            string versionReference,
+            string rollForward,
+            string resolvedFramework)
+        {
+            RunTest(
+                runtimeConfig => runtimeConfig
+                    .WithFramework(MicrosoftNETCoreApp, "5.1.1")
+                    .WithFramework(MiddleWare, "2.1.0"),
+                dotnetCustomizer => dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
+                    runtimeConfig.GetFramework(MicrosoftNETCoreApp)
+                        .WithRollForward(rollForward)
+                        .Version = versionReference))
+                .ShouldHaveResolvedFrameworkOrFailedToReconcileFrameworkReference(
+                    MicrosoftNETCoreApp, resolvedFramework, versionReference, "5.1.1");
+        }
+
+        // Verify that inner framework reference (<fxRefVersion>, <rollForward>)
+        // is correctly reconciled with app's framework reference 5.1.1 (defaults = RollForward:Minor). App fx reference is lower.
+        // Also validates that since all relevant available versions are release, 
+        // the DOTNET_ROLL_FORWARD_TO_PRERELEASE has no effect on the result.
+        [Theory] // fxRefVersion  rollForward                               rollForwadToPreRelease resolvedFramework
+        [InlineData("5.1.3",      Constants.RollForwardSetting.Disable,     false,                 "5.1.3")]
+        [InlineData("5.4.0",      null,                                     false,                 "5.4.1")]
+        [InlineData("5.4.0",      Constants.RollForwardSetting.Minor,       false,                 "5.4.1")]
+        [InlineData("5.4.0",      Constants.RollForwardSetting.Minor,       true,                  "5.4.1")]
+        [InlineData("5.4.0",      Constants.RollForwardSetting.LatestMinor, false,                 "5.4.1")] // The app's settings (Minor) wins, so effective reference is "5.4.0 Minor"
+        [InlineData("5.4.0",      Constants.RollForwardSetting.LatestMinor, true,                  "5.4.1")]
+        [InlineData("5.4.0",      Constants.RollForwardSetting.Major,       false,                 "5.4.1")] // The app's settings (Minor) wins, so effective reference is "5.4.0 Minor"
+        [InlineData("5.4.0",      Constants.RollForwardSetting.Major,       true,                  "5.4.1")]
+        [InlineData("5.4.0",      Constants.RollForwardSetting.LatestMajor, false,                 "5.4.1")] // The app's settings (Minor) wins, so effective reference is "5.4.0 Minor"
+        [InlineData("5.4.0",      Constants.RollForwardSetting.LatestMajor, true,                  "5.4.1")]
+        [InlineData("5.4.1",      Constants.RollForwardSetting.Disable,     false,                 "5.4.1")]
+        [InlineData("5.7.0",      Constants.RollForwardSetting.Minor,       false,                 ResolvedFramework.NotFound)]
+        [InlineData("5.7.0",      Constants.RollForwardSetting.Minor,       true,                  ResolvedFramework.NotFound)]
+        [InlineData("5.7.0",      Constants.RollForwardSetting.LatestMinor, false,                 ResolvedFramework.NotFound)]
+        [InlineData("5.7.0",      Constants.RollForwardSetting.Major,       false,                 ResolvedFramework.NotFound)]
+        [InlineData("5.7.0",      Constants.RollForwardSetting.LatestMajor, false,                 ResolvedFramework.NotFound)]
+        [InlineData("6.0.0",      Constants.RollForwardSetting.Minor,       false,                 ResolvedFramework.FailedToReconcile)]
+        [InlineData("6.0.0",      Constants.RollForwardSetting.Minor,       true,                  ResolvedFramework.FailedToReconcile)]
+        [InlineData("6.0.0",      Constants.RollForwardSetting.Major,       false,                 ResolvedFramework.FailedToReconcile)]
+        [InlineData("6.0.0",      Constants.RollForwardSetting.LatestMajor, false,                 ResolvedFramework.FailedToReconcile)]
+        public void ReconcileFrameworkReferences_InnerFrameworkReference_ToLower(
+            string versionReference,
+            string rollForward,
+            bool rollForwardToPreRelease,
+            string resolvedFramework)
+        {
+            RunTest(
+                runtimeConfig => runtimeConfig
+                    .WithFramework(MiddleWare, "2.1.0")
+                    .WithFramework(MicrosoftNETCoreApp, "5.1.1"),
+                dotnetCustomizer => dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
+                    runtimeConfig.GetFramework(MicrosoftNETCoreApp)
+                        .WithRollForward(rollForward)
+                        .Version = versionReference),
+                rollForwardToPreRelease)
+                .ShouldHaveResolvedFrameworkOrFail(
+                    MicrosoftNETCoreApp, resolvedFramework, "5.1.1", versionReference);
+        }
+
+        // Verify that inner framework reference (<fxRefVersion>, <rollForward>)
+        // is correctly reconciled with app's framework reference 5.1.1 (defaults = RollForward:Minor). App fx reference is lower.
+        // In this case the direct reference from app is first, so the framework reference from app
+        // is actually resolved against the disk - and the resolved framework is than compared to
+        // the inner framework reference (potentially causing re-resolution).
+        [Theory] // fxRefVersion  rollForward                               resolvedFramework
+        [InlineData("5.1.3",      Constants.RollForwardSetting.Disable,     "5.1.3")]
+        [InlineData("5.4.0",      null,                                     "5.4.1")]
+        [InlineData("5.4.0",      Constants.RollForwardSetting.Minor,       "5.4.1")]
+        [InlineData("5.4.0",      Constants.RollForwardSetting.LatestMinor, "5.4.1")]
+        [InlineData("5.4.0",      Constants.RollForwardSetting.Major,       "5.4.1")]
+        [InlineData("5.4.0",      Constants.RollForwardSetting.LatestMajor, "5.4.1")]
+        [InlineData("5.4.1",      Constants.RollForwardSetting.Disable,     "5.4.1")]
+        [InlineData("5.7.0",      Constants.RollForwardSetting.Minor,       ResolvedFramework.NotFound)]
+        [InlineData("5.7.0",      Constants.RollForwardSetting.LatestMinor, ResolvedFramework.NotFound)]
+        [InlineData("5.7.0",      Constants.RollForwardSetting.Major,       ResolvedFramework.NotFound)]
+        [InlineData("5.7.0",      Constants.RollForwardSetting.LatestMajor, ResolvedFramework.NotFound)]
+        [InlineData("6.0.0",      Constants.RollForwardSetting.Minor,       ResolvedFramework.FailedToReconcile)]
+        [InlineData("6.0.0",      Constants.RollForwardSetting.Major,       ResolvedFramework.FailedToReconcile)]
+        public void ReconcileFrameworkReferences_InnerFrameworkReference_ToLower_HardResolve(
+            string versionReference,
+            string rollForward,
+            string resolvedFramework)
+        {
+            RunTest(
+                runtimeConfig => runtimeConfig
+                    .WithFramework(MicrosoftNETCoreApp, "5.1.1")
+                    .WithFramework(MiddleWare, "2.1.0"),
+                dotnetCustomizer => dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
+                    runtimeConfig.GetFramework(MicrosoftNETCoreApp)
+                        .WithRollForward(rollForward)
+                        .Version = versionReference))
+                .ShouldHaveResolvedFrameworkOrFail(
+                    MicrosoftNETCoreApp, resolvedFramework, "5.1.1", versionReference);
+        }
+
+        // Verify that inner framework reference (<fxRefVersion>, <rollForward>)
+        // is correctly reconciled with app's framework reference 6.1.1-preview.0 (defaults = RollForward:Minor).
+        // Also validates the effect of DOTNET_ROLL_FORWARD_TO_PRERELEASE on the result.
+        [Theory] // fxRefVersion       rollForward                               rollForwadToPreRelease resolvedFramework
+        [InlineData("6.0.0-preview.1", null,                                     false,                 "6.1.1-preview.2")]
+        [InlineData("6.0.0",           null,                                     false,                 "6.2.1")]
+        [InlineData("6.0.0",           Constants.RollForwardSetting.LatestPatch, false,                 ResolvedFramework.FailedToReconcile)]
+        [InlineData("6.0.0-preview.1", Constants.RollForwardSetting.LatestPatch, false,                 ResolvedFramework.FailedToReconcile)]
+        [InlineData("6.0.0-preview.1", Constants.RollForwardSetting.Minor,       false,                 "6.1.1-preview.2")]
+        [InlineData("6.0.0",           Constants.RollForwardSetting.Minor,       false,                 "6.2.1")]
+        [InlineData("6.0.1-preview.0", Constants.RollForwardSetting.LatestPatch, false,                 ResolvedFramework.FailedToReconcile)]
+        [InlineData("6.1.0-preview.0", null,                                     false,                 "6.1.1-preview.2")]
+        [InlineData("6.1.0-preview.0", null,                                     true,                  "6.1.1-preview.2")]
+        [InlineData("6.1.0",           null,                                     false,                 "6.2.1")]
+        [InlineData("6.1.0",           null,                                     true,                  "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.0", null,                                     false,                 "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.0", null,                                     true,                  "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.0", Constants.RollForwardSetting.LatestPatch, false,                 "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.0", Constants.RollForwardSetting.Disable,     false,                 ResolvedFramework.NotFound)]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.Disable,     false,                 "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.Disable,     true,                  "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.LatestPatch, false,                 "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.LatestPatch, true,                  "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", null,                                     false,                 "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", null,                                     true,                  "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.Minor,       false,                 "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.Minor,       true,                  "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.LatestMinor, false,                 "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.LatestMinor, true,                  "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.Major,       false,                 "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.Major,       true,                  "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.LatestMajor, false,                 "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.LatestMajor, true,                  "6.1.1-preview.2")]
+        [InlineData("6.2.1-preview.1", Constants.RollForwardSetting.Disable,     false,                 ResolvedFramework.NotFound)]
+        [InlineData("6.2.1-preview.1", Constants.RollForwardSetting.LatestPatch, false,                 "6.2.1")]
+        [InlineData("6.2.1-preview.1", null,                                     false,                 "6.2.1")]
+        [InlineData("6.2.1-preview.1", Constants.RollForwardSetting.Minor,       false,                 "6.2.1")]
+        [InlineData("6.2.1-preview.1", Constants.RollForwardSetting.LatestMinor, false,                 "6.2.1")]
+        [InlineData("6.2.1-preview.1", Constants.RollForwardSetting.Major,       false,                 "6.2.1")]
+        [InlineData("6.2.1-preview.1", Constants.RollForwardSetting.LatestMajor, false,                 "6.2.1")]
+        public void ReconcileFrameworkReferences_InnerFrameworkReference_PreRelease(
+            string versionReference,
+            string rollForward,
+            bool rollForwardToPreRelease,
+            string resolvedFramework)
+        {
+            RunTest(
+                runtimeConfig => runtimeConfig
+                    .WithFramework(MicrosoftNETCoreApp, "6.1.1-preview.0")
+                    .WithFramework(MiddleWare, "2.1.0"),
+                dotnetCustomizer => dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
+                    runtimeConfig.GetFramework(MicrosoftNETCoreApp)
+                        .WithRollForward(rollForward)
+                        .Version = versionReference),
+                rollForwardToPreRelease).ShouldHaveResolvedFrameworkOrFail(
+                    MicrosoftNETCoreApp, resolvedFramework, versionReference, "6.1.1-preview.0");
+        }
+
+        // Verify that inner framework reference (<fxRefVersion>, <rollForward>)
+        // is correctly reconciled with app's framework reference 6.1.0 (defaults = RollForward:Minor).
+        // Also validates the effect of DOTNET_ROLL_FORWARD_TO_PRERELEASE on the result.
+        [Theory] // fxRefVersion       rollForward                               rollForwadToPreRelease resolvedFramework
+        [InlineData("6.0.0",           null,                                     false,                 "6.1.0")]
+        [InlineData("6.0.0",           null,                                     true,                  "6.1.1-preview.2")]
+        [InlineData("6.0.0",           Constants.RollForwardSetting.LatestPatch, false,                 ResolvedFramework.FailedToReconcile)]
+        [InlineData("6.0.0",           Constants.RollForwardSetting.Minor,       false,                 "6.1.0")]
+        [InlineData("6.0.0",           Constants.RollForwardSetting.Minor,       true,                  "6.1.1-preview.2")]
+        [InlineData("6.0.1-preview.0", Constants.RollForwardSetting.LatestPatch, false,                 ResolvedFramework.FailedToReconcile)]
+        [InlineData("6.1.0",           null,                                     false,                 "6.1.0")]
+        [InlineData("6.1.0",           null,                                     true,                  "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.0", null,                                     false,                 "6.2.1")]
+        [InlineData("6.1.1-preview.0", null,                                     true,                  "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.0", Constants.RollForwardSetting.Disable,     false,                 ResolvedFramework.NotFound)]
+        [InlineData("6.1.1-preview.0", Constants.RollForwardSetting.LatestPatch, false,                 "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.Disable,     false,                 "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.Disable,     true,                  "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.LatestPatch, false,                 "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.LatestPatch, true,                  "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", null,                                     false,                 "6.2.1")]
+        [InlineData("6.1.1-preview.2", null,                                     true,                  "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.Minor,       false,                 "6.2.1")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.Minor,       true,                  "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.LatestMinor, false,                 "6.2.1")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.LatestMinor, true,                  "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.Major,       false,                 "6.2.1")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.Major,       true,                  "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.LatestMajor, false,                 "6.2.1")]
+        [InlineData("6.1.1-preview.2", Constants.RollForwardSetting.LatestMajor, true,                  "6.1.1-preview.2")]
+        [InlineData("6.2.1-preview.1", Constants.RollForwardSetting.Disable,     false,                 ResolvedFramework.NotFound)]
+        [InlineData("6.2.1-preview.1", Constants.RollForwardSetting.LatestPatch, false,                 "6.2.1")]
+        [InlineData("6.2.1-preview.1", null,                                     false,                 "6.2.1")]
+        [InlineData("6.2.1-preview.1", Constants.RollForwardSetting.Minor,       false,                 "6.2.1")]
+        [InlineData("6.2.1-preview.1", Constants.RollForwardSetting.LatestMinor, false,                 "6.2.1")]
+        [InlineData("6.2.1-preview.1", Constants.RollForwardSetting.Major,       false,                 "6.2.1")]
+        [InlineData("6.2.1-preview.1", Constants.RollForwardSetting.LatestMajor, false,                 "6.2.1")]
+        public void ReconcileFrameworkReferences_InnerFrameworkReference_Release(
+            string versionReference,
+            string rollForward,
+            bool rollForwardToPreRelease,
+            string resolvedFramework)
+        {
+            RunTest(
+                runtimeConfig => runtimeConfig
+                    .WithFramework(MicrosoftNETCoreApp, "6.1.0")
+                    .WithFramework(MiddleWare, "2.1.0"),
+                dotnetCustomizer => dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
+                    runtimeConfig.GetFramework(MicrosoftNETCoreApp)
+                        .WithRollForward(rollForward)
+                        .Version = versionReference),
+                rollForwardToPreRelease)
+                .ShouldHaveResolvedFrameworkOrFail(
+                    MicrosoftNETCoreApp, resolvedFramework, versionReference, "6.1.0");
+        }
+
+        // Verify that inner framework reference 5.1.1 (defaults = RollForward:Minor)
+        // is correctly reconciled with app's framework reference (<fxRefVersion>, <rollForward>).
+        // App fx reference is lower.
+        [Theory] // fxRefVersion  rollForward                               resolvedFramework
+        [InlineData("5.0.0",      Constants.RollForwardSetting.Disable,     ResolvedFramework.FailedToReconcile)]
+        [InlineData("5.1.1",      Constants.RollForwardSetting.Disable,     "5.1.1")]
+        [InlineData("5.1.3",      Constants.RollForwardSetting.Disable,     "5.1.3")]
+        [InlineData("5.0.0",      Constants.RollForwardSetting.LatestPatch, ResolvedFramework.FailedToReconcile)]
+        [InlineData("5.1.0",      Constants.RollForwardSetting.LatestPatch, "5.1.3")]
+        [InlineData("5.1.1",      Constants.RollForwardSetting.LatestPatch, "5.1.3")]
+        [InlineData("5.0.0",      null,                                     "5.1.3")]
+        [InlineData("5.1.1",      null,                                     "5.1.3")]
+        [InlineData("5.0.0",      Constants.RollForwardSetting.Minor,       "5.1.3")]
+        [InlineData("5.1.1",      Constants.RollForwardSetting.Minor,       "5.1.3")]
+        [InlineData("5.0.0",      Constants.RollForwardSetting.LatestMinor, "5.1.3")]
+        [InlineData("5.1.1",      Constants.RollForwardSetting.LatestMinor, "5.1.3")]
+        [InlineData("1.0.0",      Constants.RollForwardSetting.Minor,       ResolvedFramework.FailedToReconcile)]
+        [InlineData("1.0.0",      Constants.RollForwardSetting.Major,       "5.1.3")]
+        [InlineData("5.1.1",      Constants.RollForwardSetting.Major,       "5.1.3")]
+        [InlineData("1.0.0",      Constants.RollForwardSetting.LatestMajor, "5.1.3")]
+        [InlineData("5.1.1",      Constants.RollForwardSetting.LatestMajor, "5.1.3")]
+        public void ReconcileFrameworkReferences_AppFrameworkReference_ToLower(
+            string versionReference,
+            string rollForward,
+            string resolvedFramework)
+        {
+            RunTest(
+                runtimeConfig => runtimeConfig
+                    .WithFramework(MiddleWare, "2.1.0")
+                    .WithFramework(new RuntimeConfig.Framework(MicrosoftNETCoreApp, versionReference)
+                        .WithRollForward(rollForward)),
+                dotnetCustomizer => dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
+                    runtimeConfig.GetFramework(MicrosoftNETCoreApp)
+                        .Version = "5.1.1"))
+                .ShouldHaveResolvedFrameworkOrFailedToReconcileFrameworkReference(
+                    MicrosoftNETCoreApp, resolvedFramework, versionReference, "5.1.1");
+        }
+
+        // Verify that inner framework reference 5.1.1 (defaults = RollForward:Minor)
+        // is correctly reconciled with app's framework reference (<fxRefVersion>, <rollForward>).
+        // App fx reference is lower.
+        // In this case the direct reference from app is first, so the framework reference from app
+        // is actually resolved against the disk - and the resolved framework is than compared to
+        // the inner framework reference (potentially causing re-resolution).
+        [Theory] // fxRefVersion  rollForward                               resolvedFramework
+        [InlineData("5.0.0",      Constants.RollForwardSetting.Disable,     ResolvedFramework.NotFound)]
+        [InlineData("5.1.1",      Constants.RollForwardSetting.Disable,     "5.1.1")]
+        [InlineData("5.1.3",      Constants.RollForwardSetting.Disable,     "5.1.3")]
+        [InlineData("5.0.0",      Constants.RollForwardSetting.LatestPatch, ResolvedFramework.NotFound)]
+        [InlineData("5.1.0",      Constants.RollForwardSetting.LatestPatch, "5.1.3")]
+        [InlineData("5.1.1",      Constants.RollForwardSetting.LatestPatch, "5.1.3")]
+        [InlineData("5.0.0",      null,                                     "5.1.3")]
+        [InlineData("5.1.1",      null,                                     "5.1.3")]
+        [InlineData("5.0.0",      Constants.RollForwardSetting.Minor,       "5.1.3")]
+        [InlineData("5.1.1",      Constants.RollForwardSetting.Minor,       "5.1.3")]
+        [InlineData("5.0.0",      Constants.RollForwardSetting.LatestMinor, "5.1.3")]
+        [InlineData("5.1.1",      Constants.RollForwardSetting.LatestMinor, "5.1.3")]
+        [InlineData("1.0.0",      Constants.RollForwardSetting.Minor,       ResolvedFramework.NotFound)]
+        [InlineData("1.0.0",      Constants.RollForwardSetting.Major,       "5.1.3")]
+        [InlineData("1.0.0",      Constants.RollForwardSetting.LatestMajor, "5.1.3")]
+        public void ReconcileFrameworkReferences_AppFrameworkReference_ToLower_HardResolve(
+            string versionReference,
+            string rollForward,
+            string resolvedFramework)
+        {
+            RunTest(
+                runtimeConfig => runtimeConfig
+                    .WithFramework(new RuntimeConfig.Framework(MicrosoftNETCoreApp, versionReference)
+                        .WithRollForward(rollForward))
+                    .WithFramework(MiddleWare, "2.1.0"),
+                dotnetCustomizer => dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
+                    runtimeConfig.GetFramework(MicrosoftNETCoreApp)
+                        .Version = "5.1.1"))
+                // Note that in this case (since the app reference is first) if the app's framework reference
+                // can't be resolved against the available frameworks, the error is actually a regular
+                // "can't find framework" and not a framework reconcile event.
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verify that inner framework reference 5.1.1 (defaults = RollForward:Minor)
+        // is correctly reconciled with app's framework reference (<fxRefVersion>, <rollForward>).
+        // App fx reference is higher.
+        [Theory] // fxRefVersion  rollForward                               resolvedFramework
+        [InlineData("5.4.0",      null,                                     "5.4.1")]
+        [InlineData("5.4.0",      Constants.RollForwardSetting.Minor,       "5.4.1")]
+        [InlineData("5.4.0",      Constants.RollForwardSetting.LatestMinor, "5.4.1")]
+        [InlineData("5.4.0",      Constants.RollForwardSetting.Major,       "5.4.1")]
+        [InlineData("5.4.0",      Constants.RollForwardSetting.LatestMajor, "5.4.1")]
+        [InlineData("5.4.1",      Constants.RollForwardSetting.Disable,     "5.4.1")]
+        [InlineData("6.0.0",      Constants.RollForwardSetting.Minor,       ResolvedFramework.FailedToReconcile)]
+        [InlineData("6.0.0",      Constants.RollForwardSetting.Major,       ResolvedFramework.FailedToReconcile)]
+        public void ReconcileFrameworkReferences_AppFrameworkReference_ToHigher(
+            string versionReference,
+            string rollForward,
+            string resolvedFramework)
+        {
+            RunTest(
+                runtimeConfig => runtimeConfig
+                    .WithFramework(MiddleWare, "2.1.0")
+                    .WithFramework(new RuntimeConfig.Framework(MicrosoftNETCoreApp, versionReference)
+                        .WithRollForward(rollForward)),
+                dotnetCustomizer => dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
+                    runtimeConfig.GetFramework(MicrosoftNETCoreApp)
+                        .Version = "5.1.1"))
+                .ShouldHaveResolvedFrameworkOrFailedToReconcileFrameworkReference(
+                    MicrosoftNETCoreApp, resolvedFramework, "5.1.1", versionReference);
+        }
+
+        // Verify that inner framework reference 5.1.1 (defaults = RollForward:Minor)
+        // is correctly reconciled with app's framework reference (<fxRefVersion>, <rollForward>).
+        // App fx reference is higher.
+        // In this case the direct reference from app is first, so the framework reference from app
+        // is actually resolved against the disk - and the resolved framework is than compared to
+        // the inner framework reference (potentially causing re-resolution).
+        [Theory] // fxRefVersion  rollForward                               resolvedFramework
+        [InlineData("5.4.0",      null,                                     "5.4.1")]
+        [InlineData("5.4.0",      Constants.RollForwardSetting.Minor,       "5.4.1")]
+        [InlineData("5.4.0",      Constants.RollForwardSetting.LatestMinor, "5.4.1")]
+        [InlineData("5.4.0",      Constants.RollForwardSetting.Major,       "5.4.1")]
+        [InlineData("5.4.0",      Constants.RollForwardSetting.LatestMajor, "5.4.1")]
+        [InlineData("5.4.1",      Constants.RollForwardSetting.Disable,     "5.4.1")]
+        [InlineData("6.0.0",      Constants.RollForwardSetting.Minor,       ResolvedFramework.FailedToReconcile)]
+        [InlineData("6.0.0",      Constants.RollForwardSetting.Major,       ResolvedFramework.FailedToReconcile)]
+        public void ReconcileFrameworkReferences_AppFrameworkReference_ToHigher_HardResolve(
+            string versionReference,
+            string rollForward,
+            string resolvedFramework)
+        {
+            RunTest(
+                runtimeConfig => runtimeConfig
+                    .WithFramework(new RuntimeConfig.Framework(MicrosoftNETCoreApp, versionReference)
+                        .WithRollForward(rollForward))
+                    .WithFramework(MiddleWare, "2.1.0"),
+                dotnetCustomizer => dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
+                    runtimeConfig.GetFramework(MicrosoftNETCoreApp)
+                        .Version = "5.1.1"))
+                .ShouldHaveResolvedFrameworkOrFailedToReconcileFrameworkReference(
+                    MicrosoftNETCoreApp, resolvedFramework, "5.1.1", versionReference);
+        }
+
+        // Verify that inner framework reference 5.1.1 (defaults = RollForward:Minor)
+        // is correctly reconciled with another framework's framework reference (<fxRefVersion>, <rollForward>).
+        // The higher framework has fx reference with higher version.
+        [Theory] // fxRefVersion  rollForward                               resolvedFramework
+        [InlineData("5.0.0",      Constants.RollForwardSetting.Disable,     ResolvedFramework.FailedToReconcile)]
+        [InlineData("5.1.1",      Constants.RollForwardSetting.Disable,     "5.1.1")]
+        [InlineData("5.0.0",      Constants.RollForwardSetting.LatestPatch, ResolvedFramework.FailedToReconcile)]
+        [InlineData("5.1.0",      Constants.RollForwardSetting.LatestPatch, "5.1.3")]
+        [InlineData("5.1.1",      Constants.RollForwardSetting.LatestPatch, "5.1.3")]
+        [InlineData("5.0.0",      null,                                     "5.1.3")]
+        [InlineData("5.1.1",      null,                                     "5.1.3")]
+        [InlineData("5.0.0",      Constants.RollForwardSetting.Minor,       "5.1.3")]
+        [InlineData("5.1.1",      Constants.RollForwardSetting.Minor,       "5.1.3")]
+        [InlineData("5.0.0",      Constants.RollForwardSetting.LatestMinor, "5.1.3")]
+        [InlineData("5.1.1",      Constants.RollForwardSetting.LatestMinor, "5.1.3")]
+        [InlineData("1.0.0",      Constants.RollForwardSetting.Minor,       ResolvedFramework.FailedToReconcile)]
+        [InlineData("1.0.0",      Constants.RollForwardSetting.Major,       "5.1.3")]
+        [InlineData("1.0.0",      Constants.RollForwardSetting.LatestMajor, "5.1.3")]
+        public void ReconcileFrameworkReferences_InnerToInnerFrameworkReference_ToLower(
+            string versionReference,
+            string rollForward,
+            string resolvedFramework)
+        {
+            RunTest(
+                runtimeConfig => runtimeConfig
+                    .WithFramework(HighWare, "7.0.0"),
+                dotnetCustomizer =>
+                {
+                    dotnetCustomizer.Framework(HighWare).RuntimeConfig(runtimeConfig =>
+                        runtimeConfig.GetFramework(MicrosoftNETCoreApp)
+                            .Version = "5.1.1");
+                    dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
+                        runtimeConfig.GetFramework(MicrosoftNETCoreApp)
+                            .WithRollForward(rollForward)
+                            .Version = versionReference);
+                })
+                .ShouldHaveResolvedFrameworkOrFailedToReconcileFrameworkReference(
+                    MicrosoftNETCoreApp, resolvedFramework, versionReference, "5.1.1");
+        }
+
+        // Verify that inner framework reference 5.1.1 (defaults = RollForward:Minor)
+        // is correctly reconciled with another framework's framework reference (<fxRefVersion>, <rollForward>).
+        // The higher framework has fx reference with lower version.
+        [Theory] // fxRefVersion  rollForward                               resolvedFramework
+        [InlineData("5.1.3",      Constants.RollForwardSetting.Disable,     "5.1.3")]
+        [InlineData("5.4.0",      null,                                     "5.4.1")]
+        [InlineData("5.4.0",      Constants.RollForwardSetting.Minor,       "5.4.1")]
+        [InlineData("5.4.0",      Constants.RollForwardSetting.LatestMinor, "5.4.1")]
+        [InlineData("5.4.0",      Constants.RollForwardSetting.Major,       "5.4.1")]
+        [InlineData("5.4.0",      Constants.RollForwardSetting.LatestMajor, "5.4.1")]
+        [InlineData("5.4.1",      Constants.RollForwardSetting.Disable,     "5.4.1")]
+        [InlineData("6.0.0",      Constants.RollForwardSetting.Minor,       ResolvedFramework.FailedToReconcile)]
+        [InlineData("6.0.0",      Constants.RollForwardSetting.Major,       ResolvedFramework.FailedToReconcile)]
+        public void ReconcileFrameworkReferences_InnerToInnerFrameworkReference_ToHigher(
+            string versionReference,
+            string rollForward,
+            string resolvedFramework)
+        {
+            RunTest(
+                runtimeConfig => runtimeConfig
+                    .WithFramework(HighWare, "7.0.0"),
+                dotnetCustomizer =>
+                {
+                    dotnetCustomizer.Framework(HighWare).RuntimeConfig(runtimeConfig =>
+                        runtimeConfig.GetFramework(MicrosoftNETCoreApp)
+                            .Version = "5.1.1");
+                    dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
+                        runtimeConfig.GetFramework(MicrosoftNETCoreApp)
+                            .WithRollForward(rollForward)
+                            .Version = versionReference);
+                })
+                .ShouldHaveResolvedFrameworkOrFailedToReconcileFrameworkReference(
+                    MicrosoftNETCoreApp, resolvedFramework, "5.1.1", versionReference);
+        }
+
+        // This test:
+        //  - Forces hard resolve of 5.1.1 -> 5.1.3 (direct reference from app)
+        //  - Loads HighWare which has 5.4.1 
+        //    - This forces a retry since 5.1.3 was hard resolved, so we have reload with 5.4.1 instead
+        //  - Loads MiddleWare which has 5.6.0
+        //    - This forces a retry since by this time 5.4.1 was hard resolved, so we have to reload with 5.6.0 instead
+        [Fact]
+        public void FrameworkResolutionRetry_FrameworkChain()
+        {
+            RunTest(
+                runtimeConfig => runtimeConfig
+                    .WithRollForward(Constants.RollForwardSetting.Major)
+                    .WithFramework(MicrosoftNETCoreApp, "5.1.1")
+                    .WithFramework(HighWare, "7.3.1"),
+                dotnetCustomizer =>
+                {
+                    dotnetCustomizer.Framework(HighWare).RuntimeConfig(runtimeConfig =>
+                        runtimeConfig.GetFramework(MicrosoftNETCoreApp)
+                            .Version = "5.4.1");
+                    dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
+                        runtimeConfig.GetFramework(MicrosoftNETCoreApp)
+                            .Version = "5.6.0");
+                })
+                .Should().Pass()
+                .And.RestartedFrameworkResolution("5.1.1", "5.4.1")
+                .And.RestartedFrameworkResolution("5.4.1", "5.6.0")
+                .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.6.0");
+        }
+
+        // This test:
+        //  - Forces hard resolve of 5.1.1 -> 5.1.3 (direct reference from app)
+        //  - Loads MiddleWare which has 5.4.1 
+        //    - This forces a retry since 5.1.3 was hard resolved, so we have reload with 5.4.1 instead
+        //  - Loads AnotherMiddleWare which has 5.6.0
+        //    - This forces a retry since by this time 5.4.1 was hard resolved, so we have to reload with 5.6.0 instead
+        [Fact]
+        public void FrameworkResolutionRetry_FrameworkTree()
+        {
+            RunTest(
+                runtimeConfig => runtimeConfig
+                    .WithRollForward(Constants.RollForwardSetting.Major)
+                    .WithFramework(MicrosoftNETCoreApp, "5.1.1")
+                    .WithFramework(MiddleWare, "2.1.2")
+                    .WithFramework(AnotherMiddleWare, "3.0.0"),
+                dotnetCustomizer =>
+                {
+                    dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
+                        runtimeConfig.GetFramework(MicrosoftNETCoreApp)
+                            .Version = "5.4.1");
+                    dotnetCustomizer.Framework(AnotherMiddleWare).RuntimeConfig(runtimeConfig =>
+                        runtimeConfig.GetFramework(MicrosoftNETCoreApp)
+                            .Version = "5.6.0");
+                })
+                .Should().Pass()
+                .And.RestartedFrameworkResolution("5.1.1", "5.4.1")
+                .And.RestartedFrameworkResolution("5.4.1", "5.6.0")
+                .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.6.0");
+        }
+
+        // Verifies that reconciling framework references correctly remembers whether it should prefer release versions or not.
+        [Theory]
+        [InlineData("6.0.0",           "6.1.1-preview.0", "6.2.1")]           // Release should prefer release even if there's a pre-release in the middle
+        [InlineData("6.1.0",           "6.1.1-preview.0", "6.2.1")]           // Release should prefer release even if there's a pre-release in the middle
+        [InlineData("6.1.1",           "6.1.1-preview.0", "6.2.1")]           // Release should prefer release even if there's a pre-release in the middle
+        [InlineData("6.0.0-preview.1", "6.1.1-preview.0", "6.1.1-preview.2")] // Both pre-relelase, take the closest even if it's pre-release
+        [InlineData("6.1.0-preview.0", "6.1.1",           "6.2.1")]           // Release should prefer release
+        [InlineData("6.1.1-preview.0", "6.1.0",           "6.2.1")]           // Release should prefer release
+        [InlineData("6.1.1-preview.0", "6.1.1",           "6.2.1")]           // Release should prefer release
+        public void PreferReleaseToRelease(string appVersionReference, string frameworkVersionReference, string resolvedFramework)
+        {
+            RunTest(
+                runtimeConfig => runtimeConfig
+                    .WithFramework(MiddleWare, "2.1.2")
+                    .WithFramework(MicrosoftNETCoreApp, appVersionReference),
+                dotnetCustomizer =>
+                {
+                    dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
+                        runtimeConfig.GetFramework(MicrosoftNETCoreApp)
+                            .Version = frameworkVersionReference);
+                })
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verify that inner framework reference (<fxRefVersion>, <fxRollForward>)
+        // is correctly reconciled with app's framework reference (<appRefVersion>, <appRollForward>).
+        // It then also tests it the other way round (as the result should not depend on which setting comes from FX and which from app)
+        // In this case the direct reference from app is first, so the framework reference from app
+        // is actually resolved against the disk - and the resolved framework is than compared to
+        // the inner framework reference (potentially causing re-resolution).
+        // This is mostly a collection of interesting cases as testing the full matrix is prohibitively large
+        [Theory] // appRefVersion appRollForward                            fxRefVersion fxRollForward                             resolvedFramework
+        // Disable + anything -> Disable
+        [InlineData("5.1.0",      Constants.RollForwardSetting.Disable,     "5.1.0",     Constants.RollForwardSetting.Disable,     ResolvedFramework.NotFound)]
+        [InlineData("5.1.0",      Constants.RollForwardSetting.Disable,     "5.1.0",     Constants.RollForwardSetting.LatestPatch, ResolvedFramework.NotFound)]
+        [InlineData("5.1.0",      Constants.RollForwardSetting.Disable,     "5.1.0",     Constants.RollForwardSetting.Minor,       ResolvedFramework.NotFound)]
+        [InlineData("5.1.0",      Constants.RollForwardSetting.Disable,     "5.1.0",     Constants.RollForwardSetting.LatestMinor, ResolvedFramework.NotFound)]
+        [InlineData("5.1.0",      Constants.RollForwardSetting.Disable,     "5.1.0",     Constants.RollForwardSetting.Major,       ResolvedFramework.NotFound)]
+        [InlineData("5.1.0",      Constants.RollForwardSetting.Disable,     "5.1.0",     Constants.RollForwardSetting.LatestMajor, ResolvedFramework.NotFound)]
+        // Default - should apply normal Minor semantics
+        [InlineData("5.0.0",      null,                                     "5.0.0",     null,                                     "5.1.3")]
+        // Default + LatestPatch -> LatestPatch
+        [InlineData("5.0.0",      null,                                     "5.0.0",     Constants.RollForwardSetting.LatestPatch, ResolvedFramework.NotFound)]
+        // Default + LatestMinor -> Minor
+        [InlineData("5.0.0",      null,                                     "5.0.0",     Constants.RollForwardSetting.LatestMinor, "5.1.3")]
+        // Default + Major -> Minor
+        [InlineData("5.0.0",      null,                                     "5.0.0",     Constants.RollForwardSetting.Major,       "5.1.3")]
+        // Default + LatestMajor -> Minor
+        [InlineData("5.0.0",      null,                                     "5.0.0",     Constants.RollForwardSetting.LatestMajor, "5.1.3")]
+        // LatestMinor + Major -> for now picks the most restrictive and thus Minor behavior
+        [InlineData("5.0.0",      Constants.RollForwardSetting.LatestMinor, "5.0.0",     Constants.RollForwardSetting.Major,       "5.1.3")]
+        // LatestMinor + LatestMajor -> LatestMinor
+        [InlineData("5.0.0",      Constants.RollForwardSetting.LatestMinor, "5.0.0",     Constants.RollForwardSetting.LatestMajor, "5.6.0")]
+        // LatestMajor + Major -> Major
+        [InlineData("4.0.0",      Constants.RollForwardSetting.LatestMajor, "4.0.0",     Constants.RollForwardSetting.Major,       "5.1.3")]
+        // LatestMajor + Minor -> Minor
+        [InlineData("4.0.0",      Constants.RollForwardSetting.LatestMajor, "4.0.0",     Constants.RollForwardSetting.Minor,       ResolvedFramework.NotFound)]
+        // LatestMinor + LatestPatch -> LatestPatch
+        [InlineData("5.1.0",      Constants.RollForwardSetting.LatestMinor, "5.1.0",     Constants.RollForwardSetting.LatestPatch, "5.1.3")]
+        [InlineData("5.0.0",      Constants.RollForwardSetting.LatestMinor, "5.0.0",     Constants.RollForwardSetting.LatestPatch, ResolvedFramework.NotFound)]
+        // LatestMajor + LatestPatch -> LatestPatch
+        [InlineData("5.1.0",      Constants.RollForwardSetting.LatestMajor, "5.1.0",     Constants.RollForwardSetting.LatestPatch, "5.1.3")]
+        [InlineData("5.0.0",      Constants.RollForwardSetting.LatestMajor, "5.0.0",     Constants.RollForwardSetting.LatestPatch, ResolvedFramework.NotFound)]
+        public void ReconcileFrameworkReferences_MergeRollForward(
+            string appVersionReference,
+            string appRollForward,
+            string fxVersionReference,
+            string fxRollForward,
+            string resolvedFramework)
+        {
+            RunTest(
+                runtimeConfig => runtimeConfig
+                    .WithFramework(new RuntimeConfig.Framework(MicrosoftNETCoreApp, appVersionReference)
+                        .WithRollForward(appRollForward))
+                    .WithFramework(MiddleWare, "2.1.0"),
+                dotnetCustomizer => dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
+                    runtimeConfig.GetFramework(MicrosoftNETCoreApp)
+                        .WithRollForward(fxRollForward)
+                        .Version = fxVersionReference))
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+
+            RunTest(
+                runtimeConfig => runtimeConfig
+                    .WithFramework(new RuntimeConfig.Framework(MicrosoftNETCoreApp, fxVersionReference)
+                        .WithRollForward(fxRollForward))
+                    .WithFramework(MiddleWare, "2.1.0"),
+                dotnetCustomizer => dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
+                    runtimeConfig.GetFramework(MicrosoftNETCoreApp)
+                        .WithRollForward(appRollForward)
+                        .Version = appVersionReference))
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        private CommandResult RunTest(
+            Func<RuntimeConfig, RuntimeConfig> runtimeConfig,
+            Action<DotNetCliExtensions.DotNetCliCustomizer> customizeDotNet = null,
+            bool rollForwardToPreRelease = false)
+        {
+            return RunTest(
+                SharedState.DotNetWithMultipleFrameworks,
+                SharedState.FrameworkReferenceApp,
+                new TestSettings()
+                    .WithRuntimeConfigCustomizer(runtimeConfig)
+                    .WithDotnetCustomizer(customizeDotNet)
+                    .WithEnvironment(Constants.RollForwardToPreRelease.EnvironmentVariable, rollForwardToPreRelease ? "1" : "0"));
+        }
+    }
+}
index 4a46596..68f3cc8 100644 (file)
@@ -20,116 +20,161 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
             SharedState = sharedState;
         }
 
+        public class SharedTestState : SharedTestStateBase
+        {
+            public TestApp FrameworkReferenceApp { get; }
+
+            public DotNetCli DotNetWithOneFramework { get; }
+
+            public DotNetCli DotNetWithPreReleaseFramework { get; }
+
+            public DotNetCli DotNetWithManyVersions { get; }
+
+            public SharedTestState()
+            {
+                DotNetWithOneFramework = DotNet("WithOneFramework")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.3")
+                    .Build();
+
+                DotNetWithPreReleaseFramework = DotNet("WithPreReleaseFramework")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.3-preview.2")
+                    .Build();
+
+                DotNetWithManyVersions = DotNet("WithManyVersions")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("2.3.1-preview.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("2.3.2")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("4.1.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("4.1.2")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("4.1.3-preview.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("4.2.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("4.5.1-preview.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("4.5.2")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.3-preview.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.3-preview.2")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.4-preview.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.2.3-preview.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.2.3-preview.2")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("6.1.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("6.1.2-preview.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("7.1.1-preview.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("7.1.2-preview.1")
+                    .Build();
+
+                FrameworkReferenceApp = CreateFrameworkReferenceApp();
+            }
+        }
+
         #region With one release framework
         // RunTestWithOneFramework
         //   dotnet with
         //     - Microsoft.NETCore.App 5.1.3
 
+        // Verifies that exact match for release version picks that version by default.
         [Fact]
         public void ExactMatchOnRelease_NoSettings()
         {
             RunTestWithOneFramework(
                 runtimeConfig => runtimeConfig
-                    .WithFramework(MicrosoftNETCoreApp, "5.1.3"),
-                commandResult => commandResult.Should().Pass()
-                    .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3"));
-        }
-
-        [Theory]
-        [InlineData(null, null)]
-        [InlineData(0, null)]
-        [InlineData(1, null)]
-        [InlineData(1, false)]  // Rolls on patches even when applyPatches = false if rollForwardOnNoCandidateFx != 0, but only to the lowest higher
-        [InlineData(2, null)]
-        [InlineData(2, false)]  // Rolls on patches even when applyPatches = false if rollForwardOnNoCandidateFx != 0, but only to the lowest higher
+                    .WithFramework(MicrosoftNETCoreApp, "5.1.3"))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3");
+        }
+
+        // Verifies that rollForwardOnNoCandidateFx and applyPatches work as expected.
+        // Rolling from 5.1.0 to 5.1.3 version. So roll on patch version.
+        [Theory] // rollForwardOnNoCandidateFx  applyPatches
+        [InlineData(null,                       null)]
+        [InlineData(0,                          null)]
+        [InlineData(1,                          null)]
+        // Rolls on patches even when applyPatches = false if rollForwardOnNoCandidateFx != 0, but only to the lowest higher
+        [InlineData(1,                          false)]
+        [InlineData(2,                          null)]
+        // Rolls on patches even when applyPatches = false if rollForwardOnNoCandidateFx != 0, but only to the lowest higher
+        [InlineData(2,                          false)]
         public void RollForwardToLatestPatch_RollForwardOnNoCandidateFx(int? rollForwardOnNoCandidateFx, bool? applyPatches)
         {
             RunTestWithOneFramework(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                     .WithApplyPatches(applyPatches)
-                    .WithFramework(MicrosoftNETCoreApp, "5.1.0"),
-                commandResult => commandResult.Should().Pass()
-                    .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3"));
-        }
-
-        [Theory]
-        [InlineData(null, null, true)]
-        [InlineData(null, false, true)]
-        [InlineData(0, null, false)]
-        [InlineData(1, null, true)]
-        [InlineData(1, false, true)] // Rolls on patches even when applyPatches = false if rollForwardOnNoCandidateFx != 0, but only to the lowest higher
-        [InlineData(2, null, true)]
-        [InlineData(2, false, true)] // Rolls on patches even when applyPatches = false if rollForwardOnNoCandidateFx != 0, but only to the lowest higher
+                    .WithFramework(MicrosoftNETCoreApp, "5.1.0"))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3");
+        }
+
+        // Verifies that rollForwardOnNoCandidateFx and applyPatches work as expected.
+        // Rolling from 5.0.0 to 5.1.3 version. So roll on minor version.
+        [Theory] // rollForwardOnNoCandidateFx  applyPatches  passes
+        [InlineData(null,                       null,         true)]
+        [InlineData(null,                       false,        true)]
+        [InlineData(0,                          null,         false)]
+        [InlineData(1,                          null,         true)]
+        // Rolls on patches even when applyPatches = false if rollForwardOnNoCandidateFx != 0, but only to the lowest higher
+        [InlineData(1,                          false,        true)]
+        [InlineData(2,                          null,         true)]
+        // Rolls on patches even when applyPatches = false if rollForwardOnNoCandidateFx != 0, but only to the lowest higher
+        [InlineData(2,                          false,        true)]
         public void RollForwardOnMinor_RollForwardOnNoCandidateFx(int? rollForwardOnNoCandidateFx, bool? applyPatches, bool passes)
         {
-            RunTestWithOneFramework(
+            CommandResult result = RunTestWithOneFramework(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                     .WithApplyPatches(applyPatches)
-                    .WithFramework(MicrosoftNETCoreApp, "5.0.0"),
-                commandResult =>
-                {
-                    if (passes)
-                    {
-                        commandResult.Should().Pass()
-                            .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3");
-                    }
-                    else
-                    {
-                        commandResult.Should().Fail()
-                            .And.DidNotFindCompatibleFrameworkVersion();
-                    }
-                });
-        }
-
-        [Theory]
-        [InlineData(null, null, false)]
-        [InlineData(0, null, false)]
-        [InlineData(1, null, false)]
-        [InlineData(1, false, false)]
-        [InlineData(2, null, true)]
-        [InlineData(2, false, true)] // Rolls on patches even when applyPatches = false if rollForwardOnNoCandidateFx != 0, but only to the lowest higher
+                    .WithFramework(MicrosoftNETCoreApp, "5.0.0"));
+            if (passes)
+            {
+                result.ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3");
+            }
+            else
+            {
+                result.ShouldFailToFindCompatibleFrameworkVersion();
+            }
+        }
+
+        // Verifies that rollForwardOnNoCandidateFx and applyPatches work as expected.
+        // Rolling from 4.1.0 to 5.1.3 version. So roll on major version.
+        [Theory] // rollForwardOnNoCandidateFx  applyPatches  passes
+        [InlineData(null,                       null,         false)]
+        [InlineData(0,                          null,         false)]
+        [InlineData(1,                          null,         false)]
+        [InlineData(1,                          false,        false)]
+        [InlineData(2,                          null,         true)]
+        // Rolls on patches even when applyPatches = false if rollForwardOnNoCandidateFx != 0, but only to the lowest higher
+        [InlineData(2,                          false,        true)]
         public void RollForwardOnMajor_RollForwardOnNoCandidateFx(int? rollForwardOnNoCandidateFx, bool? applyPatches, bool passes)
         {
-            RunTestWithOneFramework(
+            CommandResult result = RunTestWithOneFramework(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                     .WithApplyPatches(applyPatches)
-                    .WithFramework(MicrosoftNETCoreApp, "4.1.0"),
-                commandResult =>
-                {
-                    if (passes)
-                    {
-                        commandResult.Should().Pass()
-                            .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3");
-                    }
-                    else
-                    {
-                        commandResult.Should().Fail()
-                            .And.DidNotFindCompatibleFrameworkVersion();
-                    }
-                });
-        }
-
-        [Theory]
-        [InlineData(0, null)]
-        [InlineData(0, false)]
-        [InlineData(1, null)]
-        [InlineData(1, false)]
-        [InlineData(2, null)]
-        [InlineData(2, false)]
+                    .WithFramework(MicrosoftNETCoreApp, "4.1.0"));
+            if (passes)
+            {
+                result.ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3");
+            }
+            else
+            {
+                result.ShouldFailToFindCompatibleFrameworkVersion();
+            }
+        }
+
+        // Verifies that no matter what setting the version will never roll back.
+        [Theory] // rollForwardOnNoCandidateFx  applyPatches
+        [InlineData(0,                          null)]
+        [InlineData(0,                          false)]
+        [InlineData(1,                          null)]
+        [InlineData(1,                          false)]
+        [InlineData(2,                          null)]
+        [InlineData(2,                          false)]
         public void NeverRollBackOnRelease(int? rollForwardOnNoCandidateFx, bool? applyPatches)
         {
             RunTestWithOneFramework(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                     .WithApplyPatches(applyPatches)
-                    .WithFramework(MicrosoftNETCoreApp, "5.1.4"),
-                commandResult => commandResult.Should().Fail()
-                    .And.DidNotFindCompatibleFrameworkVersion());
+                    .WithFramework(MicrosoftNETCoreApp, "5.1.4"))
+                .ShouldFailToFindCompatibleFrameworkVersion();
         }
 
+        // Verifies that if both rollForwardOnNoCandidateFx=0 and applyPatches=0 there will be no rolling forward.
         [Fact]
         public void RollForwardDisabledOnCandidateFxAndDisabledApplyPatches_FailsToRollPatches()
         {
@@ -137,11 +182,12 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(0)
                     .WithApplyPatches(false)
-                    .WithFramework(MicrosoftNETCoreApp, "5.1.0"),
-                commandResult => commandResult.Should().Fail()
-                    .And.HaveStdErrContaining("Did not roll forward because patch_roll_fwd=0, roll_fwd_on_no_candidate_fx=0, use_exact_version=0 chose [5.1.0]"));
+                    .WithFramework(MicrosoftNETCoreApp, "5.1.0"))
+                .Should().Fail()
+                .And.HaveStdErrContaining("Did not roll forward because apply_patches=0, roll_forward=LatestPatch chose [5.1.0]");
         }
 
+        // Verifies that if both rollForwardOnNoCandidateFx=0 and applyPatches=0 there can still resolve exact match
         [Fact]
         public void RollForwardDisabledOnCandidateFxAndDisabledApplyPatches_MatchesExact()
         {
@@ -149,39 +195,43 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(0)
                     .WithApplyPatches(false)
-                    .WithFramework(MicrosoftNETCoreApp, "5.1.3"),
-                commandResult => commandResult.Should().Pass()
-                    .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3"));
+                    .WithFramework(MicrosoftNETCoreApp, "5.1.3"))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3");
         }
 
+        // Verifies that if rollForwardOnNoCandidateFx=0 (and applyPatches=<default> that is true)
+        // the product will fail to roll on minor, but will try.
         [Fact]
         public void RollForwardOnMinorDisabledOnNoCandidateFx_FailsToRoll()
         {
             RunTestWithOneFramework(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(0)
-                    .WithFramework(MicrosoftNETCoreApp, "5.0.0"),
+                    .WithFramework(MicrosoftNETCoreApp, "5.0.0"))
                 // Will still attempt roll forward to latest patch
-                commandResult => commandResult.Should().Fail()
-                    .And.HaveStdErrContaining("Attempting FX roll forward")
-                    .And.DidNotFindCompatibleFrameworkVersion());
+                .Should().Fail()
+                .And.HaveStdErrContaining("Attempting FX roll forward")
+                .And.DidNotFindCompatibleFrameworkVersion();
         }
 
+        // 3.0 change: In 2.* pre-release never rolled to release. In 3.* it will follow normal roll-forward rules.
         [Fact]
-        public void PreReleaseReference_FailsToRollToRelease()
+        public void PreReleaseReference_CanRollToRelease()
         {
             RunTestWithOneFramework(
                 runtimeConfig => runtimeConfig
-                    .WithFramework(MicrosoftNETCoreApp, "5.1.0-preview.1"),
-                commandResult => commandResult.Should().Fail()
-                    .And.DidNotFindCompatibleFrameworkVersion());
+                    .WithFramework(MicrosoftNETCoreApp, "5.1.0-preview.1"))
+                .Should().Pass()
+                .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3");
         }
 
-        private void RunTestWithOneFramework(
-            Func<RuntimeConfig, RuntimeConfig> runtimeConfig,
-            Action<CommandResult> resultAction)
+        private CommandResult RunTestWithOneFramework(Func<RuntimeConfig, RuntimeConfig> runtimeConfig)
         {
-            RunTest(SharedState.DotNetWithOneFramework, runtimeConfig, resultAction);
+            return RunTest(
+                SharedState.DotNetWithOneFramework,
+                SharedState.FrameworkReferenceApp,
+                new TestSettings()
+                    .WithRuntimeConfigCustomizer(runtimeConfig));
         }
         #endregion
 
@@ -190,152 +240,153 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
         //   dotnet with
         //     - Microsoft.NETCore.App 5.1.3-preview.2
 
+        // Verifies that exact match for pre-release version picks that version by default.
         [Fact]
         public void ExactMatchOnPreRelease_NoSettings()
         {
             RunTestWithPreReleaseFramework(
                 runtimeConfig => runtimeConfig
-                    .WithFramework(MicrosoftNETCoreApp, "5.1.3-preview.2"),
-                commandResult => commandResult.Should().Pass()
-                    .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3-preview.2"));
+                    .WithFramework(MicrosoftNETCoreApp, "5.1.3-preview.2"))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3-preview.2");
         }
 
+        // 3.0 change:
+        // 2.* - Pre-Release only rolls on the exact same major.minor.patch (it only rolls over the pre-release portion of the version)
+        // 3.* - Pre-Release follows normal roll-forward rules, including rolling over patches
         [Fact]
-        public void RollForwardToPreRelease_FailsOnVersionMismatch()
+        public void RollForwardToPreRelease_CanRollOnPatch()
         {
             RunTestWithPreReleaseFramework(
                 runtimeConfig => runtimeConfig
-                    .WithFramework(MicrosoftNETCoreApp, "5.1.2-preview.2"),
-                commandResult => commandResult.Should().Fail()
-                    .And.DidNotFindCompatibleFrameworkVersion());
-        }
-
-        [Theory]
-        [InlineData(null, null)]
-        [InlineData(0, false)] // Pre-Release ignores roll forward on no candidate FX and apply patches settings
-        [InlineData(2, true)]
+                    .WithFramework(MicrosoftNETCoreApp, "5.1.2-preview.2"))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3-preview.2");
+        }
+
+        // Verifies that rollForwardOnNoCandidateFx and applyPatches work as expected.
+        // Rolling from 5.1.3-preview.1 to 5.1.3-preview.2 version. So roll on pre-release version.
+        [Theory] // rollForwardOnNoCandidateFx  applyPatches
+        [InlineData(null,                       null)]
+        // 3.0 change:
+        // 2.* - Pre-Release ignores rollForwardOnNoCandidateFX and applyPatches settings
+        // 3.* - Pre-Release follows normal roll-forward rules, including all the roll-forward settings
+        //   with the exception of applyPatches=false for pre-release roll.
+        [InlineData(0,                          false)]
+        [InlineData(2,                          true)]
         public void RollForwardToPreRelease(int? rollForwardOnNoCandidateFx, bool? applyPatches)
         {
             RunTestWithPreReleaseFramework(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                     .WithApplyPatches(applyPatches)
-                    .WithFramework(MicrosoftNETCoreApp, "5.1.3-preview.1"),
-                commandResult => commandResult.Should().Pass()
-                    .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3-preview.2"));
-        }
-
-        [Theory]
-        [InlineData(null, null, true)]
-        [InlineData(0, null, false)]  // Roll forward to pre-release on patch from release is blocked
-        [InlineData(1, null, true)]
-        [InlineData(1, false, true)]  // Rolls on patches even when applyPatches = false if rollForwardOnNoCandidateFx != 0, but only to the lowest higher
-        [InlineData(2, null, true)]
-        [InlineData(2, false, true)]  // Rolls on patches even when applyPatches = false if rollForwardOnNoCandidateFx != 0, but only to the lowest higher
-        public void RollForwardToPreReleaseLatestPatch_RollForwardOnNoCandidateFx(int? rollForwardOnNoCandidateFx, bool? applyPatches, bool passes)
+                    .WithFramework(MicrosoftNETCoreApp, "5.1.3-preview.1"))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3-preview.2");
+        }
+
+        // Verifies that rollForwardOnNoCandidateFx and applyPatches work as expected.
+        // Rolling from release 5.1.0 to pre-release 5.1.3-preview.2 version. So roll on patch version.
+        [Theory] // rollForwardOnNoCandidateFx  applyPatches
+        [InlineData(null,                       null)]
+        // This is a different behavior in 3.0. In 2.* the app would fail in this case as it was explicitly disallowed
+        // to roll forward from release to pre-release when rollForwardOnNoCandidateFx=0 (and only then).
+        [InlineData(0,                          null)]
+        [InlineData(1,                          null)]
+        [InlineData(1,                          false)]
+        [InlineData(2,                          null)]
+        [InlineData(2,                          false)]
+        public void RollForwardToPreReleaseLatestPatch_RollForwardOnNoCandidateFx(int? rollForwardOnNoCandidateFx, bool? applyPatches)
         {
             RunTestWithPreReleaseFramework(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                     .WithApplyPatches(applyPatches)
-                    .WithFramework(MicrosoftNETCoreApp, "5.1.0"),
-                commandResult =>
-                {
-                    if (passes)
-                    {
-                        commandResult.Should().Pass()
-                            .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3-preview.2");
-                    }
-                    else
-                    {
-                        commandResult.Should().Fail()
-                            .And.DidNotFindCompatibleFrameworkVersion();
-                    }
-                });
-        }
-
-        [Theory]
-        [InlineData(null, null, true)]
-        [InlineData(null, false, true)]
-        [InlineData(0, null, false)]
-        [InlineData(1, null, true)]
-        [InlineData(1, false, true)] // Rolls on patches even when applyPatches = false if rollForwardOnNoCandidateFx != 0, but only to the lowest higher
-        [InlineData(2, null, true)]
-        [InlineData(2, false, true)] // Rolls on patches even when applyPatches = false if rollForwardOnNoCandidateFx != 0, but only to the lowest higher
-        public void RollForwardToPreReleaseOnMinor_RollForwardOnNoCandidateFx(int? rollForwardOnNoCandidateFx, bool? applyPatches, bool passes)
-        {
-            RunTestWithPreReleaseFramework(
+                    .WithFramework(MicrosoftNETCoreApp, "5.1.0"))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3-preview.2");
+        }
+
+        // Verifies that rollForwardOnNoCandidateFx and applyPatches work as expected.
+        // Rolling from release 5.0.0 to pre-release 5.1.3-preview.2 version. So roll on minor version.
+        [Theory] // rollForwardOnNoCandidateFx  applyPatches  passes
+        [InlineData(null,                       null,         true)]
+        [InlineData(null,                       false,        true)]
+        [InlineData(0,                          null,         false)]
+        [InlineData(1,                          null,         true)]
+        [InlineData(1,                          false,        true)]
+        [InlineData(2,                          null,         true)]
+        [InlineData(2,                          false,        true)]
+        public void RollForwardToPreReleaseOnMinor_RollForwardOnNoCandidateFx(
+            int? rollForwardOnNoCandidateFx, 
+            bool? applyPatches, 
+            bool passes)
+        {
+            CommandResult result = RunTestWithPreReleaseFramework(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                     .WithApplyPatches(applyPatches)
-                    .WithFramework(MicrosoftNETCoreApp, "5.0.0"),
-                commandResult =>
-                {
-                    if (passes)
-                    {
-                        commandResult.Should().Pass()
-                            .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3-preview.2");
-                    }
-                    else
-                    {
-                        commandResult.Should().Fail()
-                            .And.DidNotFindCompatibleFrameworkVersion();
-                    }
-                });
-        }
-
-        [Theory]
-        [InlineData(null, null, false)]
-        [InlineData(0, null, false)]
-        [InlineData(1, null, false)]
-        [InlineData(1, false, false)]
-        [InlineData(2, null, true)]
-        [InlineData(2, false, true)] // Rolls on patches even when applyPatches = false if rollForwardOnNoCandidateFx != 0, but only to the lowest higher
-        public void RollForwardToPreReleaseOnMajor_RollForwardOnNoCandidateFx(int? rollForwardOnNoCandidateFx, bool? applyPatches, bool passes)
-        {
-            RunTestWithPreReleaseFramework(
+                    .WithFramework(MicrosoftNETCoreApp, "5.0.0"));
+            if (passes)
+            {
+                result.ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3-preview.2");
+            }
+            else
+            {
+                result.ShouldFailToFindCompatibleFrameworkVersion();
+            }
+        }
+
+        // Verifies that rollForwardOnNoCandidateFx and applyPatches work as expected.
+        // Rolling from release 4.1.0 to pre-release 5.1.3-preview.2 version. So roll on major version.
+        [Theory] // rollForwardOnNoCandidateFx  applyPatches  passes
+        [InlineData(null,                       null,         false)]
+        [InlineData(0,                          null,         false)]
+        [InlineData(1,                          null,         false)]
+        [InlineData(1,                          false,        false)]
+        [InlineData(2,                          null,         true)]
+        [InlineData(2,                          false,        true)]
+        public void RollForwardToPreReleaseOnMajor_RollForwardOnNoCandidateFx(
+            int? rollForwardOnNoCandidateFx,
+            bool? applyPatches,
+            bool passes)
+        {
+            CommandResult result = RunTestWithPreReleaseFramework(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                     .WithApplyPatches(applyPatches)
-                    .WithFramework(MicrosoftNETCoreApp, "4.1.0"),
-                commandResult =>
-                {
-                    if (passes)
-                    {
-                        commandResult.Should().Pass()
-                            .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3-preview.2");
-                    }
-                    else
-                    {
-                        commandResult.Should().Fail()
-                            .And.DidNotFindCompatibleFrameworkVersion();
-                    }
-                });
-        }
-
-        [Theory]
-        [InlineData(0, null)]
-        [InlineData(0, false)]
-        [InlineData(1, null)]
-        [InlineData(1, false)]
-        [InlineData(2, null)]
-        [InlineData(2, false)]
+                    .WithFramework(MicrosoftNETCoreApp, "4.1.0"));
+            if (passes)
+            {
+                result.ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3-preview.2");
+            }
+            else
+            {
+                result.ShouldFailToFindCompatibleFrameworkVersion();
+            }
+        }
+
+        // Verifies that the produce never rolls back even on pre-release versions
+        [Theory] // rollForwardOnNoCandidateFx  applyPatches
+        [InlineData(0,                          null)]
+        [InlineData(0,                          false)]
+        [InlineData(1,                          null)]
+        [InlineData(1,                          false)]
+        [InlineData(2,                          null)]
+        [InlineData(2,                          false)]
         public void NeverRollBackOnPreRelease(int? rollForwardOnNoCandidateFx, bool? applyPatches)
         {
             RunTestWithPreReleaseFramework(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                     .WithApplyPatches(applyPatches)
-                    .WithFramework(MicrosoftNETCoreApp, "5.1.3-preview.9"),
-                commandResult => commandResult.Should().Fail()
-                    .And.DidNotFindCompatibleFrameworkVersion());
+                    .WithFramework(MicrosoftNETCoreApp, "5.1.3-preview.9"))
+                .ShouldFailToFindCompatibleFrameworkVersion();
         }
 
-        private void RunTestWithPreReleaseFramework(
-            Func<RuntimeConfig, RuntimeConfig> runtimeConfig,
-            Action<CommandResult> resultAction)
+        private CommandResult RunTestWithPreReleaseFramework(Func<RuntimeConfig, RuntimeConfig> runtimeConfig)
         {
-            RunTest(SharedState.DotNetWithPreReleaseFramework, runtimeConfig, resultAction);
+            return RunTest(
+                SharedState.DotNetWithPreReleaseFramework,
+                SharedState.FrameworkReferenceApp,
+                new TestSettings()
+                    .WithRuntimeConfigCustomizer(runtimeConfig));
         }
         #endregion
 
@@ -359,350 +410,305 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
         //  - Microsoft.NETCore.App 7.1.1-preview.1
         //  - Microsoft.NETCore.App 7.1.2-preview.1
 
-        [Theory]
-        [InlineData(null, null, "4.1.2")]
-        [InlineData(null, false, "4.1.1")]
-        [InlineData(0, null, "4.1.2")]
-        [InlineData(0, false, "4.1.1")]  // No roll forward
-        [InlineData(1, null, "4.1.2")]
-        [InlineData(1, false, "4.1.1")]  // Doesn't roll to latest patch
-        [InlineData(2, null, "4.1.2")]
-        [InlineData(2, false, "4.1.1")]  // Doesn't roll to latest patch
-        public void RollForwardToLatestPatch_PicksLatestReleasePatch(int? rollForwardOnNoCandidateFx, bool? applyPatches, string resolvedVersion)
+        // Verifies that rollForwardOnNoCandidateFx and applyPatches settings correctly roll
+        // from a release version 4.1.1 to the latest patch if allowed.
+        [Theory] // rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData(null,                       null,         "4.1.2")]
+        [InlineData(null,                       false,        "4.1.1")]
+        [InlineData(0,                          null,         "4.1.2")]
+        [InlineData(0,                          false,        "4.1.1")]  // No roll forward
+        [InlineData(1,                          null,         "4.1.2")]
+        [InlineData(1,                          false,        "4.1.1")]  // Doesn't roll to latest patch
+        [InlineData(2,                          null,         "4.1.2")]
+        [InlineData(2,                          false,        "4.1.1")]  // Doesn't roll to latest patch
+        public void RollForwardToLatestPatch_PicksLatestReleasePatch(
+            int? rollForwardOnNoCandidateFx,
+            bool? applyPatches,
+            string resolvedFramework)
         {
             RunTestWithManyVersions(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                     .WithApplyPatches(applyPatches)
-                    .WithFramework(MicrosoftNETCoreApp, "4.1.1"),
-                commandResult => commandResult.Should().Pass()
-                    .And.HaveResolvedFramework(MicrosoftNETCoreApp, resolvedVersion));
-        }
-
-        [Theory]
-        [InlineData(null, null, "4.1.2")]
-        [InlineData(null, false, "4.1.1")]
-        [InlineData(0, null, null)]
-        [InlineData(0, false, null)]
-        [InlineData(1, null, "4.1.2")]
-        [InlineData(1, false, "4.1.1")]  // Rolls to nearest higher even on patches, but not to latest patch.
-        [InlineData(2, null, "4.1.2")]
-        [InlineData(2, false, "4.1.1")]  // Rolls to nearest higher even on patches, but not to latest patch.
-        public void RollForwardOnMinor_PicksLatestReleasePatch(int? rollForwardOnNoCandidateFx, bool? applyPatches, string resolvedVersion)
+                    .WithFramework(MicrosoftNETCoreApp, "4.1.1"))
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForwardOnNoCandidateFx and applyPatches settings correctly roll
+        // from a release version 4.0.0 the the closest minor version with the latest patch.
+        [Theory] // rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData(null,                       null,         "4.1.2")]
+        [InlineData(null,                       false,        "4.1.1")]
+        [InlineData(0,                          null,         ResolvedFramework.NotFound)]
+        [InlineData(0,                          false,        ResolvedFramework.NotFound)]
+        [InlineData(1,                          null,         "4.1.2")]
+        [InlineData(1,                          false,        "4.1.1")]  // Rolls to nearest higher even on patches, but not to latest patch.
+        [InlineData(2,                          null,         "4.1.2")]
+        [InlineData(2,                          false,        "4.1.1")]  // Rolls to nearest higher even on patches, but not to latest patch.
+        public void RollForwardOnMinor_PicksLatestReleasePatch(
+            int? rollForwardOnNoCandidateFx,
+            bool? applyPatches,
+            string resolvedFramework)
         {
             RunTestWithManyVersions(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                     .WithApplyPatches(applyPatches)
-                    .WithFramework(MicrosoftNETCoreApp, "4.0.0"),
-                commandResult =>
-                {
-                    if (resolvedVersion != null)
-                    {
-                        commandResult.Should().Pass()
-                            .And.HaveResolvedFramework(MicrosoftNETCoreApp, resolvedVersion);
-                    }
-                    else
-                    {
-                        commandResult.Should().Fail()
-                            .And.DidNotFindCompatibleFrameworkVersion();
-                    }
-                });
-        }
-
-        [Theory]
-        [InlineData(null, null, "4.5.2")]
-        [InlineData(null, false, "4.5.2")]
-        [InlineData(0, null, null)]
-        [InlineData(0, false, null)]
-        [InlineData(1, null, "4.5.2")]
-        [InlineData(1, false, "4.5.2")]
-        [InlineData(2, null, "4.5.2")]
-        [InlineData(2, false, "4.5.2")]
-        public void RollForwardOnMinor_RollOverPreRelease(int? rollForwardOnNoCandidateFx, bool? applyPatches, string resolvedVersion)
+                    .WithFramework(MicrosoftNETCoreApp, "4.0.0"))
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForwardOnNoCandidateFx and applyPatches settings correctly roll
+        // from a release version 4.4.0 over the pre-release 4.5.1-preview.1 to the closest release minor version with the latest patch.
+        [Theory] // rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData(null,                       null,         "4.5.2")]
+        [InlineData(null,                       false,        "4.5.2")]
+        [InlineData(0,                          null,         ResolvedFramework.NotFound)]
+        [InlineData(0,                          false,        ResolvedFramework.NotFound)]
+        [InlineData(1,                          null,         "4.5.2")]
+        [InlineData(1,                          false,        "4.5.2")]
+        [InlineData(2,                          null,         "4.5.2")]
+        [InlineData(2,                          false,        "4.5.2")]
+        public void RollForwardOnMinor_RollOverPreRelease(
+            int? rollForwardOnNoCandidateFx,
+            bool? applyPatches,
+            string resolvedFramework)
         {
             RunTestWithManyVersions(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                     .WithApplyPatches(applyPatches)
-                    .WithFramework(MicrosoftNETCoreApp, "4.4.0"),
-                commandResult =>
-                {
-                    if (resolvedVersion != null)
-                    {
-                        commandResult.Should().Pass()
-                            .And.HaveResolvedFramework(MicrosoftNETCoreApp, resolvedVersion);
-                    }
-                    else
-                    {
-                        commandResult.Should().Fail()
-                            .And.DidNotFindCompatibleFrameworkVersion();
-                    }
-                });
-        }
-
-        [Theory]
-        [InlineData(null, null, null)]
-        [InlineData(null, false, null)]
-        [InlineData(0, null, null)]
-        [InlineData(0, false, null)]
-        [InlineData(1, null, null)]
-        [InlineData(1, false, null)]
-        [InlineData(2, null, "4.1.2")]
-        [InlineData(2, false, "4.1.1")]  // Rolls to nearest higher even on patches, but not to latest patch.
-        public void RollForwardOnMajor_PicksLatestReleasePatch(int? rollForwardOnNoCandidateFx, bool? applyPatches, string resolvedVersion)
+                    .WithFramework(MicrosoftNETCoreApp, "4.4.0"))
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForwardOnNoCandidateFx and applyPatches settings correctly roll
+        // from a release version 3.0.0 to the closest release major version with the latest patch.
+        [Theory] // rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData(null,                       null,         ResolvedFramework.NotFound)]
+        [InlineData(null,                       false,        ResolvedFramework.NotFound)]
+        [InlineData(0,                          null,         ResolvedFramework.NotFound)]
+        [InlineData(0,                          false,        ResolvedFramework.NotFound)]
+        [InlineData(1,                          null,         ResolvedFramework.NotFound)]
+        [InlineData(1,                          false,        ResolvedFramework.NotFound)]
+        [InlineData(2,                          null,         "4.1.2")]
+        [InlineData(2,                          false,        "4.1.1")]  // Rolls to nearest higher even on patches, but not to latest patch.
+        public void RollForwardOnMajor_PicksLatestReleasePatch(
+            int? rollForwardOnNoCandidateFx,
+            bool? applyPatches,
+            string resolvedFramework)
         {
             RunTestWithManyVersions(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                     .WithApplyPatches(applyPatches)
-                    .WithFramework(MicrosoftNETCoreApp, "3.0.0"),
-                commandResult =>
-                {
-                    if (resolvedVersion != null)
-                    {
-                        commandResult.Should().Pass()
-                            .And.HaveResolvedFramework(MicrosoftNETCoreApp, resolvedVersion);
-                    }
-                    else
-                    {
-                        commandResult.Should().Fail()
-                            .And.DidNotFindCompatibleFrameworkVersion();
-                    }
-                });
-        }
-
-        [Theory]
-        [InlineData(null, null, "5.1.4-preview.1")]
-        [InlineData(null, false, "5.1.3-preview.1")]
-        [InlineData(0, null, null)]   // This is interesting - we prevent roll forward from release to preview on patch alone
-        [InlineData(0, false, null)]
-        [InlineData(1, null, "5.1.4-preview.1")]
-        [InlineData(1, false, "5.1.3-preview.1")]  // Rolls to nearest higher even on patches, but not to latest patch.
-        [InlineData(2, null, "6.1.1")]   // Not really testing the pre-release roll forward, but valid test anyway
-        [InlineData(2, false, "6.1.1")]  // Not really testing the pre-release roll forward, but valid test anyway
-        public void RollForwardToPreReleaseToLatestPatch_FromRelease(int? rollForwardOnNoCandidateFx, bool? applyPatches, string resolvedVersion)
+                    .WithFramework(MicrosoftNETCoreApp, "3.0.0"))
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForwardOnNoCandidateFx and applyPatches settings correctly roll
+        // from a release version 5.1.2 to the latest patch pre-release version (since there's no release available)
+        [Theory] // rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData(null,                       null,         "5.1.4-preview.1")]
+        [InlineData(null,                       false,        "5.1.3-preview.1")]
+        // This is a different behavior in 3.0. In 2.* the app would fail in this case as it was explicitly disallowed
+        // to roll forward from release to pre-release when rollForwardOnNoCandidateFx=0 (and only then).
+        [InlineData(0,                          null,         "5.1.4-preview.1")]
+        [InlineData(0,                          false,        ResolvedFramework.NotFound)]
+        [InlineData(1,                          null,         "5.1.4-preview.1")]
+        [InlineData(1,                          false,        "5.1.3-preview.1")]  // Rolls to nearest higher even on patches, but not to latest patch.
+        [InlineData(2,                          null,         "6.1.1")]   // Not really testing the pre-release roll forward, but valid test anyway
+        [InlineData(2,                          false,        "6.1.1")]  // Not really testing the pre-release roll forward, but valid test anyway
+        public void RollForwardToPreReleaseToLatestPatch_FromRelease(
+            int? rollForwardOnNoCandidateFx,
+            bool? applyPatches,
+            string resolvedFramework)
         {
             RunTestWithManyVersions(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                     .WithApplyPatches(applyPatches)
-                    .WithFramework(MicrosoftNETCoreApp, "5.1.2"),
-                commandResult =>
-                {
-                    if (resolvedVersion != null)
-                    {
-                        commandResult.Should().Pass()
-                            .And.HaveResolvedFramework(MicrosoftNETCoreApp, resolvedVersion);
-                    }
-                    else
-                    {
-                        commandResult.Should().Fail()
-                            .And.DidNotFindCompatibleFrameworkVersion();
-                    }
-                });
-        }
-
-        [Theory]
-        [InlineData(null, null, "5.1.4-preview.1")]
-        [InlineData(null, false, "5.1.3-preview.1")]
-        [InlineData(0, null, null)]
-        [InlineData(0, false, null)]
-        [InlineData(1, null, "5.1.4-preview.1")]
-        [InlineData(1, false, "5.1.3-preview.1")]   // Rolls to nearest higher even on patches, but not to latest patch.
-        [InlineData(2, null, "6.1.1")]   // Not really testing the pre-release roll forward, but valid test anyway
-        [InlineData(2, false, "6.1.1")]  // Not really testing the pre-release roll forward, but valid test anyway
-        public void RollForwardToPreReleaseOnMinor_FromRelease(int? rollForwardOnNoCandidateFx, bool? applyPatches, string resolvedVersion)
+                    .WithFramework(MicrosoftNETCoreApp, "5.1.2"))
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForwardOnNoCandidateFx and applyPatches settings correctly roll
+        // from a release version 5.0.0 to the closest minor and latest patch pre-release version (since there's no release available)
+        [Theory] // rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData(null,                       null,         "5.1.4-preview.1")]
+        [InlineData(null,                       false,        "5.1.3-preview.1")]
+        [InlineData(0,                          null,         ResolvedFramework.NotFound)]
+        [InlineData(0,                          false,        ResolvedFramework.NotFound)]
+        [InlineData(1,                          null,         "5.1.4-preview.1")]
+        [InlineData(1,                          false,        "5.1.3-preview.1")]   // Rolls to nearest higher even on patches, but not to latest patch.
+        [InlineData(2,                          null,         "6.1.1")]   // Not really testing the pre-release roll forward, but valid test anyway
+        [InlineData(2,                          false,        "6.1.1")]  // Not really testing the pre-release roll forward, but valid test anyway
+        public void RollForwardToPreReleaseOnMinor_FromRelease(
+            int? rollForwardOnNoCandidateFx,
+            bool? applyPatches,
+            string resolvedFramework)
         {
             RunTestWithManyVersions(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                     .WithApplyPatches(applyPatches)
-                    .WithFramework(MicrosoftNETCoreApp, "5.0.0"),
-                commandResult =>
-                {
-                    if (resolvedVersion != null)
-                    {
-                        commandResult.Should().Pass()
-                            .And.HaveResolvedFramework(MicrosoftNETCoreApp, resolvedVersion);
-                    }
-                    else
-                    {
-                        commandResult.Should().Fail()
-                            .And.DidNotFindCompatibleFrameworkVersion();
-                    }
-                });
-        }
-
-        [Theory]
-        [InlineData(null, null, null)]
-        [InlineData(null, false, null)]
-        [InlineData(0, null, null)]
-        [InlineData(0, false, null)]
-        [InlineData(1, null, null)]
-        [InlineData(1, false, null)]
-        [InlineData(2, null, "7.1.2-preview.1")]
-        [InlineData(2, false, "7.1.1-preview.1")]    // Rolls to nearest higher even on patches, but not to latest patch.
-        public void RollForwardToPreReleaseOnMajor_FromRelease(int? rollForwardOnNoCandidateFx, bool? applyPatches, string resolvedVersion)
+                    .WithFramework(MicrosoftNETCoreApp, "5.0.0"))
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForwardOnNoCandidateFx and applyPatches settings correctly roll
+        // from a release version 6.2.0 to the closest major and latest patch pre-release version (since there's no release available)
+        [Theory] // rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData(null,                       null,         ResolvedFramework.NotFound)]
+        [InlineData(null,                       false,        ResolvedFramework.NotFound)]
+        [InlineData(0,                          null,         ResolvedFramework.NotFound)]
+        [InlineData(0,                          false,        ResolvedFramework.NotFound)]
+        [InlineData(1,                          null,         ResolvedFramework.NotFound)]
+        [InlineData(1,                          false,        ResolvedFramework.NotFound)]
+        [InlineData(2,                          null,         "7.1.2-preview.1")]
+        [InlineData(2,                          false,        "7.1.1-preview.1")]    // Rolls to nearest higher even on patches, but not to latest patch.
+        public void RollForwardToPreReleaseOnMajor_FromRelease(
+            int? rollForwardOnNoCandidateFx,
+            bool? applyPatches,
+            string resolvedFramework)
         {
             RunTestWithManyVersions(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                     .WithApplyPatches(applyPatches)
-                    .WithFramework(MicrosoftNETCoreApp, "6.2.0"),
-                commandResult =>
-                {
-                    if (resolvedVersion != null)
-                    {
-                        commandResult.Should().Pass()
-                            .And.HaveResolvedFramework(MicrosoftNETCoreApp, resolvedVersion);
-                    }
-                    else
-                    {
-                        commandResult.Should().Fail()
-                            .And.DidNotFindCompatibleFrameworkVersion();
-                    }
-                });
-        }
-
-        [Theory] // Both 5.2.3-preview.1 and 5.2.3-preview.2 are available
-        [InlineData(null, null, "5.2.3-preview.2")]     // Rolls to latest patch - including latest pre-release
-        [InlineData(null, false, "5.2.3-preview.1")]
-        [InlineData(0, null, null)]   // This is interesting - we prevent roll forward from release to preview on patch alone
-        [InlineData(0, false, null)]
-        [InlineData(1, null, "5.2.3-preview.2")]   // Rolls to latest patch - including latest pre-release
-        [InlineData(1, false, "5.2.3-preview.1")]  // Rolls to nearest higher even on patches, but not to latest patch.
-        [InlineData(2, null, "6.1.1")]   // Not really testing the pre-release roll forward, but valid test anyway
-        [InlineData(2, false, "6.1.1")]  // Not really testing the pre-release roll forward, but valid test anyway
-        public void RollForwardToPreReleaseToClosestPreRelease_FromRelease(int? rollForwardOnNoCandidateFx, bool? applyPatches, string resolvedVersion)
+                    .WithFramework(MicrosoftNETCoreApp, "6.2.0"))
+               .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForwardOnNoCandidateFx and applyPatches settings correctly roll
+        // from a release version 5.2.2 to the closest patch and latest pre-release version (since there's no release available)
+        // Both 5.2.3-preview.1 and 5.2.3-preview.2 are available
+        [Theory] // rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData(null,                       null,         "5.2.3-preview.2")]     // Rolls to latest patch - including latest pre-release
+        [InlineData(null,                       false,        "5.2.3-preview.1")]
+        // This is a different behavior in 3.0. In 2.* the app would fail in this case as it was explicitly disallowed
+        // to roll forward from release to pre-release when rollForwardOnNoCandidateFx=0 (and only then).
+        [InlineData(0,                          null,         "5.2.3-preview.2")]
+        [InlineData(0,                          false,        ResolvedFramework.NotFound)]
+        [InlineData(1,                          null,         "5.2.3-preview.2")]   // Rolls to latest patch - including latest pre-release
+        [InlineData(1,                          false,        "5.2.3-preview.1")]  // Rolls to nearest higher even on patches, but not to latest patch.
+        [InlineData(2,                          null,         "6.1.1")]   // Not really testing the pre-release roll forward, but valid test anyway
+        [InlineData(2,                          false,        "6.1.1")]  // Not really testing the pre-release roll forward, but valid test anyway
+        public void RollForwardToPreReleaseToClosestPreRelease_FromRelease(
+            int? rollForwardOnNoCandidateFx,
+            bool? applyPatches,
+            string resolvedFramework)
         {
             RunTestWithManyVersions(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                     .WithApplyPatches(applyPatches)
-                    .WithFramework(MicrosoftNETCoreApp, "5.2.2"),
-                commandResult =>
-                {
-                    if (resolvedVersion != null)
-                    {
-                        commandResult.Should().Pass()
-                            .And.HaveResolvedFramework(MicrosoftNETCoreApp, resolvedVersion);
-                    }
-                    else
-                    {
-                        commandResult.Should().Fail()
-                            .And.DidNotFindCompatibleFrameworkVersion();
-                    }
-                });
-        }
-
-        [Theory] // Both 2.3.1-preview.1 and 2.3.2 are available
-        [InlineData(null, null, "2.3.2")]
-        [InlineData(null, false, "2.3.2")]
-        [InlineData(0, null, "2.3.2")]  // Pre-release is ignored, roll forward to latest release patch
-        [InlineData(0, false, null)]    // No exact match available
-        [InlineData(1, null, "2.3.2")]
-        [InlineData(1, false, "2.3.2")] // Pre-release is ignored, roll forward to closest release available
-        [InlineData(2, null, "2.3.2")]
-        [InlineData(2, false, "2.3.2")] // Pre-release is ignored, roll forward to closest release available
-        public void RollForwardToClosestReleaseWithPreReleaseAvailable_FromRelease(int? rollForwardOnNoCandidateFx, bool? applyPatches, string resolvedVersion)
+                    .WithFramework(MicrosoftNETCoreApp, "5.2.2"))
+               .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForwardOnNoCandidateFx and applyPatches settings correctly roll
+        // from a release version 2.3.0 to the closest release (and latest patch) over pre-release versions
+        // Both 2.3.1-preview.1 and 2.3.2 are available
+        [Theory] // rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData(null,                       null,         "2.3.2")]
+        [InlineData(null,                       false,        "2.3.2")]
+        [InlineData(0,                          null,         "2.3.2")]  // Pre-release is ignored, roll forward to latest release patch
+        [InlineData(0,                          false,        ResolvedFramework.NotFound)] // No exact match available
+        [InlineData(1,                          null,         "2.3.2")]
+        [InlineData(1,                          false,        "2.3.2")] // Pre-release is ignored, roll forward to closest release available
+        [InlineData(2,                          null,         "2.3.2")]
+        [InlineData(2,                          false,        "2.3.2")] // Pre-release is ignored, roll forward to closest release available
+        public void RollForwardToClosestReleaseWithPreReleaseAvailable_FromRelease(
+            int? rollForwardOnNoCandidateFx,
+            bool? applyPatches, 
+            string resolvedFramework)
         {
             RunTestWithManyVersions(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                     .WithApplyPatches(applyPatches)
-                    .WithFramework(MicrosoftNETCoreApp, "2.3.0"),
-                commandResult =>
-                {
-                    if (resolvedVersion != null)
-                    {
-                        commandResult.Should().Pass()
-                            .And.HaveResolvedFramework(MicrosoftNETCoreApp, resolvedVersion);
-                    }
-                    else
-                    {
-                        commandResult.Should().Fail()
-                            .And.DidNotFindCompatibleFrameworkVersion();
-                    }
-                });
-        }
-
-        [Theory]
-        [InlineData(null, null, null)]   // Pre-release will only match the extact x.y.z version, regardless of settings
-        [InlineData(0, false, null)]
-        [InlineData(1, null, null)]
-        [InlineData(2, null, null)]
-        public void RollForwardToPreRelease_FromDifferentPreRelease(int? rollForwardOnNoCandidateFx, bool? applyPatches, string resolvedVersion)
+                    .WithFramework(MicrosoftNETCoreApp, "2.3.0"))
+               .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForwardOnNoCandidateFx and applyPatches settings correctly roll
+        // from a pre-release version 5.1.1-preview.1 to another pre-release - latest patch.
+        // 3.0 change:
+        // 2.* - Pre-release will only match the extact x.y.z version, regardless of settings
+        // 3.* - Pre-release uses normal roll forward rules, including rolling over minor/patches and obeying settings.
+        [Theory] // rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData(null,                       null,         "5.1.4-preview.1")]
+        [InlineData(0,                          false,        ResolvedFramework.NotFound)]  // Roll-forward fully disabled
+        [InlineData(1,                          null,         "5.1.4-preview.1")]
+        [InlineData(2,                          null,         "5.1.4-preview.1")]
+        public void RollForwardToPreRelease_FromDifferentPreRelease(
+            int? rollForwardOnNoCandidateFx,
+            bool? applyPatches,
+            string resolvedFramework)
         {
             RunTestWithManyVersions(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                     .WithApplyPatches(applyPatches)
-                    .WithFramework(MicrosoftNETCoreApp, "5.1.1-preview.1"),
-                commandResult =>
-                {
-                    if (resolvedVersion != null)
-                    {
-                        commandResult.Should().Pass()
-                            .And.HaveResolvedFramework(MicrosoftNETCoreApp, resolvedVersion);
-                    }
-                    else
-                    {
-                        commandResult.Should().Fail()
-                            .And.DidNotFindCompatibleFrameworkVersion();
-                    }
-                });
-        }
-
-        [Theory]
-        // Pre-release with exact match will not try to roll forward at all
-        [InlineData(null, null)]
-        [InlineData(null, false)]
-        [InlineData(0, false)]
-        [InlineData(1, null)]
-        [InlineData(2, null)]
-        public void RollForwardToPreRelease_ExactPreReleaseMatch(int? rollForwardOnNoCandidateFx, bool? applyPatches)
+                    .WithFramework(MicrosoftNETCoreApp, "5.1.1-preview.1"))
+               .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForwardOnNoCandidateFx and applyPatches settings correctly roll
+        // from a pre-release version 5.1.3-preview.1 which exists to another pre-release - latest patch.
+        // 3.0 change:
+        // 2.* - Pre-release with exact match will not try to roll forward at all
+        // 3.* - Pre-release uses normal roll forward rules, it will roll forward on patches even on exact match.
+        [Theory] // rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData(null,                       null,         "5.1.4-preview.1")]
+        [InlineData(null,                       false,        "5.1.3-preview.2")]
+        [InlineData(0,                          false,        "5.1.3-preview.2")]
+        [InlineData(1,                          null,         "5.1.4-preview.1")]
+        [InlineData(2,                          null,         "5.1.4-preview.1")]
+        public void RollForwardToPreRelease_ExactPreReleaseMatch(
+            int? rollForwardOnNoCandidateFx,
+            bool? applyPatches,
+            string resolvedFramework)
         {
             RunTestWithManyVersions(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                     .WithApplyPatches(applyPatches)
-                    .WithFramework(MicrosoftNETCoreApp, "5.1.3-preview.1"),
-                commandResult =>
-                    commandResult.Should().Pass()
-                        .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3-preview.1")
-                        .And.HaveStdErrContaining("Did not roll forward"));
-        }
-
-        [Theory]
-        [InlineData(null, null, "5.1.3-preview.1")]   // Pre-release will select the closest higher version (5.1.3-preview.2 is available)
-        [InlineData(null, false, "5.1.3-preview.1")]
-        [InlineData(0, false, "5.1.3-preview.1")]
-        [InlineData(1, null, "5.1.3-preview.1")]
-        [InlineData(2, null, "5.1.3-preview.1")]
-        public void RollForwardToPreRelease_FromSamePreRelease(int? rollForwardOnNoCandidateFx, bool? applyPatches, string resolvedVersion)
+                    .WithFramework(MicrosoftNETCoreApp, "5.1.3-preview.1"))
+               .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForwardOnNoCandidateFx and applyPatches settings correctly roll
+        // from a pre-release version 5.1.3-preview.0 which doesn't exists to another pre-release - latest patch.
+        // 3.0 change:
+        // 2.* - Pre-release will select the closest higher version (5.1.3-preview.2 is available in this test, but 5.1.3-preview.1 will be selected over it)
+        // 3.* - Pre-release applies roll forward on patches if enabled, always selecting the latest patch version.
+        [Theory] // rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData(null,                       null,         "5.1.4-preview.1")]
+        [InlineData(null,                       false,        "5.1.3-preview.2")]
+        [InlineData(0,                          false,        "5.1.3-preview.2")]
+        [InlineData(1,                          null,         "5.1.4-preview.1")]
+        [InlineData(2,                          null,         "5.1.4-preview.1")]
+        public void RollForwardToPreRelease_FromSamePreRelease(
+            int? rollForwardOnNoCandidateFx,
+            bool? applyPatches,
+            string resolvedFramework)
         {
             RunTestWithManyVersions(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                     .WithApplyPatches(applyPatches)
-                    .WithFramework(MicrosoftNETCoreApp, "5.1.3-preview.0"),
-                commandResult =>
-                {
-                    if (resolvedVersion != null)
-                    {
-                        commandResult.Should().Pass()
-                            .And.HaveResolvedFramework(MicrosoftNETCoreApp, resolvedVersion);
-                    }
-                    else
-                    {
-                        commandResult.Should().Fail()
-                            .And.DidNotFindCompatibleFrameworkVersion();
-                    }
-                });
-        }
-
-        [Theory]  // When rolling from release, pre-release is ignored if any release which matches can be found
+                    .WithFramework(MicrosoftNETCoreApp, "5.1.3-preview.0"))
+               .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForwardOnNoCandidateFx and applyPatches settings correctly roll
+        // from a release version 6.1.0 to another release version.
+        // When rolling from release, pre-release is ignored if any release which matches can be found
+        // 6.1.1 and 6.1.2-preview.1 is available so pure latest patch should pick the 6.1.2-preview.1
+        // but release is prefered if available.
+        [Theory] // rollForwardOnNoCandidateFx
         [InlineData(null)]
         [InlineData(1)]
         [InlineData(2)]
@@ -711,75 +717,18 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
             RunTestWithManyVersions(
                 runtimeConfig => runtimeConfig
                     .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
-                    .WithFramework(MicrosoftNETCoreApp, "6.1.0"),
-                commandResult => commandResult.Should().Pass()
-                    .And.HaveResolvedFramework(MicrosoftNETCoreApp, "6.1.1"));
-        }
-
-
-
-        private void RunTestWithManyVersions(
-            Func<RuntimeConfig, RuntimeConfig> runtimeConfig,
-            Action<CommandResult> resultAction)
-        {
-            RunTest(SharedState.DotNetWithManyVersions, runtimeConfig, resultAction);
+                    .WithFramework(MicrosoftNETCoreApp, "6.1.0"))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "6.1.1");
         }
-        #endregion
 
-        private void RunTest(
-            DotNetCli dotNet,
-            Func<RuntimeConfig, RuntimeConfig> runtimeConfig,
-            Action<CommandResult> resultAction)
+        private CommandResult RunTestWithManyVersions(Func<RuntimeConfig, RuntimeConfig> runtimeConfig)
         {
-            RunTest(
-                dotNet,
+            return RunTest(
+                SharedState.DotNetWithManyVersions,
                 SharedState.FrameworkReferenceApp,
-                runtimeConfig,
-                resultAction);
-        }
-
-        public class SharedTestState : SharedTestStateBase
-        {
-            public TestApp FrameworkReferenceApp { get; }
-
-            public DotNetCli DotNetWithOneFramework { get; }
-
-            public DotNetCli DotNetWithPreReleaseFramework { get; }
-
-            public DotNetCli DotNetWithManyVersions { get; }
-
-            public SharedTestState()
-            {
-                DotNetWithOneFramework = DotNet("WithOneFramework")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.3")
-                    .Build();
-
-                DotNetWithPreReleaseFramework = DotNet("WithPreReleaseFramework")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.3-preview.2")
-                    .Build();
-
-                DotNetWithManyVersions = DotNet("WithManyVersions")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("2.3.1-preview.1")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("2.3.2")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("4.1.1")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("4.1.2")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("4.1.3-preview.1")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("4.2.1")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("4.5.1-preview.1")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("4.5.2")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.3-preview.1")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.3-preview.2")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.4-preview.1")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.2.3-preview.1")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.2.3-preview.2")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("6.1.1")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("6.1.2-preview.1")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("7.1.1-preview.1")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("7.1.2-preview.1")
-                    .Build();
-
-                FrameworkReferenceApp = CreateFrameworkReferenceApp();
-            }
+                new TestSettings()
+                    .WithRuntimeConfigCustomizer(runtimeConfig));
         }
+        #endregion
     }
 }
index ba5ed7c..a99d844 100644 (file)
@@ -24,18 +24,49 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
             SharedState = sharedState;
         }
 
-        // Soft roll forward from the inner framework reference [specified] to app's 5.1.1 (defaults)
-        [Theory]
-        [InlineData("5.0.0", 0,    null,  null)]
-        [InlineData("5.1.0", 0,    null,  "5.1.3")]
-        [InlineData("5.1.0", 0,    false, null)]
-        [InlineData("5.1.1", 0,    false, "5.1.1")]
-        [InlineData("5.0.0", null, null,  "5.1.3")]
-        [InlineData("5.0.0", 1,    null,  "5.1.3")]
-        [InlineData("5.1.0", 1,    false, "5.1.1")]
-        [InlineData("1.0.0", 1,    null,  null)]
-        [InlineData("1.0.0", 2,    null,  "5.1.3")]
-        public void SoftRollForward_InnerFrameworkReference_ToHigher(
+        public class SharedTestState : SharedTestStateBase
+        {
+            public TestApp FrameworkReferenceApp { get; }
+
+            public DotNetCli DotNetWithMultipleFrameworks { get; }
+
+            public SharedTestState()
+            {
+                DotNetWithMultipleFrameworks = DotNet("WithOneFramework")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.3")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.4.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.6.0")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("6.0.0")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("6.1.1-preview.2")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("6.2.1")
+                    .AddFramework(MiddleWare, "2.1.2", runtimeConfig =>
+                        runtimeConfig.WithFramework(MicrosoftNETCoreApp, "5.1.3"))
+                    .AddFramework(AnotherMiddleWare, "3.0.0", runtimeConfig =>
+                        runtimeConfig.WithFramework(MicrosoftNETCoreApp, "5.1.3"))
+                    .AddFramework(HighWare, "7.3.1", runtimeConfig =>
+                        runtimeConfig
+                            .WithFramework(MicrosoftNETCoreApp, "5.1.3")
+                            .WithFramework(MiddleWare, "2.1.2"))
+                    .Build();
+
+                FrameworkReferenceApp = CreateFrameworkReferenceApp();
+            }
+        }
+
+        // Verify that inner framework reference (<fxRefVersion>, <rollForwardOnNoCandidateFx>, <applyPatches>)
+        // is correctly reconciled with app's framework reference 5.1.1 (defaults = RollForward:Minor). App fx reference is higher.
+        [Theory] // fxRefVersion  rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData("5.0.0",      0,                          null,         ResolvedFramework.FailedToReconcile)]
+        [InlineData("5.1.0",      0,                          null,         "5.1.3")]
+        [InlineData("5.1.0",      0,                          false,        ResolvedFramework.FailedToReconcile)]
+        [InlineData("5.1.1",      0,                          false,        "5.1.1")]
+        [InlineData("5.0.0",      null,                       null,         "5.1.3")]
+        [InlineData("5.0.0",      1,                          null,         "5.1.3")]
+        [InlineData("5.1.0",      1,                          false,        "5.1.1")]
+        [InlineData("1.0.0",      1,                          null,         ResolvedFramework.FailedToReconcile)]
+        [InlineData("1.0.0",      2,                          null,         "5.1.3")]
+        public void ReconcileFxReferences_InnerFrameworkReference_ToHigher(
             string versionReference,
             int? rollForwardOnNoCandidateFx,
             bool? applyPatches,
@@ -49,27 +80,28 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
                     runtimeConfig.GetFramework(MicrosoftNETCoreApp)
                         .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                         .WithApplyPatches(applyPatches)
-                        .Version = versionReference),
-                resolvedFramework,
-                commandResult => commandResult.Should().Fail().And.FailedToSoftRollForward(MicrosoftNETCoreApp, versionReference, "5.1.1"));
+                        .Version = versionReference))
+                .ShouldHaveResolvedFrameworkOrFailedToReconcileFrameworkReference(
+                    MicrosoftNETCoreApp, resolvedFramework, versionReference, "5.1.1");
         }
 
-        // Soft roll forward from the inner framework reference [specified] to app's 5.1.1 (defaults)
+        // Verify that inner framework reference (<fxRefVersion>, <rollForwardOnNoCandidateFx>, <applyPatches>)
+        // is correctly reconciled with app's framework reference 5.1.1 (defaults = RollForward:Minor). App fx reference is higher.
         // In this case the direct reference from app is first, so the framework reference from app
         // is actually resolved against the disk - and the resolved framework is than compared to
-        // the inner framework reference .
-        [Theory]
-        [InlineData("5.0.0", 0, null, null)]
-        [InlineData("5.1.0", 0, null, "5.1.3")]
-        [InlineData("5.1.0", 0, false, null)]
-        [InlineData("5.1.3", 0, false, "5.1.3")]
-        [InlineData("5.0.0", null, null, "5.1.3")]
-        [InlineData("5.0.0", 1, null, "5.1.3")]
-        // Ordering issue - if the order of FX references in app is swapped, the output would be 5.1.1
-        [InlineData("5.1.0", 1, false, "5.1.3")]
-        [InlineData("1.0.0", 1, null, null)]
-        [InlineData("1.0.0", 2, null, "5.1.3")]
-        public void SoftRollForward_InnerFrameworkReference_ToHigher_HardResolve(
+        // the inner framework reference (potentially causing re-resolution).
+        [Theory] // fxRefVersion  rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData("5.0.0",      0,                          null,         ResolvedFramework.FailedToReconcile)]
+        [InlineData("5.1.0",      0,                          null,         "5.1.3")]
+        [InlineData("5.1.0",      0,                          false,        ResolvedFramework.FailedToReconcile)]
+        [InlineData("5.1.3",      0,                          false,        "5.1.3")]
+        [InlineData("5.0.0",      null,                       null,         "5.1.3")]
+        [InlineData("5.0.0",      1,                          null,         "5.1.3")]
+        [InlineData("5.0.0",      1,                          false,        "5.1.1")]
+        [InlineData("5.1.0",      1,                          false,        "5.1.1")]
+        [InlineData("1.0.0",      1,                          null,         ResolvedFramework.FailedToReconcile)]
+        [InlineData("1.0.0",      2,                          null,         "5.1.3")]
+        public void ReconcileFrameworkReferences_InnerFrameworkReference_ToHigher_HardResolve(
             string versionReference,
             int? rollForwardOnNoCandidateFx,
             bool? applyPatches,
@@ -83,16 +115,17 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
                     runtimeConfig.GetFramework(MicrosoftNETCoreApp)
                         .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                         .WithApplyPatches(applyPatches)
-                        .Version = versionReference),
-                resolvedFramework,
-                commandResult => commandResult.Should().Fail().And.FailedToSoftRollForward(MicrosoftNETCoreApp, versionReference, "5.1.3"));
+                        .Version = versionReference))
+                .ShouldHaveResolvedFrameworkOrFailedToReconcileFrameworkReference(
+                    MicrosoftNETCoreApp, resolvedFramework, versionReference, "5.1.1");
         }
 
-        // Soft roll forward from inner framework reference [specified] to  app's 5.1.1 (defaults)
-        [Theory]
-        [InlineData("5.4.0", null, null, "5.4.1")]
-        [InlineData("6.0.0", null, null, null)]
-        public void SoftRollForward_InnerFrameworkReference_ToLower(
+        // Verify that inner framework reference (<fxRefVersion>, <rollForwardOnNoCandidateFx>, <applyPatches>)
+        // is correctly reconciled with app's framework reference 5.1.1 (defaults = RollForward:Minor). App fx reference is lower.
+        [Theory] // fxRefVersion  rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData("5.4.0",      null,                       null,         "5.4.1")]
+        [InlineData("6.0.0",      null,                       null,         ResolvedFramework.FailedToReconcile)]
+        public void ReconcileFrameworkReferences_InnerFrameworkReference_ToLower(
             string versionReference,
             int? rollForwardOnNoCandidateFx,
             bool? applyPatches,
@@ -106,19 +139,20 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
                     runtimeConfig.GetFramework(MicrosoftNETCoreApp)
                         .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                         .WithApplyPatches(applyPatches)
-                        .Version = versionReference),
-                resolvedFramework,
-                commandResult => commandResult.Should().Fail().And.FailedToSoftRollForward(MicrosoftNETCoreApp, "5.1.1", versionReference));
+                        .Version = versionReference))
+                .ShouldHaveResolvedFrameworkOrFailedToReconcileFrameworkReference(
+                    MicrosoftNETCoreApp, resolvedFramework, "5.1.1", versionReference);
         }
 
-        // Soft roll forward from inner framework reference [specified] to  app's 5.1.1 (defaults)
-        // In this case the app reference to core framework comes first, which means it's going to be hard resolved
-        // and only then the soft roll forward to the inner reference is performed. So the hard resolved version
-        // is use in the soft roll forward.
-        [Theory]
-        [InlineData("5.4.0", null, null, "5.4.1")]
-        [InlineData("6.0.0", null, null, null)]
-        public void SoftRollForward_InnerFrameworkReference_ToLower_HardResolve(
+        // Verify that inner framework reference (<fxRefVersion>, <rollForwardOnNoCandidateFx>, <applyPatches>)
+        // is correctly reconciled with app's framework reference 5.1.1 (defaults = RollForward:Minor). App fx reference is lower.
+        // In this case the direct reference from app is first, so the framework reference from app
+        // is actually resolved against the disk - and the resolved framework is than compared to
+        // the inner framework reference (potentially causing re-resolution).
+        [Theory] // fxRefVersion  rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData("5.4.0",      null,                       null,         "5.4.1")]
+        [InlineData("6.0.0",      null,                       null,         ResolvedFramework.FailedToReconcile)]
+        public void ReconcileFrameworkReferences_InnerFrameworkReference_ToLower_HardResolve(
             string versionReference,
             int? rollForwardOnNoCandidateFx,
             bool? applyPatches,
@@ -132,21 +166,28 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
                     runtimeConfig.GetFramework(MicrosoftNETCoreApp)
                         .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                         .WithApplyPatches(applyPatches)
-                        .Version = versionReference),
-                resolvedFramework,
-                commandResult => commandResult.Should().Fail().And.FailedToSoftRollForward(MicrosoftNETCoreApp, "5.1.3", versionReference));
+                        .Version = versionReference))
+                .ShouldHaveResolvedFrameworkOrFailedToReconcileFrameworkReference(
+                    MicrosoftNETCoreApp, resolvedFramework, "5.1.1", versionReference);
         }
 
-        // Soft roll forward from inner framework reference [specified] to app's 6.1.1-preview.0 (defaults)
-        [Theory]
-        [InlineData("6.0.0", null, null, null)]    // Can't roll forward from release to pre-release
-        [InlineData("6.0.1-preview.0", null, null, "6.1.1-preview.1")]
-        [InlineData("6.1.1-preview.0", null, null, "6.1.1-preview.1")]
-        [InlineData("6.1.0-preview.0", 0, null, "6.1.1-preview.1")] // This is effectively a bug, the design was that pre-release should never roll on patches
-        [InlineData("6.1.1-preview.0", 0, null, "6.1.1-preview.1")]
-        [InlineData("6.1.1-preview.0", 0, false, "6.1.1-preview.1")]
-        [InlineData("6.1.1-preview.1", 0, null, "6.1.1-preview.1")]
-        public void SoftRollForward_InnerFrameworkReference_PreRelease(
+        // Verify that inner framework reference (<fxRefVersion>, <rollForwardOnNoCandidateFx>, <applyPatches>)
+        // is correctly reconciled with app's framework reference 5.1.1 (defaults = RollForward:Minor). App fx reference is higher.
+        // 3.0 change:
+        // 2.* - release would never roll forward to pre-release
+        // 3.* - release rolls forward to pre-release if there is no available release match
+        [Theory] // fxRefVersion       rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData("6.0.0",           null,                       null,         "6.2.1")]   // Starting from release version should prefer release version
+        [InlineData("6.0.1-preview.0", null,                       null,         "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.1", null,                       null,         "6.1.1-preview.2")]
+        [InlineData("6.0.1-preview.0", 0,                          null,         ResolvedFramework.FailedToReconcile)]
+        [InlineData("6.1.0-preview.0", 0,                          false,        ResolvedFramework.FailedToReconcile)]
+        [InlineData("6.1.0-preview.0", 0,                          null,         "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.0", 0,                          false,        "6.1.1-preview.2")] // applyPatches=false is ignored for pre-release roll
+        [InlineData("6.1.1-preview.1", 0,                          null,         "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.1", 0,                          false,        "6.1.1-preview.2")]
+        [InlineData("6.1.1-preview.2", 0,                          null,         "6.1.1-preview.2")]
+        public void ReconcileFrameworkReferences_InnerFrameworkReference_PreRelease(
             string versionReference,
             int? rollForwardOnNoCandidateFx,
             bool? applyPatches,
@@ -154,30 +195,32 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
         {
             RunTest(
                 runtimeConfig => runtimeConfig
-                    .WithFramework(MicrosoftNETCoreApp, "6.1.1-preview.0")
+                    .WithFramework(MicrosoftNETCoreApp, "6.1.1-preview.1")
                     .WithFramework(MiddleWare, "2.1.0"),
                 dotnetCustomizer => dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
                     runtimeConfig.GetFramework(MicrosoftNETCoreApp)
                         .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                         .WithApplyPatches(applyPatches)
-                        .Version = versionReference),
-                resolvedFramework,
-                commandResult => commandResult.Should().Fail().And.FailedToSoftRollForward(MicrosoftNETCoreApp, versionReference, "6.1.1-preview.1"));
+                        .Version = versionReference))
+                .ShouldHaveResolvedFrameworkOrFailedToReconcileFrameworkReference(
+                    MicrosoftNETCoreApp, resolvedFramework, versionReference, "6.1.1-preview.1");
         }
 
-        // Soft roll forward from inner framework reference 5.1.1 to app [specified version]
-        [Theory]
-        [InlineData("5.0.0", 0, null, null)]
-        [InlineData("5.1.0", 0, null, "5.1.3")]
-        [InlineData("5.1.0", 0, false, null)]
-        [InlineData("5.1.1", 0, false, "5.1.1")]
-        [InlineData("5.0.0", null, null, "5.1.3")]
-        [InlineData("5.1.0", 1, null, "5.1.3")]
-        [InlineData("5.1.0", 1, false, "5.1.1")]
-        [InlineData("5.0.0", 1, null, "5.1.3")]
-        [InlineData("1.0.0", 1, null, null)]
-        [InlineData("1.0.0", 2, null, "5.1.3")]
-        public void SoftRollForward_AppFrameworkReference_ToLower(
+        // Verify that inner framework reference 5.1.1 (defaults = RollForward:Minor)
+        // is correctly reconciled with app's framework reference (<fxRefVersion>, <rollForwardOnNoCandidateFx>, <applyPatches>).
+        // App fx reference is lower.
+        [Theory] // fxRefVersion  rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData("5.0.0",      0,                          null,         ResolvedFramework.FailedToReconcile)]
+        [InlineData("5.1.0",      0,                          null,         "5.1.3")]
+        [InlineData("5.1.0",      0,                          false,        ResolvedFramework.FailedToReconcile)]
+        [InlineData("5.1.1",      0,                          false,        "5.1.1")]
+        [InlineData("5.0.0",      null,                       null,         "5.1.3")]
+        [InlineData("5.1.0",      1,                          null,         "5.1.3")]
+        [InlineData("5.1.0",      1,                          false,        "5.1.1")]
+        [InlineData("5.0.0",      1,                          null,         "5.1.3")]
+        [InlineData("1.0.0",      1,                          null,         ResolvedFramework.FailedToReconcile)]
+        [InlineData("1.0.0",      2,                          null,         "5.1.3")]
+        public void ReconcileFrameworkReferences_AppFrameworkReference_ToLower(
             string versionReference,
             int? rollForwardOnNoCandidateFx,
             bool? applyPatches,
@@ -191,24 +234,29 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
                         .WithApplyPatches(applyPatches)),
                 dotnetCustomizer => dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
                     runtimeConfig.GetFramework(MicrosoftNETCoreApp)
-                        .Version = "5.1.1"),
-                resolvedFramework,
-                commandResult => commandResult.Should().Fail().And.FailedToSoftRollForward(MicrosoftNETCoreApp, versionReference, "5.1.1"));
+                        .Version = "5.1.1"))
+                .ShouldHaveResolvedFrameworkOrFailedToReconcileFrameworkReference(
+                    MicrosoftNETCoreApp, resolvedFramework, versionReference, "5.1.1");
         }
 
-        // Soft roll forward from app [specified version] to inner framework reference 5.1.1
-        [Theory]
-        [InlineData("5.0.0", 0, null, null)]
-        [InlineData("5.1.0", 0, null, "5.1.3")]
-        [InlineData("5.1.0", 0, false, null)]
-        [InlineData("5.1.1", 0, false, "5.1.1")]
-        [InlineData("5.0.0", null, null, "5.1.3")]
-        [InlineData("5.1.0", 1, null, "5.1.3")]
-        [InlineData("5.1.0", 1, false, "5.1.1")]
-        [InlineData("5.0.0", 1, null, "5.1.3")]
-        [InlineData("1.0.0", 1, null, null)]
-        [InlineData("1.0.0", 2, null, "5.1.3")]
-        public void SoftRollForward_AppFrameworkReference_ToLower_HardResolve(
+        // Verify that inner framework reference 5.1.1 (defaults = RollForward:Minor)
+        // is correctly reconciled with app's framework reference (<fxRefVersion>, <rollForwardOnNoCandidateFx>, <applyPatches>).
+        // App fx reference is lower.
+        // In this case the direct reference from app is first, so the framework reference from app
+        // is actually resolved against the disk - and the resolved framework is than compared to
+        // the inner framework reference (potentially causing re-resolution).
+        [Theory] // fxRefVersion  rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData("5.0.0",      0,                          null,         ResolvedFramework.NotFound)]
+        [InlineData("5.1.0",      0,                          null,         "5.1.3")]
+        [InlineData("5.1.0",      0,                          false,        ResolvedFramework.NotFound)]
+        [InlineData("5.1.1",      0,                          false,        "5.1.1")]
+        [InlineData("5.0.0",      null,                       null,         "5.1.3")]
+        [InlineData("5.1.0",      1,                          null,         "5.1.3")]
+        [InlineData("5.1.0",      1,                          false,        "5.1.1")]
+        [InlineData("5.0.0",      1,                          null,         "5.1.3")]
+        [InlineData("1.0.0",      1,                          null,         ResolvedFramework.NotFound)]
+        [InlineData("1.0.0",      2,                          null,         "5.1.3")]
+        public void ReconcileFrameworkReferences_AppFrameworkReference_ToLower_HardResolve(
             string versionReference,
             int? rollForwardOnNoCandidateFx,
             bool? applyPatches,
@@ -222,16 +270,20 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
                     .WithFramework(MiddleWare, "2.1.0"),
                 dotnetCustomizer => dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
                     runtimeConfig.GetFramework(MicrosoftNETCoreApp)
-                        .Version = "5.1.1"),
-                resolvedFramework,
-                commandResult => commandResult.Should().Fail().And.DidNotFindCompatibleFrameworkVersion());
+                        .Version = "5.1.1"))
+                // Note that in this case (since the app reference is first) if the app's framework reference
+                // can't be resolved against the available frameworks, the error is actually a regular
+                // "can't find framework" and not a framework reconcile event.
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
         }
 
-        // Soft roll forward from inner framework reference 5.1.1 to app [specified version]
-        [Theory]
-        [InlineData("5.4.0", null, null, "5.4.1")]
-        [InlineData("6.0.0", null, null, null)]
-        public void SoftRollForward_AppFrameworkReference_ToHigher(
+        // Verify that inner framework reference 5.1.1 (defaults = RollForward:Minor)
+        // is correctly reconciled with app's framework reference (<fxRefVersion>, <rollForwardOnNoCandidateFx>, <applyPatches>).
+        // App fx reference is higher.
+        [Theory] // fxRefVersion  rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData("5.4.0",      null,                       null,         "5.4.1")]
+        [InlineData("6.0.0",      null,                       null,         ResolvedFramework.FailedToReconcile)]
+        public void ReconcileFrameworkReferences_AppFrameworkReference_ToHigher(
             string versionReference,
             int? rollForwardOnNoCandidateFx,
             bool? applyPatches,
@@ -245,16 +297,21 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
                         .WithApplyPatches(applyPatches)),
                 dotnetCustomizer => dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
                     runtimeConfig.GetFramework(MicrosoftNETCoreApp)
-                        .Version = "5.1.1"),
-                resolvedFramework,
-                commandResult => commandResult.Should().Fail().And.FailedToSoftRollForward(MicrosoftNETCoreApp, "5.1.1", versionReference));
+                        .Version = "5.1.1"))
+                .ShouldHaveResolvedFrameworkOrFailedToReconcileFrameworkReference(
+                    MicrosoftNETCoreApp, resolvedFramework, "5.1.1", versionReference);
         }
 
-        // Soft roll forward from inner framework reference 5.1.1 to app [specified version]
-        [Theory]
-        [InlineData("5.4.0", null, null, "5.4.1")]
-        [InlineData("6.0.0", null, null, null)]
-        public void SoftRollForward_AppFrameworkReference_ToHigher_HardResolve(
+        // Verify that inner framework reference 5.1.1 (defaults = RollForward:Minor)
+        // is correctly reconciled with app's framework reference (<fxRefVersion>, <rollForwardOnNoCandidateFx>, <applyPatches>).
+        // App fx reference is higher.
+        // In this case the direct reference from app is first, so the framework reference from app
+        // is actually resolved against the disk - and the resolved framework is than compared to
+        // the inner framework reference (potentially causing re-resolution).
+        [Theory] // fxRefVersion  rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData("5.4.0",      null,                       null,         "5.4.1")]
+        [InlineData("6.0.0",      null,                       null,         null)]
+        public void ReconcileFrameworkReferences_AppFrameworkReference_ToHigher_HardResolve(
             string versionReference,
             int? rollForwardOnNoCandidateFx,
             bool? applyPatches,
@@ -268,21 +325,23 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
                     .WithFramework(MiddleWare, "2.1.0"),
                 dotnetCustomizer => dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
                     runtimeConfig.GetFramework(MicrosoftNETCoreApp)
-                        .Version = "5.1.1"),
-                resolvedFramework,
-                commandResult => commandResult.Should().Fail().And.FailedToSoftRollForward(MicrosoftNETCoreApp, "5.1.1", versionReference));
+                        .Version = "5.1.1"))
+                .ShouldHaveResolvedFrameworkOrFailedToReconcileFrameworkReference(
+                    MicrosoftNETCoreApp, resolvedFramework, "5.1.1", versionReference);
         }
 
-        // Soft roll forward inner framework reference (defaults) to inner framework reference with [specified version]
-        [Theory]
-        [InlineData("5.0.0", 0,    null,  null)]
-        [InlineData("5.1.0", 0,    null,  "5.1.3")]
-        [InlineData("5.1.0", 0,    false, null)]
-        [InlineData("5.0.0", null, null,  "5.1.3")]
-        [InlineData("5.0.0", 1,    null,  "5.1.3")]
-        [InlineData("1.0.0", 1,    null,  null)]
-        [InlineData("1.0.0", 2,    null,  "5.1.3")]
-        public void SoftRollForward_InnerToInnerFrameworkReference_ToLower(
+        // Verify that inner framework reference 5.1.1 (defaults = RollForward:Minor)
+        // is correctly reconciled with another's framework reference (<fxRefVersion>, <rollForwardOnNoCandidateFx>, <applyPatches>).
+        // The higher framework has fx reference with higher version.
+        [Theory] // fxRefVersion  rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData("5.0.0",      0,                          null,         ResolvedFramework.FailedToReconcile)]
+        [InlineData("5.1.0",      0,                          null,         "5.1.3")]
+        [InlineData("5.1.0",      0,                          false,        ResolvedFramework.FailedToReconcile)]
+        [InlineData("5.0.0",      null,                       null,         "5.1.3")]
+        [InlineData("5.0.0",      1,                          null,         "5.1.3")]
+        [InlineData("1.0.0",      1,                          null,         ResolvedFramework.FailedToReconcile)]
+        [InlineData("1.0.0",      2,                          null,         "5.1.3")]
+        public void ReconcileFrameworkReferences_InnerToInnerFrameworkReference_ToLower(
             string versionReference,
             int? rollForwardOnNoCandidateFx,
             bool? applyPatches,
@@ -301,16 +360,18 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
                             .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                             .WithApplyPatches(applyPatches)
                             .Version = versionReference);
-                },
-                resolvedFramework,
-                commandResult => commandResult.Should().Fail().And.FailedToSoftRollForward(MicrosoftNETCoreApp, versionReference, "5.1.3"));
+                })
+                .ShouldHaveResolvedFrameworkOrFailedToReconcileFrameworkReference(
+                    MicrosoftNETCoreApp, resolvedFramework, versionReference, "5.1.1");
         }
 
-        // Soft roll forward inner framework reference (defaults) to inner framework reference with [specified version]
-        [Theory]
-        [InlineData("5.4.0", null, null, "5.4.1")]
-        [InlineData("6.0.0", null, null, null)]
-        public void SoftRollForward_InnerToInnerFrameworkReference_ToHigher(
+        // Verify that inner framework reference 5.1.1 (defaults = RollForward:Minor)
+        // is correctly reconciled with another's framework reference (<fxRefVersion>, <rollForwardOnNoCandidateFx>, <applyPatches>).
+        // The higher framework has fx reference with lower version.
+        [Theory] // fxRefVersion  rollForwardOnNoCandidateFx  applyPatches  resolvedFramework
+        [InlineData("5.4.0",      null,                       null,         "5.4.1")]
+        [InlineData("6.0.0",      null,                       null,         ResolvedFramework.FailedToReconcile)]
+        public void ReconcileFrameworkReferences_InnerToInnerFrameworkReference_ToHigher(
             string versionReference,
             int? rollForwardOnNoCandidateFx,
             bool? applyPatches,
@@ -329,12 +390,12 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
                             .WithRollForwardOnNoCandidateFx(rollForwardOnNoCandidateFx)
                             .WithApplyPatches(applyPatches)
                             .Version = versionReference);
-                },
-                resolvedFramework,
-                commandResult => commandResult.Should().Fail().And.FailedToSoftRollForward(MicrosoftNETCoreApp, "5.1.3", versionReference));
+                })
+                .ShouldHaveResolvedFrameworkOrFailedToReconcileFrameworkReference(
+                    MicrosoftNETCoreApp, resolvedFramework, "5.1.1", versionReference);
         }
 
-        // This test does:
+        // This test:
         //  - Forces hard resolve of 5.1.1 -> 5.1.3 (direct reference from app)
         //  - Loads HighWare which has 5.4.1 
         //    - This forces a retry since 5.1.3 was hard resolved, so we have reload with 5.4.1 instead
@@ -356,15 +417,14 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
                     dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
                         runtimeConfig.GetFramework(MicrosoftNETCoreApp)
                             .Version = "5.6.0");
-                },
-                resultValidator: commandResult =>
-                    commandResult.Should().Pass()
-                        .And.RestartedFrameworkResolution("5.1.3", "5.4.1")
-                        .And.RestartedFrameworkResolution("5.4.1", "5.6.0")
-                        .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.6.0"));
+                })
+                .Should().Pass()
+                .And.RestartedFrameworkResolution("5.1.1", "5.4.1")
+                .And.RestartedFrameworkResolution("5.4.1", "5.6.0")
+                .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.6.0");
         }
 
-        // This test does:
+        // This test:
         //  - Forces hard resolve of 5.1.1 -> 5.1.3 (direct reference from app)
         //  - Loads MiddleWare which has 5.4.1 
         //    - This forces a retry since 5.1.3 was hard resolved, so we have reload with 5.4.1 instead
@@ -387,14 +447,14 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
                     dotnetCustomizer.Framework(AnotherMiddleWare).RuntimeConfig(runtimeConfig =>
                         runtimeConfig.GetFramework(MicrosoftNETCoreApp)
                             .Version = "5.6.0");
-                },
-                resultValidator: commandResult =>
-                    commandResult.Should().Pass()
-                        .And.RestartedFrameworkResolution("5.1.3", "5.4.1")
-                        .And.RestartedFrameworkResolution("5.4.1", "5.6.0")
-                        .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.6.0"));
+                })
+                .Should().Pass()
+                .And.RestartedFrameworkResolution("5.1.1", "5.4.1")
+                .And.RestartedFrameworkResolution("5.4.1", "5.6.0")
+                .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.6.0");
         }
 
+        // Verifies that roll forward acts on all framework references (3 frameworks in chain)
         [Fact]
         public void RollForwardOnAllFrameworks()
         {
@@ -415,70 +475,23 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
                         runtimeConfig.GetFramework(MicrosoftNETCoreApp)
                             .Version = "5.0.0";
                     });
-                },
-                resultValidator: commandResult =>
-                    commandResult.Should().Pass()
-                        .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3")
-                        .And.HaveResolvedFramework(MiddleWare, "2.1.2")
-                        .And.HaveResolvedFramework(HighWare, "7.3.1"));
+                })
+                .Should().Pass()
+                .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3")
+                .And.HaveResolvedFramework(MiddleWare, "2.1.2")
+                .And.HaveResolvedFramework(HighWare, "7.3.1");
         }
 
-        private void RunTest(
+        private CommandResult RunTest(
             Func<RuntimeConfig, RuntimeConfig> runtimeConfig,
-            Action<DotNetCliExtensions.DotNetCliCustomizer> customizeDotNet = null,
-            string resolvedFramework = null,
-            Action<CommandResult> resultValidator = null)
+            Action<DotNetCliExtensions.DotNetCliCustomizer> customizeDotNet = null)
         {
-            using (DotNetCliExtensions.DotNetCliCustomizer dotnetCustomizer = SharedState.DotNetWithMultipleFrameworks.Customize())
-            {
-                customizeDotNet?.Invoke(dotnetCustomizer);
-
-                RunTest(
-                    SharedState.DotNetWithMultipleFrameworks,
-                    SharedState.FrameworkReferenceApp,
-                    new TestSettings().WithRuntimeConfigCustomizer(runtimeConfig),
-                    commandResult =>
-                    {
-                        if (resolvedFramework != null)
-                        {
-                            commandResult.Should().Pass()
-                                .And.HaveResolvedFramework(MicrosoftNETCoreApp, resolvedFramework);
-                        }
-                        else
-                        {
-                            resultValidator?.Invoke(commandResult);
-                        }
-                    });
-            }
-        }
-
-        public class SharedTestState : SharedTestStateBase
-        {
-            public TestApp FrameworkReferenceApp { get; }
-
-            public DotNetCli DotNetWithMultipleFrameworks { get; }
-
-            public SharedTestState()
-            {
-                DotNetWithMultipleFrameworks = DotNet("WithOneFramework")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.1")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.3")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.4.1")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.6.0")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("6.0.0")
-                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("6.1.1-preview.1")
-                    .AddFramework(MiddleWare, "2.1.2", runtimeConfig =>
-                        runtimeConfig.WithFramework(MicrosoftNETCoreApp, "5.1.3"))
-                    .AddFramework(AnotherMiddleWare, "3.0.0", runtimeConfig =>
-                        runtimeConfig.WithFramework(MicrosoftNETCoreApp, "5.1.3"))
-                    .AddFramework(HighWare, "7.3.1", runtimeConfig =>
-                        runtimeConfig
-                            .WithFramework(MicrosoftNETCoreApp, "5.1.3")
-                            .WithFramework(MiddleWare, "2.1.2"))
-                    .Build();
-
-                FrameworkReferenceApp = CreateFrameworkReferenceApp();
-            }
+            return RunTest(
+                SharedState.DotNetWithMultipleFrameworks,
+                SharedState.FrameworkReferenceApp,
+                new TestSettings()
+                    .WithRuntimeConfigCustomizer(runtimeConfig)
+                    .WithDotnetCustomizer(customizeDotNet));
         }
     }
 }
index e916cf0..6540bcf 100644 (file)
@@ -3,7 +3,7 @@
 // See the LICENSE file in the project root for more information.
 
 using Microsoft.DotNet.Cli.Build;
-using System;
+using Microsoft.DotNet.Cli.Build.Framework;
 using Xunit;
 
 namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
@@ -21,209 +21,174 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
             SharedState = sharedState;
         }
 
+        // Verifies the default behavior is 1 (Minor)
         [Fact]
         public void Default()
         {
             RunTest(
                 new TestSettings()
                     .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
-                        .WithFramework(MicrosoftNETCoreApp, "4.0.0")),
-                resolvedFramework: null);
+                        .WithFramework(MicrosoftNETCoreApp, "4.0.0")))
+                .Should().Fail()
+                .And.DidNotFindCompatibleFrameworkVersion();
 
             RunTest(
                 new TestSettings()
                     .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
-                        .WithFramework(MicrosoftNETCoreApp, "5.0.0")),
-                resolvedFramework: "5.1.3");
+                        .WithFramework(MicrosoftNETCoreApp, "5.0.0")))
+                .Should().Pass()
+                .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3");
         }
 
-        [Fact]
-        public void RuntimeConfigOnly()
-        {
-            RunTest(
-                new TestSettings()
-                    .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
-                        .WithRollForwardOnNoCandidateFx(2)
-                        .WithFramework(MicrosoftNETCoreApp, "4.0.0")),
-                resolvedFramework: "5.1.3");
-        }
-
-        [Fact]
-        public void FrameworkReferenceOnly()
-        {
-            RunTest(
-                new TestSettings()
-                    .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
-                        .WithFramework(new RuntimeConfig.Framework(MicrosoftNETCoreApp, "4.0.0")
-                            .WithRollForwardOnNoCandidateFx(2))),
-                resolvedFramework: "5.1.3");
-        }
-
-        [Fact]
-        public void EnvironmentVariableOnly()
-        {
-            RunTest(
-                new TestSettings()
-                    .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
-                        .WithFramework(MicrosoftNETCoreApp, "4.0.0"))
-                    .WithEnvironment(Constants.RollForwardOnNoCandidateFxSetting.EnvironmentVariable, "2"),
-                resolvedFramework: "5.1.3");
-        }
-
-        [Fact]
-        public void CommandLineOnly()
+        // Verifies that it works in all supported locations
+        [Theory]
+        [InlineData(SettingLocation.CommandLine)]
+        [InlineData(SettingLocation.Environment)]
+        [InlineData(SettingLocation.RuntimeOptions)]
+        [InlineData(SettingLocation.FrameworkReference)]
+        public void AllLocations(SettingLocation location)
         {
             RunTest(
                 new TestSettings()
                     .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
                         .WithFramework(MicrosoftNETCoreApp, "4.0.0"))
-                    .WithCommandLine(Constants.RollForwardOnNoCandidateFxSetting.CommandLineArgument, "2"),
-                resolvedFramework: "5.1.3");
+                    .With(RollForwardOnNoCandidateFxSetting(location, 2)))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3");
         }
 
-        [Theory]  // CLI wins over everything
-        [InlineData(SettingLocation.Environment, "5.1.3")]
-        [InlineData(SettingLocation.RuntimeOptions, "5.1.3")]
-        [InlineData(SettingLocation.FrameworkReference, "5.1.3")]
-        public void CommandLinePriority(SettingLocation settingLocation, string resolvedFramework)
+        // Verifies that CLI setting wins over any other <settingLocation>
+        [Theory] // settingLocation                     commandLineWins
+        [InlineData(SettingLocation.Environment,        true)]
+        [InlineData(SettingLocation.RuntimeOptions,     true)]
+        [InlineData(SettingLocation.FrameworkReference, true)]
+        public void CommandLinePriority(SettingLocation settingLocation, bool commandLineWins)
         {
             RunTest(
                 new TestSettings()
                     .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
                         .WithFramework(MicrosoftNETCoreApp, "4.0.0"))
                     .With(RollForwardOnNoCandidateFxSetting(settingLocation, 0))
-                    .WithCommandLine(Constants.RollForwardOnNoCandidateFxSetting.CommandLineArgument, "2"),
-                resolvedFramework: resolvedFramework);
+                    .WithCommandLine(Constants.RollForwardOnNoCandidateFxSetting.CommandLineArgument, "2"))
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, commandLineWins ? "5.1.3" : null);
         }
 
-        [Theory]  // Framework loses only to CLI
-        [InlineData(SettingLocation.CommandLine, null)]
-        [InlineData(SettingLocation.Environment, "5.1.3")]
-        [InlineData(SettingLocation.RuntimeOptions, "5.1.3")]
-        public void FrameworkPriority(SettingLocation settingLocation, string resolvedFramework)
+        // Verifies that framework reference setting loses only to CLI <settingLocation>
+        [Theory] // settingLocation                 frameworkReferenceWins
+        [InlineData(SettingLocation.CommandLine,    false)]
+        [InlineData(SettingLocation.Environment,    true)]
+        [InlineData(SettingLocation.RuntimeOptions, true)]
+        public void FrameworkReferencePriority(SettingLocation settingLocation, bool frameworkReferenceWins)
         {
             RunTest(
                 new TestSettings()
                     .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
                         .WithFramework(new RuntimeConfig.Framework(MicrosoftNETCoreApp, "4.0.0")
                             .WithRollForwardOnNoCandidateFx(2)))
-                    .With(RollForwardOnNoCandidateFxSetting(settingLocation, 0)),
-                resolvedFramework: resolvedFramework);
+                    .With(RollForwardOnNoCandidateFxSetting(settingLocation, 0)))
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, frameworkReferenceWins ? "5.1.3" : null);
         }
 
-        [Theory]  // Runtime config only wins over env
-        [InlineData(SettingLocation.CommandLine, null)]
-        [InlineData(SettingLocation.Environment, "5.1.3")]
-        [InlineData(SettingLocation.FrameworkReference, null)]
-        public void RuntimeConfigPriority(SettingLocation settingLocation, string resolvedFramework)
+        // Verifies that runtime options setting only wins over env. variable <settingLocation>
+        [Theory] // settingLocation                     runtimeOptionWins
+        [InlineData(SettingLocation.CommandLine,        false)]
+        [InlineData(SettingLocation.Environment,        true)]
+        [InlineData(SettingLocation.FrameworkReference, false)]
+        public void RuntimeOptionsPriority(SettingLocation settingLocation, bool runtimeOptionWins)
         {
             RunTest(
                 new TestSettings()
                     .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
                         .WithRollForwardOnNoCandidateFx(2)
                         .WithFramework(MicrosoftNETCoreApp, "4.0.0"))
-                    .With(RollForwardOnNoCandidateFxSetting(settingLocation, 0)),
-                resolvedFramework: resolvedFramework);
+                    .With(RollForwardOnNoCandidateFxSetting(settingLocation, 0)))
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, runtimeOptionWins ? "5.1.3" : null);
         }
 
-        [Theory]  // Env loses to everything else
-        [InlineData(SettingLocation.CommandLine, null)]
-        [InlineData(SettingLocation.RuntimeOptions, null)]
-        [InlineData(SettingLocation.FrameworkReference, null)]
-        public void EnvironmentPriority(SettingLocation settingLocation, string resolvedFramework)
+        // Verifies that env. variable loses to any other <settingLocation>
+        [Theory] // settingLocation                     envVariableWins
+        [InlineData(SettingLocation.CommandLine,        false)]
+        [InlineData(SettingLocation.RuntimeOptions,     false)]
+        [InlineData(SettingLocation.FrameworkReference, false)]
+        public void EnvironmentPriority(SettingLocation settingLocation, bool envVariableWins)
         {
             RunTest(
                 new TestSettings()
                     .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
                         .WithFramework(MicrosoftNETCoreApp, "4.0.0"))
                     .With(RollForwardOnNoCandidateFxSetting(settingLocation, 0))
-                    .WithEnvironment(Constants.RollForwardOnNoCandidateFxSetting.EnvironmentVariable, "2"),
-                resolvedFramework: resolvedFramework);
+                    .WithEnvironment(Constants.RollForwardOnNoCandidateFxSetting.EnvironmentVariable, "2"))
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, envVariableWins ? "5.1.3" : null);
         }
 
-        [Theory]
-        [InlineData(SettingLocation.CommandLine, null)]   // Command line overrides everything - even inner framework references
-        [InlineData(SettingLocation.RuntimeOptions, "5.1.3")]
-        [InlineData(SettingLocation.FrameworkReference, "5.1.3")]
-        [InlineData(SettingLocation.Environment, "5.1.3")]
-        public void InnerFrameworkReference(SettingLocation settingLocation, string resolvedFramework)
+        // Verifies interaction between variour <settingLocation> and inner framework reference setting
+        [Theory] // settingLocation                     innerReferenceWins
+        // Command line overrides everything - even inner framework references
+        [InlineData(SettingLocation.CommandLine,        false)]   
+        [InlineData(SettingLocation.RuntimeOptions,     true)]
+        [InlineData(SettingLocation.FrameworkReference, true)]
+        [InlineData(SettingLocation.Environment,        true)]
+        public void InnerFrameworkReference(SettingLocation settingLocation, bool innerReferenceWins)
         {
             RunTest(
                 new TestSettings()
                     .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
                         .WithFramework(new RuntimeConfig.Framework(MiddleWare, "2.1.0")))
-                    .With(RollForwardOnNoCandidateFxSetting(settingLocation, 1, MiddleWare)),
-                dotnetCustomizer => dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
-                    runtimeConfig
-                        .WithRollForwardOnNoCandidateFx(2)
-                        .GetFramework(MicrosoftNETCoreApp).Version = "4.0.0"),
-                resolvedFramework);
+                    .With(RollForwardOnNoCandidateFxSetting(settingLocation, 1, MiddleWare))
+                    .WithDotnetCustomizer(dotnetCustomizer => dotnetCustomizer
+                        .Framework(MiddleWare).RuntimeConfig(runtimeConfig => runtimeConfig
+                            .WithRollForwardOnNoCandidateFx(2)
+                            .GetFramework(MicrosoftNETCoreApp).Version = "4.0.0")))
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, innerReferenceWins ? "5.1.3" : null);
         }
 
-        [Theory]
-        [InlineData(SettingLocation.CommandLine, "5.1.3")]     // Command line overrides everything - even inner framework references
-        [InlineData(SettingLocation.RuntimeOptions, null)]     // RuntimeOptions and FrameworkReference settings are not inherited to inner reference
-        [InlineData(SettingLocation.FrameworkReference, null)] // RuntimeOptions and FrameworkReference settings are not inherited to inner reference
-        [InlineData(SettingLocation.Environment, "5.1.3")]     // Since none is specified for the inner reference, environment is used
-        public void NoInheritance_MoreRelaxed(SettingLocation settingLocation, string resolvedFramework)
+        // Verifies that there's no inheritance between app and framework when applying more relaxed setting in the app
+        [Theory] // settingLocation                     appWins
+        // Command line overrides everything - even inner framework references
+        [InlineData(SettingLocation.CommandLine,        true)]
+        // RuntimeOptions and FrameworkReference settings are not inherited to inner reference
+        [InlineData(SettingLocation.RuntimeOptions,     false)]
+        // RuntimeOptions and FrameworkReference settings are not inherited to inner reference
+        [InlineData(SettingLocation.FrameworkReference, false)]
+        // Since none is specified for the inner reference, environment is used
+        [InlineData(SettingLocation.Environment,        true)]     
+        public void NoInheritance_MoreRelaxed(SettingLocation settingLocation, bool appWins)
         {
             RunTest(
                 new TestSettings()
                     .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
                         .WithFramework(MiddleWare, "1.0.0"))
-                    .With(RollForwardOnNoCandidateFxSetting(settingLocation, 2, MiddleWare)),
-                dotnetCustomizer => dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
-                    runtimeConfig
-                        .GetFramework(MicrosoftNETCoreApp).Version = "4.0.0"),
-                resolvedFramework);
+                    .With(RollForwardOnNoCandidateFxSetting(settingLocation, 2, MiddleWare))
+                    .WithDotnetCustomizer(dotnetCustomizer => dotnetCustomizer
+                        .Framework(MiddleWare).RuntimeConfig(runtimeConfig => runtimeConfig
+                            .GetFramework(MicrosoftNETCoreApp).Version = "4.0.0")))
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, appWins ? "5.1.3" : null);
         }
 
-        [Theory]
-        [InlineData(SettingLocation.CommandLine, null)]           // Command line overrides everything - even inner framework references
-        [InlineData(SettingLocation.RuntimeOptions, "5.1.3")]     // RuntimeOptions and FrameworkReference settings are not inherited to inner reference
-        [InlineData(SettingLocation.FrameworkReference, "5.1.3")] // RuntimeOptions and FrameworkReference settings are not inherited to inner reference
-        [InlineData(SettingLocation.Environment, null)]           // Since none is specified for the inner reference, environment is used
-        public void NoInheritance_MoreRestrictive(SettingLocation settingLocation, string resolvedFramework)
+        // Verifies that there's no inheritance between app and framework when applying more strict setting in the app
+        [Theory] // settingLocation                     appWins
+        // Command line overrides everything - even inner framework references
+        [InlineData(SettingLocation.CommandLine,        true)]
+        // RuntimeOptions and FrameworkReference settings are not inherited to inner reference
+        [InlineData(SettingLocation.RuntimeOptions,     false)]
+        // RuntimeOptions and FrameworkReference settings are not inherited to inner reference
+        [InlineData(SettingLocation.FrameworkReference, false)]
+        // Since none is specified for the inner reference, environment is used
+        [InlineData(SettingLocation.Environment,        true)]           
+        public void NoInheritance_MoreRestrictive(SettingLocation settingLocation, bool appWins)
         {
             RunTest(
                 new TestSettings()
                     .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
                         .WithFramework(new RuntimeConfig.Framework(MiddleWare, "2.1.2")))
-                    .With(RollForwardOnNoCandidateFxSetting(settingLocation, 0, MiddleWare)),
-                dotnetCustomizer => dotnetCustomizer.Framework(MiddleWare).RuntimeConfig(runtimeConfig =>
-                    runtimeConfig
-                        .GetFramework(MicrosoftNETCoreApp).Version = "5.0.0"),
-                resolvedFramework);
+                    .With(RollForwardOnNoCandidateFxSetting(settingLocation, 0, MiddleWare))
+                    .WithDotnetCustomizer(dotnetCustomizer => dotnetCustomizer
+                        .Framework(MiddleWare).RuntimeConfig(runtimeConfig => runtimeConfig
+                            .GetFramework(MicrosoftNETCoreApp).Version = "5.0.0")))
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, appWins ? null : "5.1.3");
         }
 
-        private void RunTest(
-            TestSettings testSettings,
-            Action<DotNetCliExtensions.DotNetCliCustomizer> customizeDotNet = null,
-            string resolvedFramework = null)
-        {
-            using (DotNetCliExtensions.DotNetCliCustomizer dotnetCustomizer = SharedState.DotNetWithFrameworks.Customize())
-            {
-                customizeDotNet?.Invoke(dotnetCustomizer);
-
-                RunTest(
-                    SharedState.DotNetWithFrameworks,
-                    SharedState.FrameworkReferenceApp,
-                    testSettings,
-                    commandResult =>
-                    {
-                        if (resolvedFramework != null)
-                        {
-                            commandResult.Should().Pass()
-                                .And.HaveResolvedFramework(MicrosoftNETCoreApp, resolvedFramework);
-                        }
-                        else
-                        {
-                            commandResult.Should().Fail()
-                                .And.DidNotFindCompatibleFrameworkVersion();
-                        }
-                    });
-            }
-        }
+        private CommandResult RunTest(TestSettings testSettings) => 
+            RunTest(SharedState.DotNetWithFrameworks, SharedState.FrameworkReferenceApp, testSettings);
 
         public class SharedTestState : SharedTestStateBase
         {
@@ -234,6 +199,8 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
             public SharedTestState()
             {
                 DotNetWithFrameworks = DotNet("WithOneFramework")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("2.5.4")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("2.5.5")
                     .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.3")
                     .AddFramework(
                         MiddleWare, "2.1.2", 
diff --git a/src/installer/test/HostActivationTests/FrameworkResolution/RollForwardPreReleaseOnly.cs b/src/installer/test/HostActivationTests/FrameworkResolution/RollForwardPreReleaseOnly.cs
new file mode 100644 (file)
index 0000000..02dc07b
--- /dev/null
@@ -0,0 +1,314 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+// See the LICENSE file in the project root for more information.
+
+using Microsoft.DotNet.Cli.Build;
+using Microsoft.DotNet.Cli.Build.Framework;
+using Xunit;
+
+namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
+{
+    /// <summary>
+    /// Tests for rollForward option behavior considering only pre-release versions
+    /// so only pre-release versions are available and only pre-release versions are asked for
+    /// in framework references.
+    /// </summary>
+    public class RollForwardPreReleaseOnly :
+        FrameworkResolutionBase,
+        IClassFixture<RollForwardPreReleaseOnly.SharedTestState>
+    {
+        private SharedTestState SharedState { get; }
+
+        public RollForwardPreReleaseOnly(SharedTestState sharedState)
+        {
+            SharedState = sharedState;
+        }
+
+        public class SharedTestState : SharedTestStateBase
+        {
+            public TestApp FrameworkReferenceApp { get; }
+
+            public DotNetCli DotNetWithNETCoreAppPreRelease { get; }
+
+            public SharedTestState()
+            {
+                DotNetWithNETCoreAppPreRelease = DotNet("DotNetWithNETCoreAppPreRelease")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.1-preview.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.2-preview.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.2-preview.2")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.2.0-preview.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.2.1-preview.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.2.1-preview.2")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("6.1.0-preview.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("6.1.0-preview.2")
+                    .Build();
+
+                FrameworkReferenceApp = CreateFrameworkReferenceApp();
+            }
+        }
+
+        // Verifies that rollForward settings behave as expected starting with framework reference
+        // release version 5.1.0 and rolling forward to pre-release versions only with available
+        // versions starting with 5.2.1-*. So roll over patch version.
+        [Theory] // rollForward                               applyPatches resolvedFramework
+        [InlineData(Constants.RollForwardSetting.Disable,     null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null,        "5.1.2-preview.2")]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, false,       ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.Minor,       null,        "5.1.2-preview.2")]
+        [InlineData(Constants.RollForwardSetting.Minor,       false,       "5.1.1-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null,        "5.2.1-preview.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, false,       "5.2.1-preview.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       null,        "5.1.2-preview.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       false,       "5.1.1-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, null,        "6.1.0-preview.2")]
+        public void RollForwardOnPatch_FromReleaseToPreRelease(string rollForward, bool? applyPatches, string resolvedFramework)
+        {
+            RunTest(
+                "5.1.0",
+                rollForward,
+                applyPatches)
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForward settings behave as expected starting with framework reference
+        // release version 5.0.0 and rolling forward to pre-release versions only with available
+        // versions starting with 5.2.1-*. So roll over minor version.
+        [Theory] // rollForward                               applyPatches resolvedFramework
+        [InlineData(Constants.RollForwardSetting.Disable,     null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.Minor,       null,        "5.1.2-preview.2")]
+        [InlineData(Constants.RollForwardSetting.Minor,       false,       "5.1.1-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null,        "5.2.1-preview.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, false,       "5.2.1-preview.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       null,        "5.1.2-preview.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       false,       "5.1.1-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, null,        "6.1.0-preview.2")]
+        public void RollForwardOnMinor_FromReleaseToPreRelease(string rollForward, bool? applyPatches, string resolvedFramework)
+        {
+            RunTest(
+                "5.0.0",
+                rollForward,
+                applyPatches)
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForward settings behave as expected starting with framework reference
+        // release version 4.0.0 and rolling forward to pre-release versions only with available
+        // versions starting with 5.2.1-*. So roll over major version.
+        [Theory] // rollForward                               applyPatches resolvedFramework
+        [InlineData(Constants.RollForwardSetting.Disable,     null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.Minor,       null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.Major,       null,        "5.1.2-preview.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       false,       "5.1.1-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, null,        "6.1.0-preview.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, false,       "6.1.0-preview.2")]
+        public void RollForwardOnMajor_FromReleaseToPreRelease(string rollForward, bool? applyPatches, string resolvedFramework)
+        {
+            RunTest(
+                "4.0.0",
+                rollForward,
+                applyPatches)
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForward settings won't roll back (on pre-release).
+        // Starting from 5.1.2-preview.3 which is higher than any available 5.1.2 version.
+        [Theory] // rollForward                               applyPatches
+        [InlineData(Constants.RollForwardSetting.Disable,     null)]
+        [InlineData(Constants.RollForwardSetting.Disable,     false)]
+        [InlineData(Constants.RollForwardSetting.Disable,     true)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, false)]
+        public void NeverRollBackOnPreRelease_PreReleaseOnly(string rollForward, bool? applyPatches)
+        {
+            RunTest(
+                "5.1.2-preview.3",
+                rollForward,
+                applyPatches)
+                .ShouldFailToFindCompatibleFrameworkVersion();
+        }
+
+        // Verifies that rollForward settings won't roll back (on patch).
+        // Starting from 5.1.3-preview.1 which is higher than any available 5.1.* version.
+        [Theory] // rollForward                               applyPatches
+        [InlineData(Constants.RollForwardSetting.Disable,     null)]
+        [InlineData(Constants.RollForwardSetting.Disable,     false)]
+        [InlineData(Constants.RollForwardSetting.Disable,     true)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, false)]
+        public void NeverRollBackOnPatch_PreReleaseOnly(string rollForward, bool? applyPatches)
+        {
+            RunTest(
+                "5.1.3-preview.1",
+                rollForward,
+                applyPatches)
+                .ShouldFailToFindCompatibleFrameworkVersion();
+        }
+
+        // Verifies that rollForward settings won't roll back (on minor).
+        // Starting from 5.3.0-preview.1 which is higher than any available 5.*.* version.
+        [Theory] // rollForward                               applyPatches
+        [InlineData(Constants.RollForwardSetting.Disable,     null)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null)]
+        [InlineData(Constants.RollForwardSetting.Minor,       null)]
+        [InlineData(Constants.RollForwardSetting.Minor,       false)]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null)]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, false)]
+        public void NeverRollBackOnMinor_PreReleaseOnly(string rollForward, bool? applyPatches)
+        {
+            RunTest(
+                "5.3.0-preview.1",
+                rollForward,
+                applyPatches)
+                .ShouldFailToFindCompatibleFrameworkVersion();
+        }
+
+        // Verifies that rollForward settings won't roll back (on major).
+        // Starting from 7.1.0-preview.1 which is higher than any available version.
+        [Theory] // rollForward                               applyPatches
+        [InlineData(Constants.RollForwardSetting.Disable,     null)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null)]
+        [InlineData(Constants.RollForwardSetting.Minor,       null)]
+        [InlineData(Constants.RollForwardSetting.Minor,       false)]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null)]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, false)]
+        public void NeverRollBackOnMajor_PreReleaseOnly(string rollForward, bool? applyPatches)
+        {
+            RunTest(
+                "7.1.0-preview.1",
+                rollForward,
+                applyPatches)
+                .ShouldFailToFindCompatibleFrameworkVersion();
+        }
+
+        // Verifies that rollForward settings behave as expected starting with framework reference
+        // pre-release version 5.1.1-preview.1 which is available and rolling forward to pre-release versions only with available
+        // versions starting with 5.1.1-preview.1. So roll over patch version (default behavior is latest patch).
+        [Theory] // rollForward                               applyPatches rollForward
+        [InlineData(Constants.RollForwardSetting.Disable,     null,        "5.1.1-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null,        "5.1.2-preview.2")]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, false,       "5.1.1-preview.1")]
+        [InlineData(Constants.RollForwardSetting.Minor,       null,        "5.1.2-preview.2")]
+        [InlineData(Constants.RollForwardSetting.Minor,       false,       "5.1.1-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null,        "5.2.1-preview.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, false,       "5.2.1-preview.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       null,        "5.1.2-preview.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       false,       "5.1.1-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, null,        "6.1.0-preview.2")]
+        public void RollFromExisting_PreReleaseOnly(string rollForward, bool? applyPatches, string resolvedFramework)
+        {
+            RunTest(
+                "5.1.1-preview.1",
+                rollForward,
+                applyPatches)
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForward settings behave as expected starting with framework reference
+        // pre-release version 5.1.2-preview.0 and rolling forward to pre-release versions only with available
+        // versions starting with 5.1.2-preview.1. So roll over pre-release version.
+        [Theory] // rollForward                               applyPatches rollForward
+        [InlineData(Constants.RollForwardSetting.Disable,     null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null,        "5.1.2-preview.2")]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, false,       "5.1.2-preview.2")]
+        [InlineData(Constants.RollForwardSetting.Minor,       null,        "5.1.2-preview.2")]
+        [InlineData(Constants.RollForwardSetting.Minor,       false,       "5.1.2-preview.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null,        "5.2.1-preview.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, false,       "5.2.1-preview.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       null,        "5.1.2-preview.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       false,       "5.1.2-preview.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, null,        "6.1.0-preview.2")]
+        public void RollForwardOnPreRelease_PreReleaseOnly(string rollForward, bool? applyPatches, string resolvedFramework)
+        {
+            RunTest(
+                "5.1.2-preview.0",
+                rollForward,
+                applyPatches)
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForward settings behave as expected starting with framework reference
+        // pre-release version 5.1.0-preview.1 and rolling forward to pre-release versions only with available
+        // versions starting with 5.1.2-preview.1. So roll over patch version.
+        [Theory] // rollForward                               applyPatches rollForward
+        [InlineData(Constants.RollForwardSetting.Disable,     null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null,        "5.1.2-preview.2")]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, false,       ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.Minor,       null,        "5.1.2-preview.2")]
+        [InlineData(Constants.RollForwardSetting.Minor,       false,       "5.1.1-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null,        "5.2.1-preview.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, false,       "5.2.1-preview.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       null,        "5.1.2-preview.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       false,       "5.1.1-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, null,        "6.1.0-preview.2")]
+        public void RollForwardOnPatch_PreReleaseOnly(string rollForward, bool? applyPatches, string resolvedFramework)
+        {
+            RunTest(
+                "5.1.0-preview.1",
+                rollForward,
+                applyPatches)
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForward settings behave as expected starting with framework reference
+        // pre-release version 5.0.0-preview.5 and rolling forward to pre-release versions only with available
+        // versions starting with 5.1.2-preview.1. So roll over minor version.
+        [Theory] // rollForward                               applyPatches rollForward
+        [InlineData(Constants.RollForwardSetting.Disable,     null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.Minor,       null,        "5.1.2-preview.2")]
+        [InlineData(Constants.RollForwardSetting.Minor,       false,       "5.1.1-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null,        "5.2.1-preview.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, false,       "5.2.1-preview.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       null,        "5.1.2-preview.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       false,       "5.1.1-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, null,        "6.1.0-preview.2")]
+        public void RollForwardOnMinor_PreReleaseOnly(string rollForward, bool? applyPatches, string resolvedFramework)
+        {
+            RunTest(
+                "5.0.0-preview.5",
+                rollForward,
+                applyPatches)
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForward settings behave as expected starting with framework reference
+        // pre-release version 4.1.0-preview.6 and rolling forward to pre-release versions only with available
+        // versions starting with 5.1.2-preview.1. So roll over major version.
+        [Theory] // rollForward                               applyPatches rollForward
+        [InlineData(Constants.RollForwardSetting.Disable,     null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.Minor,       null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.Major,       null,        "5.1.2-preview.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       false,       "5.1.1-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, null,        "6.1.0-preview.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, false,       "6.1.0-preview.2")]
+        public void RollForwardOnMajor_PreReleaseOnly(string rollForward, bool? applyPatches, string resolvedFramework)
+        {
+            RunTest(
+                "4.1.0-preview.6",
+                rollForward,
+                applyPatches)
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        private CommandResult RunTest(
+            string frameworkReferenceVersion,
+            string rollForward,
+            bool? applyPatches)
+        {
+            return RunTest(
+                SharedState.DotNetWithNETCoreAppPreRelease,
+                SharedState.FrameworkReferenceApp,
+                new TestSettings()
+                    .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
+                        .WithApplyPatches(applyPatches)
+                        .WithFramework(MicrosoftNETCoreApp, frameworkReferenceVersion))
+                    // Using command line, so that it's possible to mix rollForward and applyPatches
+                    .With(RollForwardSetting(SettingLocation.CommandLine, rollForward)));
+        }
+    }
+}
\ No newline at end of file
diff --git a/src/installer/test/HostActivationTests/FrameworkResolution/RollForwardReleaseAndPreRelease.cs b/src/installer/test/HostActivationTests/FrameworkResolution/RollForwardReleaseAndPreRelease.cs
new file mode 100644 (file)
index 0000000..edd7ab0
--- /dev/null
@@ -0,0 +1,338 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+// See the LICENSE file in the project root for more information.
+
+using Microsoft.DotNet.Cli.Build;
+using Microsoft.DotNet.Cli.Build.Framework;
+using Xunit;
+
+namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
+{
+    /// <summary>
+    /// Tests for rollForward option behavior considering combinatino of release and pre-release versions.
+    /// so only release versions are available and only release versions are asked for
+    /// in framework references.
+    /// </summary>
+    public class RollForwardReleaseAndPreRelease :
+        FrameworkResolutionBase,
+        IClassFixture<RollForwardReleaseAndPreRelease.SharedTestState>
+    {
+        private SharedTestState SharedState { get; }
+
+        public RollForwardReleaseAndPreRelease(SharedTestState sharedState)
+        {
+            SharedState = sharedState;
+        }
+
+        public class SharedTestState : SharedTestStateBase
+        {
+            public TestApp FrameworkReferenceApp { get; }
+
+            public DotNetCli DotNetWithNETCoreAppReleaseAndPreRelease { get; }
+
+            public SharedTestState()
+            {
+                DotNetWithNETCoreAppReleaseAndPreRelease = DotNet("DotNetWithNETCoreAppReleaseAndPreRelease")
+
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("4.1.0-preview.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("4.1.0-preview.2")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("4.1.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("4.1.2-preview.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("4.1.2")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("4.1.3-preview.1")
+
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("4.5.1-preview.2")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("4.5.2-preview.1")
+
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.0-preview.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.2-preview.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.2")
+
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.5.1-preview.2")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.5.2")
+
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("6.0.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("6.0.2-preview.1")
+
+                    .Build();
+
+                FrameworkReferenceApp = CreateFrameworkReferenceApp();
+            }
+        }
+
+        // -----------------------------------
+        // Tests where the starting reference is a release version
+        //
+        // Available (relevant) framework versions (full list see above):
+        // 4.1.0-preview.1
+        // 4.1.0-preview.2
+        // 4.1.1
+        // 4.1.2-preview.1
+        // 4.1.2
+        // 4.1.3-preview.1
+
+        // Verifies that rollForward settings behave as expected when starting from 4.1.1 which does exit
+        // to other available 4.1.* versions (both release and pre-release). So roll forward on patch version.
+        // Also verifying behavior when DOTNET_ROLL_FORWARD_TO_PRERELEASE is set.
+        [Theory] // rollForward                               applyPatches rollForwardToPreRelease resolvedFramework
+        [InlineData(Constants.RollForwardSetting.Disable,     null,        false,                  "4.1.1")]
+        [InlineData(Constants.RollForwardSetting.Disable,     false,       false,                  "4.1.1")]
+        [InlineData(Constants.RollForwardSetting.Disable,     true,        false,                  "4.1.1")]
+        [InlineData(Constants.RollForwardSetting.Disable,     null,        true,                   "4.1.1")]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null,        false,                  "4.1.2")] // Prefers release over pre-release
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null,        true,                   "4.1.3-preview.1")] // Pre-release is considered equaly to release
+        [InlineData(Constants.RollForwardSetting.LatestPatch, false,       false,                  "4.1.1")]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, false,       true,                   "4.1.1")]
+        [InlineData(Constants.RollForwardSetting.Minor,       null,        false,                  "4.1.2")] // Prefers release over pre-release
+        [InlineData(Constants.RollForwardSetting.Minor,       null,        true,                   "4.1.3-preview.1")] // Pre-release is considered equaly to release
+        [InlineData(Constants.RollForwardSetting.Minor,       false,       false,                  "4.1.1")]
+        [InlineData(Constants.RollForwardSetting.Minor,       false,       true,                   "4.1.1")]
+        [InlineData(Constants.RollForwardSetting.Major,       null,        false,                  "4.1.2")] // Prefers release over pre-release
+        [InlineData(Constants.RollForwardSetting.Major,       null,        true,                   "4.1.3-preview.1")] // Pre-release is considered equaly to release
+        [InlineData(Constants.RollForwardSetting.Major,       false,       false,                  "4.1.1")]
+        [InlineData(Constants.RollForwardSetting.Major,       false,       true,                   "4.1.1")]
+        public void RollFromExisting_FromReleaseToPreRelease(
+            string rollForward,
+            bool? applyPatches,
+            bool rollForwardToPreRelease,
+            string resolvedFramework)
+        {
+            RunTest(
+                "4.1.1",
+                rollForward,
+                applyPatches,
+                rollForwardToPreRelease)
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForward settings behave as expected when starting from 4.1.0 which doesn't exist
+        // to other available 4.1.* versions (both release and pre-release). So roll forward on patch version.
+        // Also verifying behavior when DOTNET_ROLL_FORWARD_TO_PRERELEASE is set.
+        [Theory] // rollForward                               applyPatches rollForwardToPreRelease resolvedFramework
+        [InlineData(Constants.RollForwardSetting.Disable,     null,        false,                  ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.Disable,     null,        true,                   ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null,        false,                  "4.1.2")]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null,        true,                   "4.1.3-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, false,       false,                  ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, false,       true,                   ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.Minor,       null,        false,                  "4.1.2")]
+        [InlineData(Constants.RollForwardSetting.Minor,       null,        true,                   "4.1.3-preview.1")]
+        [InlineData(Constants.RollForwardSetting.Minor,       false,       false,                  "4.1.1")]
+        [InlineData(Constants.RollForwardSetting.Minor,       false,       true,                   "4.1.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null,        false,                  "4.1.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null,        true,                   "4.5.2-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, false,       false,                  "4.1.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, false,       true,                   "4.5.2-preview.1")]
+        [InlineData(Constants.RollForwardSetting.Major,       null,        false,                  "4.1.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       null,        true,                   "4.1.3-preview.1")]
+        [InlineData(Constants.RollForwardSetting.Major,       false,       false,                  "4.1.1")]
+        [InlineData(Constants.RollForwardSetting.Major,       false,       true,                   "4.1.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, null,        false,                  "6.0.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, null,        true,                   "6.0.2-preview.1")]
+        public void RollForwardOnPatch_FromReleaseToPreRelease(
+            string rollForward,
+            bool? applyPatches,
+            bool rollForwardToPreRelease,
+            string resolvedFramework)
+        {
+            RunTest(
+                "4.1.0",
+                rollForward,
+                applyPatches,
+                rollForwardToPreRelease)
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForward settings behave as expected when starting from 4.0.0 which doesn't exit
+        // to other available 4.1.* versions (both release and pre-release). So roll forward on minor version.
+        // Specifically targetting the behavior that starting from release should by default prefer release versions.
+        // Also verifying behavior when DOTNET_ROLL_FORWARD_TO_PRERELEASE is set.
+        [Theory] // rollForward                               applyPatches rollForwardToPreRelease resolvedFramework
+        [InlineData(Constants.RollForwardSetting.Minor,       null,        false,                  "4.1.2")]
+        [InlineData(Constants.RollForwardSetting.Minor,       null,        true,                   "4.1.3-preview.1")]
+        [InlineData(Constants.RollForwardSetting.Minor,       false,       false,                  "4.1.1")]
+        [InlineData(Constants.RollForwardSetting.Minor,       false,       true,                   "4.1.0-preview.1")] // Pre-release is also considered and it's the closest higher (and no patch roll to latest)
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null,        false,                  "4.1.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null,        true,                   "4.5.2-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, false,       false,                  "4.1.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, false,       true,                   "4.5.2-preview.1")]
+        [InlineData(Constants.RollForwardSetting.Major,       null,        false,                  "4.1.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       null,        true,                   "4.1.3-preview.1")]
+        [InlineData(Constants.RollForwardSetting.Major,       false,       false,                  "4.1.1")]
+        [InlineData(Constants.RollForwardSetting.Major,       false,       true,                   "4.1.0-preview.1")] // Pre-release is also considered and it's the closest higher (and no patch roll to latest)
+        [InlineData(Constants.RollForwardSetting.LatestMajor, null,        false,                  "6.0.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, null,        true,                   "6.0.2-preview.1")]
+        public void RollForwardOnMinor_FromReleaseIgnoresPreReleaseIfReleaseAvailable(
+            string rollForward,
+            bool? applyPatches,
+            bool rollForwardToPreRelease,
+            string resolvedFramework)
+        {
+            RunTest(
+                "4.0.0",
+                rollForward,
+                applyPatches,
+                rollForwardToPreRelease)
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForward settings behave as expected when starting from 3.0.0 which does exit
+        // to other available 4.1.* versions (both release and pre-release). So roll forward on major version.
+        // Specifically targetting the behavior that starting from release should by default prefer release versions.
+        // Also verifying behavior when DOTNET_ROLL_FORWARD_TO_PRERELEASE is set.
+        [Theory] // rollForward                               applyPatches rollForwardToPreRelease resolvedFramework
+        [InlineData(Constants.RollForwardSetting.Major,       null,        false,                  "4.1.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       null,        true,                   "4.1.3-preview.1")]
+        [InlineData(Constants.RollForwardSetting.Major,       false,       false,                  "4.1.1")]
+        [InlineData(Constants.RollForwardSetting.Major,       false,       true,                   "4.1.0-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, null,        false,                  "6.0.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, null,        true,                   "6.0.2-preview.1")]
+        public void RollForwardOnMajor_FromReleaseIgnoresPreReleaseIfReleaseAvailable(
+            string rollForward,
+            bool? applyPatches,
+            bool rollForwardToPreRelease,
+            string resolvedFramework)
+        {
+            RunTest(
+                "3.0.0",
+                rollForward,
+                applyPatches,
+                rollForwardToPreRelease)
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+
+        // -----------------------------------
+        // Tests where the starting reference is a pre-release version
+        //
+        // Available (relevant) framework versions (full list see above):
+        // 5.1.0-preview.1
+        // 5.1.1
+        // 5.1.2-preview.1
+        // 5.1.2
+
+        // Verifies that rollForward settings behave as expected when starting from 5.1.0-preview.1 which does exist
+        // to other available 5.1.* versions (both release and pre-release). So roll forward on patch version.
+        // Starting from pre-release means that all versions are always considered (both release and pre-release).
+        [Theory] // rollForward                               applyPatches resolvedFramework
+        [InlineData(Constants.RollForwardSetting.Disable,     null,        "5.1.0-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null,        "5.1.2")]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, false,       "5.1.0-preview.1")]
+        [InlineData(Constants.RollForwardSetting.Minor,       null,        "5.1.2")]
+        [InlineData(Constants.RollForwardSetting.Minor,       false,       "5.1.0-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null,        "5.5.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, false,       "5.5.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       null,        "5.1.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       false,       "5.1.0-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, null,        "6.0.2-preview.1")]
+        public void RollFromExisting_FromPreReleaseToRelease(string rollForward, bool? applyPatches, string resolvedFramework)
+        {
+            RunTest(
+                "5.1.0-preview.1",
+                rollForward,
+                applyPatches)
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForward settings behave as expected when starting from 5.1.0-preview.0 which doesn't exist
+        // to other available 5.1.* versions (both release and pre-release). So roll forward on patch version.
+        // Starting from pre-release means that all versions are always considered (both release and pre-release).
+        [Theory] // rollForward                               applyPatches resolvedFramework
+        [InlineData(Constants.RollForwardSetting.Disable,     null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null,        "5.1.2")]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, false,       "5.1.0-preview.1")]
+        [InlineData(Constants.RollForwardSetting.Minor,       null,        "5.1.2")]
+        [InlineData(Constants.RollForwardSetting.Minor,       false,       "5.1.0-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null,        "5.5.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, false,       "5.5.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       null,        "5.1.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       false,       "5.1.0-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, null,        "6.0.2-preview.1")]
+        public void RollForwardOnPatch_FromPreReleaseToRelease(string rollForward, bool? applyPatches, string resolvedFramework)
+        {
+            RunTest(
+                "5.1.0-preview.0",
+                rollForward,
+                applyPatches)
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForward settings behave as expected when starting from 5.0.0-preview.5
+        // to other available 5.*.* versions (both release and pre-release). So roll forward on minor version.
+        // Starting from pre-release means that all versions are always considered (both release and pre-release).
+        [Theory] // rollForward                               applyPatches resolvedFramework
+        [InlineData(Constants.RollForwardSetting.Minor,       null,        "5.1.2")]
+        [InlineData(Constants.RollForwardSetting.Minor,       false,       "5.1.0-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null,        "5.5.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, false,       "5.5.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       null,        "5.1.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       false,       "5.1.0-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, null,        "6.0.2-preview.1")]
+        public void RollForwardOnMinor_FromPreReleaseToRelease(string rollForward, bool? applyPatches, string resolvedFramework)
+        {
+            RunTest(
+                "5.0.0-preview.5",
+                rollForward,
+                applyPatches)
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForward settings behave as expected when starting from 4.9.0-preview.6
+        // to other available 5.*.* versions (both release and pre-release). So roll forward on major version.
+        // Starting from pre-release means that all versions are always considered (both release and pre-release).
+        [Theory] // rollForward                               applyPatches resolvedFramework
+        [InlineData(Constants.RollForwardSetting.Major,       null,        "5.1.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       false,       "5.1.0-preview.1")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, null,        "6.0.2-preview.1")]
+        public void RollForwardOnMajor_FromPreReleaseToRelease(string rollForward, bool? applyPatches, string resolvedFramework)
+        {
+            RunTest(
+                "4.9.0-preview.6",
+                rollForward,
+                applyPatches)
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Just a sanity check, DOTNET_ROLL_FORWARD_TO_PRERELEASE should have no effect if the framework reference is pre-release
+        [Theory] // rollForwardToPreRelease
+        [InlineData(false)]
+        [InlineData(true)]
+        public void RollForwardOnPatch_FromPreReleaseToRelease_RollForwardToPreRelease(bool rollForwardToPreRelease)
+        {
+            // Defaults
+            RunTest(
+                "5.1.0-preview.0",
+                rollForward: null,
+                applyPatches: null,
+                rollForwardToPreRelease: rollForwardToPreRelease)
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.2");
+
+            // Minor, applyPatches=false
+            RunTest(
+                "5.1.0-preview.0",
+                Constants.RollForwardSetting.Minor,
+                applyPatches: false,
+                rollForwardToPreRelease: rollForwardToPreRelease)
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.0-preview.1");
+        }
+
+        private CommandResult RunTest(
+            string frameworkReferenceVersion,
+            string rollForward,
+            bool? applyPatches,
+            bool rollForwardToPreRelease = false)
+        {
+            return RunTest(
+                SharedState.DotNetWithNETCoreAppReleaseAndPreRelease,
+                SharedState.FrameworkReferenceApp,
+                new TestSettings()
+                    .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
+                        .WithApplyPatches(applyPatches)
+                        .WithFramework(MicrosoftNETCoreApp, frameworkReferenceVersion))
+                    // Using command line, so that it's possible to mix rollForward and applyPatches
+                    .With(RollForwardSetting(SettingLocation.CommandLine, rollForward))
+                    .WithEnvironment(Constants.RollForwardToPreRelease.EnvironmentVariable, rollForwardToPreRelease ? "1" : "0"));
+        }
+    }
+}
diff --git a/src/installer/test/HostActivationTests/FrameworkResolution/RollForwardReleaseOnly.cs b/src/installer/test/HostActivationTests/FrameworkResolution/RollForwardReleaseOnly.cs
new file mode 100644 (file)
index 0000000..605bc56
--- /dev/null
@@ -0,0 +1,227 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+// See the LICENSE file in the project root for more information.
+
+using Microsoft.DotNet.Cli.Build;
+using Microsoft.DotNet.Cli.Build.Framework;
+using Xunit;
+
+namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
+{
+    /// <summary>
+    /// Tests for rollForward option behavior considering only release versions
+    /// so only release versions are available and only release versions are asked for
+    /// in framework references.
+    /// </summary>
+    public class RollForwardReleaseOnly :
+        FrameworkResolutionBase,
+        IClassFixture<RollForwardReleaseOnly.SharedTestState>
+    {
+        private SharedTestState SharedState { get; }
+
+        public RollForwardReleaseOnly(SharedTestState sharedState)
+        {
+            SharedState = sharedState;
+        }
+
+        public class SharedTestState : SharedTestStateBase
+        {
+            public TestApp FrameworkReferenceApp { get; }
+
+            public DotNetCli DotNetWithNETCoreAppRelease { get; }
+
+            public SharedTestState()
+            {
+                DotNetWithNETCoreAppRelease = DotNet("DotNetWithNETCoreAppRelease")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("2.1.2")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("2.1.3")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("2.4.0")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("2.4.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("3.1.1")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("3.1.2")
+                    .Build();
+
+                FrameworkReferenceApp = CreateFrameworkReferenceApp();
+            }
+        }
+
+        // Verifies that exact version match is resolved by default
+        [Fact]
+        public void ExactMatchOnRelease()
+        {
+            RunTest(
+                frameworkReferenceVersion: "2.1.3",
+                rollForward: null,
+                applyPatches: null)
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "2.1.3");
+        }
+
+        // Verifies that rollForward settings behave as expected when starting from 2.1.2 which does exist
+        // to other available 2.1.* versions. So roll forward on patch version.
+        [Theory] // rollForward                               applyPatches resolvedFramework
+        [InlineData(Constants.RollForwardSetting.Disable,     null,        "2.1.2")]
+        // applyPatches is ignored for new rollForward settings
+        [InlineData(Constants.RollForwardSetting.Disable,     false,       "2.1.2")]
+        // applyPatches is ignored for new rollForward settings
+        [InlineData(Constants.RollForwardSetting.Disable,     true,        "2.1.2")]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null,        "2.1.3")]
+        // Backward compat, equivalent to rollForwardOnNoCadidateFx=0, applyPatches=false
+        [InlineData(Constants.RollForwardSetting.LatestPatch, false,       "2.1.2")]
+        [InlineData(Constants.RollForwardSetting.Minor,       null,        "2.1.3")]
+        // Backward compat, equivalent to rollForwardOnNoCadidateFx=1, applyPatches=false
+        [InlineData(Constants.RollForwardSetting.Minor,       false,       "2.1.2")]
+        [InlineData(Constants.RollForwardSetting.Major,       null,        "2.1.3")]
+        // Backward compat, equivalent to rollForwardOnNoCadidateFx=2, applyPatches=false
+        [InlineData(Constants.RollForwardSetting.Major,       false,       "2.1.2")]
+        public void RollFromExisting_ReleaseOnly(string rollForward, bool? applyPatches, string resolvedFramework)
+        {
+            RunTest(
+                "2.1.2",
+                rollForward,
+                applyPatches)
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForward settings behave as expected when starting from 2.1.0 which doesn't exist
+        // to other available 2.1.* versions. So roll forward on patch version.
+        [Theory] // rollForward                               applyPatches resolvedFramework
+        [InlineData(Constants.RollForwardSetting.Disable,     null,        ResolvedFramework.NotFound)]
+        // applyPatches is ignored for new rollForward settings
+        [InlineData(Constants.RollForwardSetting.Disable,     false,       ResolvedFramework.NotFound)]
+        // applyPatches is ignored for new rollForward settings
+        [InlineData(Constants.RollForwardSetting.Disable,     true,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null,        "2.1.3")]
+        // Backward compat, equivalent to rollForwardOnNoCadidateFx=0, applyPatches=false
+        [InlineData(Constants.RollForwardSetting.LatestPatch, false,       ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.Minor,       null,        "2.1.3")]
+        // Backward compat, equivalent to rollForwardOnNoCadidateFx=1, applyPatches=false
+        [InlineData(Constants.RollForwardSetting.Minor,       false,       "2.1.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null,        "2.4.1")]
+        // applyPatches is ignored for new rollForward settings
+        [InlineData(Constants.RollForwardSetting.LatestMinor, false,       "2.4.1")]
+        [InlineData(Constants.RollForwardSetting.Major,       null,        "2.1.3")]
+        // Backward compat, equivalent to rollForwardOnNoCadidateFx=2, applyPatches=false
+        [InlineData(Constants.RollForwardSetting.Major,       false,       "2.1.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, null,        "3.1.2")]
+        public void RollForwardOnPatch_ReleaseOnly(string rollForward, bool? applyPatches, string resolvedFramework)
+        {
+            RunTest(
+                "2.1.0",
+                rollForward,
+                applyPatches)
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForward settings behave as expected when starting from 2.0.0
+        // to other available 2.*.* and higher versions. So roll forward on minor version.
+        [Theory] // rollForward                               applyPatches resolvedFramework
+        [InlineData(Constants.RollForwardSetting.Disable,     null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.Minor,       null,        "2.1.3")]
+        // Backward compat, equivalent to rollForwardOnNoCadidateFx=1, applyPatches=false
+        [InlineData(Constants.RollForwardSetting.Minor,       false,       "2.1.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null,        "2.4.1")]
+        // applyPatches is ignored for new rollForward settings
+        [InlineData(Constants.RollForwardSetting.LatestMinor, false,       "2.4.1")]
+        [InlineData(Constants.RollForwardSetting.Major,       null,        "2.1.3")]
+        // Backward compat, equivalent to rollForwardOnNoCadidateFx=2, applyPatches=false
+        [InlineData(Constants.RollForwardSetting.Major,       false,       "2.1.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, null,        "3.1.2")]
+        public void RollForwardOnMinor_ReleaseOnly(string rollForward, bool? applyPatches, string resolvedFramework)
+        {
+            RunTest(
+                "2.0.0",
+                rollForward,
+                applyPatches)
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verifies that rollForward settings behave as expected when starting from 1.0.0
+        // to other available 2.*.* and higher versions. So roll forward on major version.
+        [Theory] // rollForward                               applyPatches resolvedFramework
+        [InlineData(Constants.RollForwardSetting.Disable,     null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.Minor,       null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null,        ResolvedFramework.NotFound)]
+        [InlineData(Constants.RollForwardSetting.Major,       null,        "2.1.3")]
+        // Backward compat, equivalent to rollForwardOnNoCadidateFx=2, applyPatches=false
+        [InlineData(Constants.RollForwardSetting.Major,       false,       "2.1.2")]
+        [InlineData(Constants.RollForwardSetting.LatestMajor, null,        "3.1.2")]
+        // applyPatches is ignored for new rollForward settings
+        [InlineData(Constants.RollForwardSetting.LatestMajor, false,       "3.1.2")]
+        public void RollForwardOnMajor_ReleaseOnly(string rollForward, bool? applyPatches, string resolvedFramework)
+        {
+            RunTest(
+                "1.1.0",
+                rollForward,
+                applyPatches)
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, resolvedFramework);
+        }
+
+        // Verify that rollForward settings will never roll back to lower patch version.
+        [Theory] // rollForward                               applyPatches
+        [InlineData(Constants.RollForwardSetting.Disable,     null)]
+        [InlineData(Constants.RollForwardSetting.Disable,     false)]
+        [InlineData(Constants.RollForwardSetting.Disable,     true)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, false)]
+        public void NeverRollBackOnPatch_ReleaseOnly(string rollForward, bool? applyPatches)
+        {
+            RunTest(
+                "2.1.4",
+                rollForward,
+                applyPatches)
+                .ShouldFailToFindCompatibleFrameworkVersion();
+        }
+
+        // Verify that rollForward settings will never roll back to lower minor version.
+        [Theory] // rollForward                               applyPatches
+        [InlineData(Constants.RollForwardSetting.Disable,     null)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null)]
+        [InlineData(Constants.RollForwardSetting.Minor,       null)]
+        [InlineData(Constants.RollForwardSetting.Minor,       false)]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null)]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, false)]
+        public void NeverRollBackOnMinor_ReleaseOnly(string rollForward, bool? applyPatches)
+        {
+            RunTest(
+                "2.5.0",
+                rollForward,
+                applyPatches)
+                .ShouldFailToFindCompatibleFrameworkVersion();
+        }
+
+        // Verify that rollForward settings will never roll back to lower major version.
+        [Theory] // rollForward                               applyPatches
+        [InlineData(Constants.RollForwardSetting.Disable,     null)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch, null)]
+        [InlineData(Constants.RollForwardSetting.Minor,       null)]
+        [InlineData(Constants.RollForwardSetting.Minor,       false)]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, null)]
+        [InlineData(Constants.RollForwardSetting.LatestMinor, false)]
+        public void NeverRollBackOnMajor_ReleaseOnly(string rollForward, bool? applyPatches)
+        {
+            RunTest(
+                "4.1.0",
+                rollForward,
+                applyPatches)
+                .ShouldFailToFindCompatibleFrameworkVersion();
+        }
+
+        private CommandResult RunTest(
+            string frameworkReferenceVersion,
+            string rollForward,
+            bool? applyPatches)
+        {
+            return RunTest(
+                SharedState.DotNetWithNETCoreAppRelease,
+                SharedState.FrameworkReferenceApp,
+                new TestSettings()
+                    .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
+                        .WithApplyPatches(applyPatches)
+                        .WithFramework(MicrosoftNETCoreApp, frameworkReferenceVersion))
+                   // Using command line, so that it's possible to mix rollForward and applyPatches
+                   .With(RollForwardSetting(SettingLocation.CommandLine, rollForward)));
+        }
+    }
+}
\ No newline at end of file
diff --git a/src/installer/test/HostActivationTests/FrameworkResolution/RollForwardSettings.cs b/src/installer/test/HostActivationTests/FrameworkResolution/RollForwardSettings.cs
new file mode 100644 (file)
index 0000000..e7a91d0
--- /dev/null
@@ -0,0 +1,277 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+// See the LICENSE file in the project root for more information.
+
+using Microsoft.DotNet.Cli.Build;
+using Microsoft.DotNet.Cli.Build.Framework;
+using Xunit;
+
+namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.FrameworkResolution
+{
+    public class RollForwardSettings :
+        FrameworkResolutionBase,
+        IClassFixture<RollForwardSettings.SharedTestState>
+    {
+        private const string MiddleWare = "MiddleWare";
+
+        private SharedTestState SharedState { get; }
+
+        public RollForwardSettings(SharedTestState sharedState)
+        {
+            SharedState = sharedState;
+        }
+
+        // Verifies that default behavior is Minor
+        [Fact]
+        public void Default()
+        {
+            RunTest(
+                new TestSettings()
+                    .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
+                        .WithFramework(MicrosoftNETCoreApp, "4.0.0")))
+                .Should().Fail()
+                .And.DidNotFindCompatibleFrameworkVersion();
+
+            RunTest(
+                new TestSettings()
+                    .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
+                        .WithFramework(MicrosoftNETCoreApp, "5.0.0")))
+                .Should().Pass()
+                .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3");
+        }
+
+        // Verifies that invalid values is checked in all settings locations
+        [Theory]
+        [InlineData(SettingLocation.CommandLine)]
+        [InlineData(SettingLocation.Environment)]
+        [InlineData(SettingLocation.RuntimeOptions)]
+        [InlineData(SettingLocation.FrameworkReference)]
+        public void InvalidValue(SettingLocation settingLocation)
+        {
+            RunTest(
+                new TestSettings()
+                    .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
+                        .WithFramework(MicrosoftNETCoreApp, "4.0.0"))
+                    .With(RollForwardSetting(settingLocation, "InvalidValue")))
+                .Should().Fail()
+                .And.DidNotRecognizeRollForwardValue("InvalidValue");
+        }
+
+        // Verifies that the value ignores casing on command line
+        [Theory]
+        [InlineData(Constants.RollForwardSetting.Disable)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch)]
+        [InlineData(Constants.RollForwardSetting.Minor)]
+        [InlineData(Constants.RollForwardSetting.LatestMinor)]
+        [InlineData(Constants.RollForwardSetting.Major)]
+        [InlineData(Constants.RollForwardSetting.LatestMajor)]
+        public void ValueIgnoresCase_CommandLine(string rollForward)
+        {
+            ValidateValueIgnoresCase(SettingLocation.CommandLine, rollForward);
+        }
+
+        // Verifies that the value ignores casing in env. variable
+        [Theory]
+        [InlineData(Constants.RollForwardSetting.Disable)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch)]
+        [InlineData(Constants.RollForwardSetting.Minor)]
+        [InlineData(Constants.RollForwardSetting.LatestMinor)]
+        [InlineData(Constants.RollForwardSetting.Major)]
+        [InlineData(Constants.RollForwardSetting.LatestMajor)]
+        public void ValueIgnoresCase_Environment(string rollForward)
+        {
+            ValidateValueIgnoresCase(SettingLocation.Environment, rollForward);
+        }
+
+        // Verifies that the value ignores casing in the runtime options
+        [Theory]
+        [InlineData(Constants.RollForwardSetting.Disable)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch)]
+        [InlineData(Constants.RollForwardSetting.Minor)]
+        [InlineData(Constants.RollForwardSetting.LatestMinor)]
+        [InlineData(Constants.RollForwardSetting.Major)]
+        [InlineData(Constants.RollForwardSetting.LatestMajor)]
+        public void ValueIgnoresCase_RuntimeOptions(string rollForward)
+        {
+            ValidateValueIgnoresCase(SettingLocation.RuntimeOptions, rollForward);
+        }
+
+        // Verifies that the value ignores casing in the framework reference
+        [Theory]
+        [InlineData(Constants.RollForwardSetting.Disable)]
+        [InlineData(Constants.RollForwardSetting.LatestPatch)]
+        [InlineData(Constants.RollForwardSetting.Minor)]
+        [InlineData(Constants.RollForwardSetting.LatestMinor)]
+        [InlineData(Constants.RollForwardSetting.Major)]
+        [InlineData(Constants.RollForwardSetting.LatestMajor)]
+        public void ValueIgnoresCase_FrameworkReference(string rollForward)
+        {
+            ValidateValueIgnoresCase(SettingLocation.FrameworkReference, rollForward);
+        }
+
+        private void ValidateValueIgnoresCase(SettingLocation settingLocation, string rollForward)
+        {
+            string[] values = new string[]
+            {
+                rollForward,
+                rollForward.ToLowerInvariant(),
+                rollForward.ToUpperInvariant()
+            };
+
+            foreach (string value in values)
+            {
+                RunTest(
+                    new TestSettings()
+                        .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
+                            .WithFramework(MicrosoftNETCoreApp, "5.1.3"))
+                        .With(RollForwardSetting(settingLocation, value)))
+                    .Should().Pass()
+                    .And.HaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3");
+            }
+        }
+
+        // Verifies that rollForward and rollForwardOnNoCandidateFx can't be used both on a command line
+        [Fact]
+        public void CollisionsOnCommandLine_RollForwardOnNoCandidateFx()
+        {
+            RunTest(
+                new TestSettings()
+                    .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
+                        .WithFramework(MicrosoftNETCoreApp, "4.0.0"))
+                    .WithCommandLine(Constants.RollForwardSetting.CommandLineArgument, Constants.RollForwardSetting.LatestPatch)
+                    .WithCommandLine(Constants.RollForwardOnNoCandidateFxSetting.CommandLineArgument, "2"))
+                .Should().Fail()
+                .And.HaveStdErrContaining(
+                    $"It's invalid to use both '{Constants.RollForwardSetting.CommandLineArgument}' and " +
+                    $"'{Constants.RollForwardOnNoCandidateFxSetting.CommandLineArgument}' command line options.");
+        }
+
+        // Verifies that rollForward can't be used together with rollForwrdOnNoCandidateFx or applyPatches in the same runtime config
+        [Theory] // rollForwardLocation                 rollForwardOnNoCandidateFxLocation  applyPatchesLocation                passes
+        [InlineData(SettingLocation.RuntimeOptions,     SettingLocation.None,               SettingLocation.None,               true )]
+        [InlineData(SettingLocation.RuntimeOptions,     SettingLocation.RuntimeOptions,     SettingLocation.None,               false)]
+        [InlineData(SettingLocation.RuntimeOptions,     SettingLocation.FrameworkReference, SettingLocation.None,               false)]
+        [InlineData(SettingLocation.RuntimeOptions,     SettingLocation.None,               SettingLocation.RuntimeOptions,     false)]
+        [InlineData(SettingLocation.RuntimeOptions,     SettingLocation.None,               SettingLocation.FrameworkReference, false)]
+        [InlineData(SettingLocation.RuntimeOptions,     SettingLocation.RuntimeOptions,     SettingLocation.RuntimeOptions,     false)]
+        [InlineData(SettingLocation.RuntimeOptions,     SettingLocation.FrameworkReference, SettingLocation.FrameworkReference, false)]
+        [InlineData(SettingLocation.FrameworkReference, SettingLocation.None,               SettingLocation.None,               true )]
+        [InlineData(SettingLocation.FrameworkReference, SettingLocation.RuntimeOptions,     SettingLocation.None,               false)]
+        [InlineData(SettingLocation.FrameworkReference, SettingLocation.FrameworkReference, SettingLocation.None,               false)]
+        [InlineData(SettingLocation.FrameworkReference, SettingLocation.None,               SettingLocation.RuntimeOptions,     false)]
+        [InlineData(SettingLocation.FrameworkReference, SettingLocation.None,               SettingLocation.FrameworkReference, false)]
+        [InlineData(SettingLocation.FrameworkReference, SettingLocation.RuntimeOptions,     SettingLocation.RuntimeOptions,     false)]
+        [InlineData(SettingLocation.FrameworkReference, SettingLocation.FrameworkReference, SettingLocation.FrameworkReference, false)]
+        public void CollisionsInRuntimeConfig(
+            SettingLocation rollForwardLocation,
+            SettingLocation rollForwardOnNoCandidateFxLocation,
+            SettingLocation applyPatchesLocation,
+            bool passes)
+        {
+            CommandResult result = RunTest(
+                new TestSettings()
+                    .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
+                        .WithFramework(MicrosoftNETCoreApp, "5.0.0"))
+                    .With(RollForwardSetting(rollForwardLocation, Constants.RollForwardSetting.Minor))
+                    .With(RollForwardOnNoCandidateFxSetting(rollForwardOnNoCandidateFxLocation, 1))
+                    .With(ApplyPatchesSetting(applyPatchesLocation, false)));
+
+            if (passes)
+            {
+                result.ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3");
+            }
+            else
+            {
+                result.Should().Fail()
+                      .And.HaveStdErrContaining(
+                        $"It's invalid to use both `{Constants.RollForwardSetting.RuntimeConfigPropertyName}` and one of " +
+                        $"`{Constants.RollForwardOnNoCandidateFxSetting.RuntimeConfigPropertyName}` or " +
+                        $"`{Constants.ApplyPatchesSetting.RuntimeConfigPropertyName}` in the same runtime config.");
+            }
+        }
+
+        // Verifies that there's no inheritance between app and framework when applying more relaxed setting in the app
+        [Theory] // settingLocation                     appWins
+        // Command line overrides everything - even inner framework references
+        [InlineData(SettingLocation.CommandLine,        true)]
+        // RuntimeOptions and FrameworkReference settings are not inherited to inner reference
+        [InlineData(SettingLocation.RuntimeOptions,     false)]
+        // RuntimeOptions and FrameworkReference settings are not inherited to inner reference
+        [InlineData(SettingLocation.FrameworkReference, false)]
+        // Since none is specified for the inner reference, environment is used
+        [InlineData(SettingLocation.Environment,        true)]     
+        public void NoInheritance_MoreRelaxed(SettingLocation settingLocation, bool appWins)
+        {
+            RunTest(
+                new TestSettings()
+                    .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
+                        .WithFramework(MiddleWare, "1.0.0"))
+                    .With(RollForwardSetting(settingLocation, Constants.RollForwardSetting.Major, MiddleWare))
+                    .WithDotnetCustomizer(dotnetCustomizer => dotnetCustomizer
+                        .Framework(MiddleWare).RuntimeConfig(runtimeConfig => runtimeConfig
+                            .GetFramework(MicrosoftNETCoreApp).Version = "4.0.0")))
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, appWins ? "5.1.3" : null);
+        }
+
+        // Verifies that there's no inheritance between app and framework when applying more strict setting in the app
+        [Theory] // settingLocation                     appWins
+        // Command line overrides everything - even inner framework references
+        [InlineData(SettingLocation.CommandLine,        true)]
+        // RuntimeOptions and FrameworkReference settings are not inherited to inner reference
+        [InlineData(SettingLocation.RuntimeOptions,     false)]
+        // RuntimeOptions and FrameworkReference settings are not inherited to inner reference
+        [InlineData(SettingLocation.FrameworkReference, false)]
+        // Since none is specified for the inner reference, environment is used
+        [InlineData(SettingLocation.Environment,        true)]           
+        public void NoInheritance_MoreRestrictive(SettingLocation settingLocation, bool appWins)
+        {
+            RunTest(
+                new TestSettings()
+                    .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
+                        .WithFramework(new RuntimeConfig.Framework(MiddleWare, "2.1.2")))
+                    .With(RollForwardSetting(settingLocation, Constants.RollForwardSetting.LatestPatch, MiddleWare))
+                    .WithDotnetCustomizer(dotnetCustomizer => dotnetCustomizer
+                        .Framework(MiddleWare).RuntimeConfig(runtimeConfig => runtimeConfig
+                            .GetFramework(MicrosoftNETCoreApp).Version = "5.0.0")))
+                .ShouldHaveResolvedFrameworkOrFailToFind(MicrosoftNETCoreApp, appWins ? null : "5.1.3");
+        }
+
+        // Verifies that the setting works in all supported locations
+        [Theory]
+        [InlineData(SettingLocation.CommandLine)]
+        [InlineData(SettingLocation.Environment)]
+        [InlineData(SettingLocation.RuntimeOptions)]
+        [InlineData(SettingLocation.FrameworkReference)]
+        public void AllLocations(SettingLocation location)
+        {
+            RunTest(
+                new TestSettings()
+                    .WithRuntimeConfigCustomizer(runtimeConfig => runtimeConfig
+                        .WithFramework(MicrosoftNETCoreApp, "4.0.0"))
+                    .With(RollForwardSetting(location, Constants.RollForwardSetting.Major)))
+                .ShouldHaveResolvedFramework(MicrosoftNETCoreApp, "5.1.3");
+        }
+
+        private CommandResult RunTest(TestSettings testSettings) =>
+            RunTest(SharedState.DotNetWithFrameworks, SharedState.FrameworkReferenceApp, testSettings);
+
+        public class SharedTestState : SharedTestStateBase
+        {
+            public TestApp FrameworkReferenceApp { get; }
+
+            public DotNetCli DotNetWithFrameworks { get; }
+
+            public SharedTestState()
+            {
+                DotNetWithFrameworks = DotNet("WithOneFramework")
+                    .AddMicrosoftNETCoreAppFrameworkMockHostPolicy("5.1.3")
+                    .AddFramework(
+                        MiddleWare, "2.1.2",
+                        runtimeConfig => runtimeConfig.WithFramework(MicrosoftNETCoreApp, "5.1.3"))
+                    .Build();
+
+                FrameworkReferenceApp = CreateFrameworkReferenceApp();
+            }
+        }
+    }
+}
index 9fc8fa4..c47ad6f 100644 (file)
@@ -18,6 +18,7 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation
             public string Name { get; }
             public string Version { get; set;  }
 
+            public string RollForward { get; set; }
             public int? RollForwardOnNoCandidateFx { get; set; }
             public bool? ApplyPatches { get; set; }
 
@@ -27,6 +28,12 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation
                 Version = version;
             }
 
+            public Framework WithRollForward(string value)
+            {
+                RollForward = value;
+                return this;
+            }
+
             public Framework WithRollForwardOnNoCandidateFx(int? value)
             {
                 RollForwardOnNoCandidateFx = value;
@@ -47,6 +54,13 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation
                         new JProperty("version", Version)
                         );
 
+                if (RollForward != null)
+                {
+                    frameworkReference.Add(
+                        Constants.RollForwardSetting.RuntimeConfigPropertyName,
+                        RollForward);
+                }
+
                 if (RollForwardOnNoCandidateFx.HasValue)
                 {
                     frameworkReference.Add(
@@ -68,12 +82,14 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation
             {
                 return new Framework((string)jobject["name"], (string)jobject["version"])
                 {
+                    RollForward = (string)jobject[Constants.RollForwardSetting.RuntimeConfigPropertyName],
                     RollForwardOnNoCandidateFx = (int?)jobject[Constants.RollForwardOnNoCandidateFxSetting.RuntimeConfigPropertyName],
                     ApplyPatches = (bool?)jobject[Constants.ApplyPatchesSetting.RuntimeConfigPropertyName]
                 };
             }
         }
 
+        private string _rollForward;
         private int? _rollForwardOnNoCandidateFx;
         private bool? _applyPatches;
         private readonly string _path;
@@ -126,6 +142,7 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation
                         }
                     }
 
+                    runtimeConfig._rollForward = (string)runtimeOptions[Constants.RollForwardSetting.RuntimeConfigPropertyName];
                     runtimeConfig._rollForwardOnNoCandidateFx = (int?)runtimeOptions[Constants.RollForwardOnNoCandidateFxSetting.RuntimeConfigPropertyName];
                     runtimeConfig._applyPatches = (bool?)runtimeOptions[Constants.ApplyPatchesSetting.RuntimeConfigPropertyName];
                 }
@@ -155,6 +172,12 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation
             return WithFramework(new Framework(name, version));
         }
 
+        public RuntimeConfig WithRollForward(string value)
+        {
+            _rollForward = value;
+            return this;
+        }
+
         public RuntimeConfig WithRollForwardOnNoCandidateFx(int? value)
         {
             _rollForwardOnNoCandidateFx = value;
@@ -180,6 +203,13 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation
                     { "frameworks", new JArray(_frameworks.Select(f => f.ToJson()).ToArray()) }
                 };
 
+            if (_rollForward != null)
+            {
+                runtimeOptions.Add(
+                    Constants.RollForwardSetting.RuntimeConfigPropertyName,
+                    _rollForward);
+            }
+
             if (_rollForwardOnNoCandidateFx.HasValue)
             {
                 runtimeOptions.Add(