# Testing Libraries on Android
+## Prerequisites
+
The following dependencies should be installed in order to be able to run tests:
+- OpenJDK
- Android NDK
- Android SDK
-- OpenJDK
-- OpenSSL
+
+To manage the dependencies, you can install them via terminal or using Android Studio.
+
+### Using a terminal
OpenJDK can be installed on Linux (Ubuntu) using `apt-get`:
```bash
sudo apt-get install openjdk-8 zip unzip
```
-Android SDK, NDK and OpenSSL can be automatically installed via the following script:
+Android SDK and NDK can be automatically installed via the following script:
```bash
#!/usr/bin/env bash
set -e
SDK_VER=6200805_latest
SDK_API_LEVEL=29
SDK_BUILD_TOOLS=29.0.3
-OPENSSL_VER=1.1.1g-alpha-1
if [[ "$OSTYPE" == "darwin"* ]]; then
HOST_OS=darwin
${ANDROID_SDK_ROOT}/cmdline-tools/tools/bin/sdkmanager --sdk_root=${ANDROID_SDK_ROOT} "platform-tools" "platforms;android-${SDK_API_LEVEL}" "build-tools;${SDK_BUILD_TOOLS}"
```
+### Using Android Studio
+
+Android Studio offers a convenient UI:
+- to install all the dependencies;
+- to manage android virtual devices;
+- to make easy use of adb logs.
+
## Building Libs and Tests for Android
+Before running a build you might want to set the Android SDK and NDK environment variables:
+```
+export ANDROID_SDK_ROOT=<PATH-TO-ANDROID-SDK>
+export ANDROID_NDK_ROOT=<PATH-TO-ANDROID-NDK>
+```
+
Now we're ready to build everything for Android:
```
./build.sh mono+libs -os Android -arch x64
./dotnet.sh build /t:Test src/libraries/System.Numerics.Vectors/tests /p:TargetOS=Android /p:TargetArchitecture=x64
```
+### Running the functional tests
+
+There are [functional tests](https://github.com/dotnet/runtime/tree/main/src/tests/FunctionalTests/) which aim to test some specific features/configurations/modes on a target mobile platform.
+
+A functional test can be run the same way as any library test suite, e.g.:
+```
+./dotnet.sh build /t:Test -c Release /p:TargetOS=Android /p:TargetArchitecture=x64 src/tests/FunctionalTests/Android/Device_Emulator/PInvoke/Android.Device_Emulator.PInvoke.Test.csproj
+```
+
+Currently functional tests are expected to return `42` as a success code so please be careful when adding a new one.
+
+### Testing various configurations
+
+It's possible to test various configurations by setting a combination of additional MSBuild properties such as `RunAOTCompilation`,`MonoForceInterpreter`, and some more.
+
+1. AOT
+
+To build for AOT only mode, add `/p:RunAOTCompilation=true /p:MonoForceInterpreter=false` to a build command.
+
+2. AOT-LLVM
+
+To build for AOT-LLVM mode, add `/p:RunAOTCompilation=true /p:MonoForceInterpreter=false /p:MonoEnableLLVM=true` to a build command.
+
+3. Interpreter
+
+To build for Interpreter mode, add `/p:RunAOTCompilation=false /p:MonoForceInterpreter=true` to a build command.
+
### Test App Design
-Android app is basically a [Java Instrumentation](https://github.com/dotnet/runtime/blob/main/src/mono/msbuild/AndroidAppBuilder/Templates/MonoRunner.java) and a simple Activity that inits the Mono Runtime via JNI. This Mono Runtime starts a simple xunit test
+Android app is basically a [Java Instrumentation](https://github.com/dotnet/runtime/blob/main/src/tasks/AndroidAppBuilder/Templates/MonoRunner.java) and a simple Activity that inits the Mono Runtime via JNI. This Mono Runtime starts a simple xunit test
runner called XHarness.TestRunner (see https://github.com/dotnet/xharness) which runs tests for all `*.Tests.dll` libs in the bundle. There is also XHarness.CLI tool with ADB embedded to deploy `*.apk` to a target (device or emulator) and obtain logs once tests are completed.
### Obtaining the logs
```
adb logcat -s "DOTNET"
```
-Or simply open `logcat` window in Android Studio or Visual Stuido.
+Or simply open `logcat` window in Android Studio or Visual Studio.
### AVD Manager
If Android Studio is installed, [AVD Manager](https://developer.android.com/studio/run/managing-avds) can be used from the IDE to create and start Android virtual devices. Otherwise, the Android SDK provides the [`avdmanager` command line tool](https://developer.android.com/studio/command-line/avdmanager).
-# Testing Libraries on iOS and tvOS
+# Testing Libraries on iOS, tvOS, and MacCatalyst
-In order to build libraries and tests for iOS or tvOS you need recent version of XCode installed (e.g. 11.3 or higher).
+## Prerequisites
-Build Libraries for iOS Simulator:
+- XCode 11.3 or higher
+- a certificate and provisioning profile if using a device
+- a simulator with a proper device type and OS version.
+Go `XCode > Window > Devices and Simulators` to revise the list of the available simulators and then `"+" button on bottom left > OS Version dropdown selection > Download more simulator runtimes` in case you need to download more simulators.
+
+## Building Libs and Tests
+
+You can build and run the library tests:
+- on a simulator;
+- on a device.
+
+Run the following command in a terminal:
+```
+./build.sh mono+libs -os <TARGET_OS> -arch <TARGET_ARCHITECTURE>
+```
+where `<TARGET_OS>` is one of the following:
+- iOSSimulator
+- tvOSSimulator
+- MacCatalyst
+- iOS
+- tvOS
+
+and `<TARGET_ARCHITECTURE>` is one of the following:
+- x64
+- arm64 (for device)
+
+e.g., to build for an iOS simulator, run:
```
./build.sh mono+libs -os iOSSimulator -arch x64
```
+
Run tests one by one for each test suite on a simulator:
```
./build.sh libs.tests -os iOSSimulator -arch x64 -test
```
+
+### Building for a device
+
In order to run the tests on a device:
-- Set the os to `iOS` instead of `iOSSimulator`
-- Specify `DevTeamProvisioning` (see [developer.apple.com/account/#/membership](https://developer.apple.com/account/#/membership), scroll down to `Team ID`):
+- Set the `-os` parameter to a device-related value (see above)
+- Specify `DevTeamProvisioning` (see [developer.apple.com/account/#/membership](https://developer.apple.com/account/#/membership), scroll down to `Team ID`).
+
+For example:
```
./build.sh libs.tests -os iOS -arch x64 -test /p:DevTeamProvisioning=H1A2B3C4D5
```
-[AppleAppBuilder](https://github.com/dotnet/runtime/blob/main/src/mono/msbuild/AppleAppBuilder/AppleAppBuilder.cs) generates temp Xcode projects you can manually open and resolve provisioning issues there using native UI and deploy to your devices.
+[AppleAppBuilder](https://github.com/dotnet/runtime/blob/main/src/tasks/AppleAppBuilder/AppleAppBuilder.cs) generates temp Xcode projects you can manually open and resolve provisioning issues there using native UI and deploy to your devices.
### Running individual test suites
+
- The following shows how to run tests for a specific library:
```
./dotnet.sh build src/libraries/System.Numerics.Vectors/tests /t:Test /p:TargetOS=iOS /p:TargetArchitecture=x64
```
+Also you can run the built test app through Xcode by opening the corresponding `.xcodeproj` and setting up the right scheme, app, and even signing if using a local device.
+
+There's also an option to run a `.app` through `xcrun`, which is simulator specific. Consider the following shell script:
+
+```
+ xcrun simctl shutdown <IOSSIMULATOR_NAME>
+ xcrun simctl boot <IOSSIMULATOR_NAME>
+ open -a Simulator
+ xcrun simctl install <IOSSIMULATOR_NAME> <APP_BUNDLE_PATH>
+ xcrun simctl launch --console booted <IOS_APP_NAME>
+```
+
+where
+`<IOSSIMULATOR_NAME>` is a name of the simulator to start, e.g. `"iPhone 11"`,
+`<APP_BUNDLE_PATH>` is a path to the iOS test app bundle,
+`<IOS_APP_NAME>` is a name of the iOS test app
+
+### Running the functional tests
+
+There are [functional tests](https://github.com/dotnet/runtime/tree/main/src/tests/FunctionalTests/) which aim to test some specific features/configurations/modes on a target mobile platform.
+
+A functional test can be run the same way as any library test suite, e.g.:
+```
+./dotnet.sh build /t:Test -c Release /p:TargetOS=iOSSimulator /p:TargetArchitecture=x64 src/tests/FunctionalTests/iOS/Simulator/PInvoke/iOS.Simulator.PInvoke.Test.csproj
+```
+
+Currently functional tests are expected to return `42` as a success code so please be careful when adding a new one.
+
+### Viewing logs
+- see the logs generated by the XHarness tool
+- use the `Console` app on macOS:
+`Command + Space`, type in `Console`, search the appropriate process (System.Buffers.Tests for example).
+
+### Testing various configurations
+
+It's possible to test various configurations by setting a combination of additional MSBuild properties such as `RunAOTCompilation`,`MonoEnableInterpreter`, and some more.
+
+1. Interpreter Only
+
+This configuration is necessary for hot reload scenarios.
+
+To enable the interpreter, add `/p:RunAOTCompilation=true /p:MonoEnableInterpreter=true` to a build command.
+
+2. AOT only
+
+To build for AOT only mode, add `/p:RunAOTCompilation=true /p:MonoEnableInterpreter=false` to a build command.
+
+3. AOT-LLVM
+
+To build for AOT-LLVM mode, add `/p:RunAOTCompilation=true /p:MonoEnableInterpreter=false /p:MonoEnableLLVM=true` to a build command.
+
### Test App Design
-iOS/tvOS `*.app` (or `*.ipa`) is basically a simple [ObjC app](https://github.com/dotnet/runtime/blob/main/src/mono/msbuild/AppleAppBuilder/Templates/main-console.m) that inits the Mono Runtime. This Mono Runtime starts a simple xunit test
+iOS/tvOS `*.app` (or `*.ipa`) is basically a simple [ObjC app](https://github.com/dotnet/runtime/blob/main/src/tasks/AppleAppBuilder/Templates/main-console.m) that inits the Mono Runtime. This Mono Runtime starts a simple xunit test
runner called XHarness.TestRunner (see https://github.com/dotnet/xharness) which runs tests for all `*.Tests.dll` libs in the bundle. There is also XHarness.CLI tool to deploy `*.app` and `*.ipa` to a target (device or simulator) and listens for logs via network sockets.
### Existing Limitations
-- Most of the test suites crash on devices due to #35674
- Simulator uses JIT mode only at the moment (to be extended with FullAOT and Interpreter)
- Interpreter is not enabled yet.
make run-tests-corefx-System.Runtime
```
### Mobile targets and WebAssembly
-Build and run library tests against Webassembly, Android or iOS. See instructions located in [Library testing document folder](../libraries/)
+Build and run library tests against WebAssembly, Android or iOS. See instructions located in [Library testing document folder](../libraries/)
+
+## Running the functional tests
+
+There are the [functional tests](https://github.com/dotnet/runtime/tree/main/src/tests/FunctionalTests/) which aim to test some specific features/configurations/modes on Android, iOS-like platforms (iOS/tvOS + simulators, MacCatalyst), and WebAssembly.
+
+A functional test can be run the same way as any library test suite, e.g.:
+```
+./dotnet.sh build /t:Test -c Release /p:TargetOS=Android /p:TargetArchitecture=x64 src/tests/FunctionalTests/Android/Device_Emulator/PInvoke/Android.Device_Emulator.PInvoke.Test.csproj
+```
+
+Currently the functional tests are expected to return `42` as a success code so please be careful when adding a new one.
+
+For more details, see instructions located in [Library testing document folder](../libraries/).
# Running the Mono samples
There are a few convenient samples located in `$(REPO_ROOT)/src/mono/sample`, which could help you test your program easily with different flavors of Mono or do a sanity check on the build. The samples are set up to work with a specific configuration; please refer to the relevant Makefile for specifics. If you would like to work with a different configuration, you can edit the Makefile.
projects / platform / upstream / dotnet / runtime.git / commitdiff