Add new Android build and install scripts
authorPyry Haulos <phaulos@google.com>
Mon, 27 Mar 2017 18:21:37 +0000 (11:21 -0700)
committerPyry Haulos <phaulos@google.com>
Mon, 8 May 2017 20:00:36 +0000 (13:00 -0700)
This change adds new Android build and install scripts under
scripts/android. Key improvements over old ones are:

 * Build no longer relies on ant or 'android project' tools.

 * Native code build leverages scripts/build code which should fix
   incremental builds and improve compatibility.

 * Build script error reporting should be much better.

 * Final APK is now built incrementally which should enable much faster
   incremental builds once asset copy targets are fixed in main build.

This work required some changes to common code:

 * Android cross-compile toolchain is set up by including
   targets/android/ndk-r11.cmake before project() in the main
   CMakeLists.txt instead of using -DCMAKE_TOOLCHAIN_FILE. CMake native
   toolchain file support seems incredbly buggy and configuring
   toolchain in regular build files seems to be much more robust.

 * scripts/build/config.py now finds CMake automatically on OS X.

 * New HostInfo class has been added into scripts/build/config.py.

Components: AOSP, Framework

Change-Id: I4b5b78c0d4d3aff248887ba5ced0c91081e24e6b

CMakeLists.txt
external/openglcts/README.md
external/vulkancts/README.md
scripts/android/build_apk.py [new file with mode: 0644]
scripts/android/install_apk.py [new file with mode: 0644]
scripts/build/build.py
scripts/build/common.py
scripts/build/config.py
targets/android/ndk-r11.cmake [new file with mode: 0644]

index 21d5d51..5b46f8f 100644 (file)
@@ -5,6 +5,11 @@ cmake_minimum_required(VERSION 2.8)
 # dEQP Target.
 set(DEQP_TARGET "default" CACHE STRING "dEQP Target (default, android...)")
 
+if (DEFINED DEQP_TARGET_TOOLCHAIN)
+       # \note Toolchain must be included before project() command
+       include(targets/${DEQP_TARGET}/${DEQP_TARGET_TOOLCHAIN}.cmake)
+endif ()
+
 project(dEQP-Core-${DEQP_TARGET})
 
 include(framework/delibs/cmake/Defs.cmake NO_POLICY_SCOPE)
index 67f1b02..27d5b87 100644 (file)
@@ -321,28 +321,28 @@ are needed in order to build an Android binary:
 
 An Android binary (for ES 3.2) can be built using command:
 
-       python external/openglcts/scripts/build_android.py
+       python scripts/android/build_apk.py --target=openglcts
 
 If Khronos Confidential CTS is present then the script will set `GLCTS_GTF_TARGET`
 to `gles32` by default.
 It is possible to specify a different `GLCTS_GTF_TARGET` target by invoking the script
-with the `--glcts-gtf-target` option, e.g.:
+with the `--kc-cts-target` option, e.g.:
 
-       python external/openglcts/scripts/build_android.py --glcts-gtf-target=gles31
+       python scripts/android/build_apk.py --target=openglcts --kc-cts-target=gles31
 
-Available values for `--glcts-gtf-target` are `gles32`, `gles31`, `gles3`, `gles2` and `gl`.
+Available values for `--kc-cts-target` are `gles32`, `gles31`, `gles3`, `gles2` and `gl`.
 
 The package can be installed by either running:
 
-       python android/scripts/install.py
+       python scripts/android/install_apk.py --target=openglcts
 
 By default the CTS package will contain libdeqp.so built for `armeabi-v7a`, `arm64-v8a`,
-`x86`, and `x86_64` ABIs, but that can be changed in `android/scripts/common.py` script.
+`x86`, and `x86_64` ABIs, but that can be changed with `--abis` command line option.
 
 To pick which ABI to use at install time, following commands must be used
 instead:
 
-       adb install --abi <ABI name> android/openglcts/bin/dEQP-debug.apk /data/local/tmp/dEQP-debug.apk
+       adb install --abi <ABI name> <build root>/Khronos-CTS.apk /data/local/tmp/Khronos-CTS.apk
 
 The script assumes some default install locations, which should be changed based
 on your environment. It is a good idea to check at least variables
index 88df950..6b19161 100644 (file)
@@ -29,11 +29,12 @@ Requirements
  * Android NDK r11
  * Android SDK with: SDK Tools, SDK Platform-tools, SDK Build-tools, and API 22
  * Java Development Kit (JDK)
- * Apache Ant
- * Windows: either NMake or JOM in PATH
+ * Windows: either NMake or Ninja in PATH
 
-See `android/scripts/common.py` for a list locations where the build system
-expects to find these.
+If you have downloaded Android SDK tools, you can install necessary components
+by running:
+
+       tools/android update sdk --no-ui --all --filter tools,platform-tools,build-tools-25.0.2,android-22
 
 
 Building CTS
@@ -78,21 +79,21 @@ Release build can be done by using -DCMAKE_BUILD_TYPE=Release
 
 ### Android
 
-Following command will build CTS into android/package/bin/dEQP-debug.apk.
+Following command will build dEQP.apk:
 
-       python android/scripts/build.py
+       python scripts/android/build_apk.py
 
 The package can be installed by either running:
 
-       python android/scripts/install.py
+       python scripts/android/install_apk.py
 
 By default the CTS package will contain libdeqp.so built for armeabi-v7a, arm64-v8a,
-and x86 ABIs, but that can be changed in android/scripts/common.py script.
+x86, and x86_64 ABIs, but that can be changed using --abis command line option.
 
 To pick which ABI to use at install time, following commands must be used
 instead:
 
-       adb install --abi <ABI name> android/package/bin/dEQP-debug.apk /data/local/tmp/dEQP-debug.apk
+       adb install --abi <ABI name> <build-root>/package/dEQP.apk /data/local/tmp/dEQP-debug.apk
 
 
 Building Mustpass
diff --git a/scripts/android/build_apk.py b/scripts/android/build_apk.py
new file mode 100644 (file)
index 0000000..796dbee
--- /dev/null
@@ -0,0 +1,936 @@
+# -*- coding: utf-8 -*-
+
+#-------------------------------------------------------------------------
+# drawElements Quality Program utilities
+# --------------------------------------
+#
+# Copyright 2017 The Android Open Source Project
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+#-------------------------------------------------------------------------
+
+# \todo [2017-04-10 pyry]
+# * Use smarter asset copy in main build
+#   * cmake -E copy_directory doesn't copy timestamps which will cause
+#     assets to be always re-packaged
+# * Consider adding an option for downloading SDK & NDK
+
+import os
+import re
+import sys
+import string
+import shutil
+import argparse
+import tempfile
+import xml.etree.ElementTree
+
+# Import from <root>/scripts
+sys.path.append(os.path.join(os.path.dirname(__file__), ".."))
+
+from build.common import *
+from build.config import *
+from build.build import *
+
+class SDKEnv:
+       def __init__(self, path):
+               self.path                               = path
+               self.buildToolsVersion  = SDKEnv.selectBuildToolsVersion(self.path)
+
+       @staticmethod
+       def getBuildToolsVersions (path):
+               buildToolsPath  = os.path.join(path, "build-tools")
+               versions                = []
+
+               if os.path.exists(buildToolsPath):
+                       for item in os.listdir(buildToolsPath):
+                               m = re.match(r'^([0-9]+)\.([0-9]+)\.([0-9]+)$', item)
+                               if m != None:
+                                       versions.append((int(m.group(1)), int(m.group(2)), int(m.group(3))))
+
+               return versions
+
+       @staticmethod
+       def selectBuildToolsVersion (path):
+               preferred       = [(25, 0, 2)]
+               versions        = SDKEnv.getBuildToolsVersions(path)
+
+               if len(versions) == 0:
+                       return (0,0,0)
+
+               for candidate in preferred:
+                       if candidate in versions:
+                               return candidate
+
+               # Pick newest
+               versions.sort()
+               return versions[-1]
+
+       def getPlatformLibrary (self, apiVersion):
+               return os.path.join(self.path, "platforms", "android-%d" % apiVersion, "android.jar")
+
+       def getBuildToolsPath (self):
+               return os.path.join(self.path, "build-tools", "%d.%d.%d" % self.buildToolsVersion)
+
+class NDKEnv:
+       def __init__(self, path):
+               self.path               = path
+               self.version    = NDKEnv.detectVersion(self.path)
+               self.hostOsName = NDKEnv.detectHostOsName(self.path)
+
+       @staticmethod
+       def getKnownAbis ():
+               return ["armeabi-v7a", "arm64-v8a", "x86", "x86_64"]
+
+       @staticmethod
+       def getAbiPrebuiltsName (abiName):
+               prebuilts = {
+                       "armeabi-v7a":  'android-arm',
+                       "arm64-v8a":    'android-arm64',
+                       "x86":                  'android-x86',
+                       "x86_64":               'android-x86_64',
+               }
+
+               if not abiName in prebuilts:
+                       raise Exception("Unknown ABI: " + abiName)
+
+               return prebuilts[abiName]
+
+       @staticmethod
+       def detectVersion (path):
+               propFilePath = os.path.join(path, "source.properties")
+               try:
+                       with open(propFilePath) as propFile:
+                               for line in propFile:
+                                       keyValue = map(lambda x: string.strip(x), line.split("="))
+                                       if keyValue[0] == "Pkg.Revision":
+                                               versionParts = keyValue[1].split(".")
+                                               return tuple(map(int, versionParts[0:2]))
+               except Exception as e:
+                       raise Exception("Failed to read source prop file '%s': %s" % (propFilePath, str(e)))
+               except:
+                       raise Exception("Failed to read source prop file '%s': unkown error")
+
+               raise Exception("Failed to detect NDK version (does %s/source.properties have Pkg.Revision?)" % path)
+
+       @staticmethod
+       def isHostOsSupported (hostOsName):
+               os                      = HostInfo.getOs()
+               bits            = HostInfo.getArchBits()
+               hostOsParts     = hostOsName.split('-')
+
+               if len(hostOsParts) > 1:
+                       assert(len(hostOsParts) == 2)
+                       assert(hostOsParts[1] == "x86_64")
+
+                       if bits != 64:
+                               return False
+
+               if os == HostInfo.OS_WINDOWS:
+                       return hostOsParts[0] == 'windows'
+               elif os == HostInfo.OS_LINUX:
+                       return hostOsParts[0] == 'linux'
+               elif os == HostInfo.OS_OSX:
+                       return hostOsParts[0] == 'darwin'
+               else:
+                       raise Exception("Unhandled HostInfo.getOs() '%d'" % os)
+
+       @staticmethod
+       def detectHostOsName (path):
+               hostOsNames = [
+                       "windows",
+                       "windows-x86_64",
+                       "darwin-x86",
+                       "darwin-x86_64",
+                       "linux-x86",
+                       "linux-x86_64"
+               ]
+
+               for name in hostOsNames:
+                       if os.path.exists(os.path.join(path, "prebuilt", name)):
+                               return name
+
+               raise Exception("Failed to determine NDK host OS")
+
+class Environment:
+       def __init__(self, sdk, ndk):
+               self.sdk                = sdk
+               self.ndk                = ndk
+
+class Configuration:
+       def __init__(self, env, buildPath, abis, nativeBuildType, gtfTarget, verbose):
+               self.env                                = env
+               self.sourcePath                 = DEQP_DIR
+               self.buildPath                  = buildPath
+               self.abis                               = abis
+               self.nativeApi                  = 21
+               self.javaApi                    = 22
+               self.nativeBuildType    = nativeBuildType
+               self.gtfTarget                  = gtfTarget
+               self.verbose                    = verbose
+               self.cmakeGenerator             = selectFirstAvailableGenerator([NINJA_GENERATOR, MAKEFILE_GENERATOR, NMAKE_GENERATOR])
+
+       def check (self):
+               if self.cmakeGenerator == None:
+                       raise Exception("Failed to find build tools for CMake")
+
+               if not os.path.exists(self.env.ndk.path):
+                       raise Exception("Android NDK not found at %s" % self.env.ndk.path)
+
+               if not NDKEnv.isHostOsSupported(self.env.ndk.hostOsName):
+                       raise Exception("NDK '%s' is not supported on this machine" % self.env.ndk.hostOsName)
+
+               supportedNDKVersion = 11
+               if self.env.ndk.version[0] != supportedNDKVersion:
+                       raise Exception("Android NDK version %d is not supported; build requires NDK version %d" % (self.env.ndk.version[0], supportedNDKVersion))
+
+               if self.env.sdk.buildToolsVersion == (0,0,0):
+                       raise Exception("No build tools directory found at %s" % os.path.join(self.env.sdk.path, "build-tools"))
+
+               androidBuildTools = ["aapt", "zipalign", "dx"]
+               for tool in androidBuildTools:
+                       if which(tool, [self.env.sdk.getBuildToolsPath()]) == None:
+                               raise Exception("Missing Android build tool: %s" % toolPath)
+
+               requiredToolsInPath = ["javac", "jar", "jarsigner", "keytool"]
+               for tool in requiredToolsInPath:
+                       if which(tool) == None:
+                               raise Exception("%s not in PATH" % tool)
+
+def log (config, msg):
+       if config.verbose:
+               print msg
+
+def executeAndLog (config, args):
+       if config.verbose:
+               print " ".join(args)
+       execute(args)
+
+# Path components
+
+class ResolvablePathComponent:
+       def __init__ (self):
+               pass
+
+class SourceRoot (ResolvablePathComponent):
+       def resolve (self, config):
+               return config.sourcePath
+
+class BuildRoot (ResolvablePathComponent):
+       def resolve (self, config):
+               return config.buildPath
+
+class NativeBuildPath (ResolvablePathComponent):
+       def __init__ (self, abiName):
+               self.abiName = abiName
+
+       def resolve (self, config):
+               return getNativeBuildPath(config, self.abiName)
+
+class GeneratedResSourcePath (ResolvablePathComponent):
+       def __init__ (self, package):
+               self.package = package
+
+       def resolve (self, config):
+               packageComps    = self.package.getPackageName(config).split('.')
+               packageDir              = os.path.join(*packageComps)
+
+               return os.path.join(config.buildPath, self.package.getAppDirName(), "src", packageDir, "R.java")
+
+def resolvePath (config, path):
+       resolvedComps = []
+
+       for component in path:
+               if isinstance(component, ResolvablePathComponent):
+                       resolvedComps.append(component.resolve(config))
+               else:
+                       resolvedComps.append(str(component))
+
+       return os.path.join(*resolvedComps)
+
+def resolvePaths (config, paths):
+       return list(map(lambda p: resolvePath(config, p), paths))
+
+class BuildStep:
+       def __init__ (self):
+               pass
+
+       def getInputs (self):
+               return []
+
+       def getOutputs (self):
+               return []
+
+       @staticmethod
+       def expandPathsToFiles (paths):
+               """
+               Expand mixed list of file and directory paths into a flattened list
+               of files. Any non-existent input paths are preserved as is.
+               """
+
+               def getFiles (dirPath):
+                       for root, dirs, files in os.walk(dirPath):
+                               for file in files:
+                                       yield os.path.join(root, file)
+
+               files = []
+               for path in paths:
+                       if os.path.isdir(path):
+                               files += list(getFiles(path))
+                       else:
+                               files.append(path)
+
+               return files
+
+       def isUpToDate (self, config):
+               inputs                          = resolvePaths(config, self.getInputs())
+               outputs                         = resolvePaths(config, self.getOutputs())
+
+               assert len(inputs) > 0 and len(outputs) > 0
+
+               expandedInputs          = BuildStep.expandPathsToFiles(inputs)
+               expandedOutputs         = BuildStep.expandPathsToFiles(outputs)
+
+               existingInputs          = filter(os.path.exists, expandedInputs)
+               existingOutputs         = filter(os.path.exists, expandedOutputs)
+
+               if len(existingInputs) != len(expandedInputs):
+                       for file in expandedInputs:
+                               if file not in existingInputs:
+                                       print "ERROR: Missing input file: %s" % file
+                       die("Missing input files")
+
+               if len(existingOutputs) != len(expandedOutputs):
+                       return False # One or more output files are missing
+
+               lastInputChange         = max(map(os.path.getmtime, existingInputs))
+               firstOutputChange       = min(map(os.path.getmtime, existingOutputs))
+
+               return lastInputChange <= firstOutputChange
+
+       def update (config):
+               die("BuildStep.update() not implemented")
+
+def getNativeBuildPath (config, abiName):
+       return os.path.join(config.buildPath, "%s-%s-%d" % (abiName, config.nativeBuildType, config.nativeApi))
+
+def buildNativeLibrary (config, abiName):
+       def makeNDKVersionString (version):
+               minorVersionString = (chr(ord('a') + version[1]) if version[1] > 0 else "")
+               return "r%d%s" % (version[0], minorVersionString)
+
+       def getBuildArgs (config, abiName):
+               toolchain = 'ndk-%s' % makeNDKVersionString((config.env.ndk.version[0], 0))
+               return ['-DDEQP_TARGET=android',
+                               '-DDEQP_TARGET_TOOLCHAIN=%s' % toolchain,
+                               '-DCMAKE_C_FLAGS=-Werror',
+                               '-DCMAKE_CXX_FLAGS=-Werror',
+                               '-DANDROID_NDK_HOST_OS=%s' % config.env.ndk.hostOsName,
+                               '-DANDROID_NDK_PATH=%s' % config.env.ndk.path,
+                               '-DANDROID_ABI=%s' % abiName,
+                               '-DDE_ANDROID_API=%s' % config.nativeApi,
+                               '-DGLCTS_GTF_TARGET=%s' % config.gtfTarget]
+
+       nativeBuildPath = getNativeBuildPath(config, abiName)
+       buildConfig             = BuildConfig(nativeBuildPath, config.nativeBuildType, getBuildArgs(config, abiName))
+
+       build(buildConfig, config.cmakeGenerator, ["deqp"])
+
+def executeSteps (config, steps):
+       for step in steps:
+               if not step.isUpToDate(config):
+                       step.update(config)
+
+def parsePackageName (manifestPath):
+       tree = xml.etree.ElementTree.parse(manifestPath)
+
+       if not 'package' in tree.getroot().attrib:
+               raise Exception("'package' attribute missing from root element in %s" % manifestPath)
+
+       return tree.getroot().attrib['package']
+
+class PackageDescription:
+       def __init__ (self, appDirName, appName, hasResources = True):
+               self.appDirName         = appDirName
+               self.appName            = appName
+               self.hasResources       = hasResources
+
+       def getAppName (self):
+               return self.appName
+
+       def getAppDirName (self):
+               return self.appDirName
+
+       def getPackageName (self, config):
+               manifestPath    = resolvePath(config, self.getManifestPath())
+
+               return parsePackageName(manifestPath)
+
+       def getManifestPath (self):
+               return [SourceRoot(), "android", self.appDirName, "AndroidManifest.xml"]
+
+       def getResPath (self):
+               return [SourceRoot(), "android", self.appDirName, "res"]
+
+       def getSourcePaths (self):
+               return [
+                               [SourceRoot(), "android", self.appDirName, "src"]
+                       ]
+
+       def getAssetsPath (self):
+               return [BuildRoot(), self.appDirName, "assets"]
+
+       def getClassesJarPath (self):
+               return [BuildRoot(), self.appDirName, "bin", "classes.jar"]
+
+       def getClassesDexPath (self):
+               return [BuildRoot(), self.appDirName, "bin", "classes.dex"]
+
+       def getAPKPath (self):
+               return [BuildRoot(), self.appDirName, "bin", self.appName + ".apk"]
+
+# Build step implementations
+
+class BuildNativeLibrary (BuildStep):
+       def __init__ (self, abi):
+               self.abi = abi
+
+       def isUpToDate (self, config):
+               return False
+
+       def update (self, config):
+               log(config, "BuildNativeLibrary: %s" % self.abi)
+               buildNativeLibrary(config, self.abi)
+
+class GenResourcesSrc (BuildStep):
+       def __init__ (self, package):
+               self.package = package
+
+       def getInputs (self):
+               return [self.package.getResPath(), self.package.getManifestPath()]
+
+       def getOutputs (self):
+               return [[GeneratedResSourcePath(self.package)]]
+
+       def update (self, config):
+               aaptPath        = which("aapt", [config.env.sdk.getBuildToolsPath()])
+               dstDir          = os.path.dirname(resolvePath(config, [GeneratedResSourcePath(self.package)]))
+
+               if not os.path.exists(dstDir):
+                       os.makedirs(dstDir)
+
+               executeAndLog(config, [
+                               aaptPath,
+                               "package",
+                               "-f",
+                               "-m",
+                               "-S", resolvePath(config, self.package.getResPath()),
+                               "-M", resolvePath(config, self.package.getManifestPath()),
+                               "-J", resolvePath(config, [BuildRoot(), self.package.getAppDirName(), "src"]),
+                               "-I", config.env.sdk.getPlatformLibrary(config.javaApi)
+                       ])
+
+# Builds classes.jar from *.java files
+class BuildJavaSource (BuildStep):
+       def __init__ (self, package, libraries = []):
+               self.package    = package
+               self.libraries  = libraries
+
+       def getSourcePaths (self):
+               srcPaths = self.package.getSourcePaths()
+
+               if self.package.hasResources:
+                       srcPaths.append([BuildRoot(), self.package.getAppDirName(), "src"]) # Generated sources
+
+               return srcPaths
+
+       def getInputs (self):
+               inputs = self.getSourcePaths()
+
+               for lib in self.libraries:
+                       inputs.append(lib.getClassesJarPath())
+
+               return inputs
+
+       def getOutputs (self):
+               return [self.package.getClassesJarPath()]
+
+       def update (self, config):
+               srcPaths        = resolvePaths(config, self.getSourcePaths())
+               srcFiles        = BuildStep.expandPathsToFiles(srcPaths)
+               jarPath         = resolvePath(config, self.package.getClassesJarPath())
+               objPath         = resolvePath(config, [BuildRoot(), self.package.getAppDirName(), "obj"])
+               classPaths      = [objPath] + [resolvePath(config, lib.getClassesJarPath()) for lib in self.libraries]
+               pathSep         = ";" if HostInfo.getOs() == HostInfo.OS_WINDOWS else ":"
+
+               if os.path.exists(objPath):
+                       shutil.rmtree(objPath)
+
+               os.makedirs(objPath)
+
+               for srcFile in srcFiles:
+                       executeAndLog(config, [
+                                       "javac",
+                                       "-source", "1.7",
+                                       "-target", "1.7",
+                                       "-d", objPath,
+                                       "-bootclasspath", config.env.sdk.getPlatformLibrary(config.javaApi),
+                                       "-classpath", pathSep.join(classPaths),
+                                       "-sourcepath", pathSep.join(srcPaths),
+                                       srcFile
+                               ])
+
+               if not os.path.exists(os.path.dirname(jarPath)):
+                       os.makedirs(os.path.dirname(jarPath))
+
+               try:
+                       pushWorkingDir(objPath)
+                       executeAndLog(config, [
+                                       "jar",
+                                       "cf",
+                                       jarPath,
+                                       "."
+                               ])
+               finally:
+                       popWorkingDir()
+
+class BuildDex (BuildStep):
+       def __init__ (self, package, libraries):
+               self.package    = package
+               self.libraries  = libraries
+
+       def getInputs (self):
+               return [self.package.getClassesJarPath()] + [lib.getClassesJarPath() for lib in self.libraries]
+
+       def getOutputs (self):
+               return [self.package.getClassesDexPath()]
+
+       def update (self, config):
+               dxPath          = which("dx", [config.env.sdk.getBuildToolsPath()])
+               srcPaths        = resolvePaths(config, self.getInputs())
+               dexPath         = resolvePath(config, self.package.getClassesDexPath())
+               jarPaths        = [resolvePath(config, self.package.getClassesJarPath())]
+
+               for lib in self.libraries:
+                       jarPaths.append(resolvePath(config, lib.getClassesJarPath()))
+
+               executeAndLog(config, [
+                               dxPath,
+                               "--dex",
+                               "--output", dexPath
+                       ] + jarPaths)
+
+class CreateKeystore (BuildStep):
+       def __init__ (self):
+               self.keystorePath       = [BuildRoot(), "debug.keystore"]
+
+       def getOutputs (self):
+               return [self.keystorePath]
+
+       def isUpToDate (self, config):
+               return os.path.exists(resolvePath(config, self.keystorePath))
+
+       def update (self, config):
+               executeAndLog(config, [
+                               "keytool",
+                               "-genkey",
+                               "-keystore", resolvePath(config, self.keystorePath),
+                               "-storepass", "android",
+                               "-alias", "androiddebugkey",
+                               "-keypass", "android",
+                               "-keyalg", "RSA",
+                               "-keysize", "2048",
+                               "-validity", "10000",
+                               "-dname", "CN=, OU=, O=, L=, S=, C=",
+                       ])
+
+# Builds APK without code
+class BuildBaseAPK (BuildStep):
+       def __init__ (self, package, libraries = []):
+               self.package    = package
+               self.libraries  = libraries
+               self.dstPath    = [BuildRoot(), self.package.getAppDirName(), "tmp", "base.apk"]
+
+       def getResPaths (self):
+               paths = []
+               for pkg in [self.package] + self.libraries:
+                       if pkg.hasResources:
+                               paths.append(pkg.getResPath())
+               return paths
+
+       def getInputs (self):
+               return [self.package.getManifestPath()] + self.getResPaths()
+
+       def getOutputs (self):
+               return [self.dstPath]
+
+       def update (self, config):
+               aaptPath        = which("aapt", [config.env.sdk.getBuildToolsPath()])
+               dstPath         = resolvePath(config, self.dstPath)
+
+               if not os.path.exists(os.path.dirname(dstPath)):
+                       os.makedirs(os.path.dirname(dstPath))
+
+               args = [
+                       aaptPath,
+                       "package",
+                       "-f",
+                       "-M", resolvePath(config, self.package.getManifestPath()),
+                       "-I", config.env.sdk.getPlatformLibrary(config.javaApi),
+                       "-F", dstPath,
+               ]
+
+               for resPath in self.getResPaths():
+                       args += ["-S", resolvePath(config, resPath)]
+
+               if config.verbose:
+                       args.append("-v")
+
+               executeAndLog(config, args)
+
+def addFilesToAPK (config, apkPath, baseDir, relFilePaths):
+       aaptPath                = which("aapt", [config.env.sdk.getBuildToolsPath()])
+       maxBatchSize    = 25
+
+       pushWorkingDir(baseDir)
+       try:
+               workQueue = list(relFilePaths)
+
+               while len(workQueue) > 0:
+                       batchSize       = min(len(workQueue), maxBatchSize)
+                       items           = workQueue[0:batchSize]
+
+                       executeAndLog(config, [
+                                       aaptPath,
+                                       "add",
+                                       "-f", apkPath,
+                               ] + items)
+
+                       del workQueue[0:batchSize]
+       finally:
+               popWorkingDir()
+
+def addFileToAPK (config, apkPath, baseDir, relFilePath):
+       addFilesToAPK(config, apkPath, baseDir, [relFilePath])
+
+class AddJavaToAPK (BuildStep):
+       def __init__ (self, package):
+               self.package    = package
+               self.srcPath    = BuildBaseAPK(self.package).getOutputs()[0]
+               self.dstPath    = [BuildRoot(), self.package.getAppDirName(), "tmp", "with-java.apk"]
+
+       def getInputs (self):
+               return [
+                               self.srcPath,
+                               self.package.getClassesDexPath(),
+                       ]
+
+       def getOutputs (self):
+               return [self.dstPath]
+
+       def update (self, config):
+               srcPath         = resolvePath(config, self.srcPath)
+               dstPath         = resolvePath(config, self.getOutputs()[0])
+               dexPath         = resolvePath(config, self.package.getClassesDexPath())
+
+               shutil.copyfile(srcPath, dstPath)
+               addFileToAPK(config, dstPath, os.path.dirname(dexPath), os.path.basename(dexPath))
+
+class AddAssetsToAPK (BuildStep):
+       def __init__ (self, package, abi):
+               self.package    = package
+               self.buildPath  = [NativeBuildPath(abi)]
+               self.srcPath    = AddJavaToAPK(self.package).getOutputs()[0]
+               self.dstPath    = [BuildRoot(), self.package.getAppDirName(), "tmp", "with-assets.apk"]
+
+       def getInputs (self):
+               return [
+                               self.srcPath,
+                               self.buildPath + ["assets"]
+                       ]
+
+       def getOutputs (self):
+               return [self.dstPath]
+
+       @staticmethod
+       def getAssetFiles (buildPath):
+               allFiles = BuildStep.expandPathsToFiles([os.path.join(buildPath, "assets")])
+               return [os.path.relpath(p, buildPath) for p in allFiles]
+
+       def update (self, config):
+               srcPath         = resolvePath(config, self.srcPath)
+               dstPath         = resolvePath(config, self.getOutputs()[0])
+               buildPath       = resolvePath(config, self.buildPath)
+               assetFiles      = AddAssetsToAPK.getAssetFiles(buildPath)
+
+               shutil.copyfile(srcPath, dstPath)
+
+               addFilesToAPK(config, dstPath, buildPath, assetFiles)
+
+class AddNativeLibsToAPK (BuildStep):
+       def __init__ (self, package, abis):
+               self.package    = package
+               self.abis               = abis
+               self.srcPath    = AddAssetsToAPK(self.package, "").getOutputs()[0]
+               self.dstPath    = [BuildRoot(), self.package.getAppDirName(), "tmp", "with-native-libs.apk"]
+
+       def getInputs (self):
+               paths = [self.srcPath]
+               for abi in self.abis:
+                       paths.append([NativeBuildPath(abi), "libdeqp.so"])
+               return paths
+
+       def getOutputs (self):
+               return [self.dstPath]
+
+       def update (self, config):
+               srcPath         = resolvePath(config, self.srcPath)
+               dstPath         = resolvePath(config, self.getOutputs()[0])
+               pkgPath         = resolvePath(config, [BuildRoot(), self.package.getAppDirName()])
+               libFiles        = []
+
+               # Create right directory structure first
+               for abi in self.abis:
+                       libSrcPath      = resolvePath(config, [NativeBuildPath(abi), "libdeqp.so"])
+                       libRelPath      = os.path.join("lib", abi, "libdeqp.so")
+                       libAbsPath      = os.path.join(pkgPath, libRelPath)
+
+                       if not os.path.exists(os.path.dirname(libAbsPath)):
+                               os.makedirs(os.path.dirname(libAbsPath))
+
+                       shutil.copyfile(libSrcPath, libAbsPath)
+                       libFiles.append(libRelPath)
+
+               shutil.copyfile(srcPath, dstPath)
+               addFilesToAPK(config, dstPath, pkgPath, libFiles)
+
+class SignAPK (BuildStep):
+       def __init__ (self, package):
+               self.package            = package
+               self.srcPath            = AddNativeLibsToAPK(self.package, []).getOutputs()[0]
+               self.dstPath            = [BuildRoot(), self.package.getAppDirName(), "tmp", "signed.apk"]
+               self.keystorePath       = CreateKeystore().getOutputs()[0]
+
+       def getInputs (self):
+               return [self.srcPath, self.keystorePath]
+
+       def getOutputs (self):
+               return [self.dstPath]
+
+       def update (self, config):
+               srcPath         = resolvePath(config, self.srcPath)
+               dstPath         = resolvePath(config, self.dstPath)
+
+               executeAndLog(config, [
+                               "jarsigner",
+                               "-keystore", resolvePath(config, self.keystorePath),
+                               "-storepass", "android",
+                               "-keypass", "android",
+                               "-signedjar", dstPath,
+                               srcPath,
+                               "androiddebugkey"
+                       ])
+
+def getBuildRootRelativeAPKPath (package):
+       return os.path.join(package.getAppDirName(), package.getAppName() + ".apk")
+
+class FinalizeAPK (BuildStep):
+       def __init__ (self, package):
+               self.package            = package
+               self.srcPath            = SignAPK(self.package).getOutputs()[0]
+               self.dstPath            = [BuildRoot(), getBuildRootRelativeAPKPath(self.package)]
+               self.keystorePath       = CreateKeystore().getOutputs()[0]
+
+       def getInputs (self):
+               return [self.srcPath]
+
+       def getOutputs (self):
+               return [self.dstPath]
+
+       def update (self, config):
+               srcPath                 = resolvePath(config, self.srcPath)
+               dstPath                 = resolvePath(config, self.dstPath)
+               zipalignPath    = os.path.join(config.env.sdk.getBuildToolsPath(), "zipalign")
+
+               executeAndLog(config, [
+                               zipalignPath,
+                               "-f", "4",
+                               srcPath,
+                               dstPath
+                       ])
+
+def getBuildStepsForPackage (abis, package, libraries = []):
+       steps = []
+
+       assert len(abis) > 0
+
+       # Build native code first
+       for abi in abis:
+               steps += [BuildNativeLibrary(abi)]
+
+       # Build library packages
+       for library in libraries:
+               if library.hasResources:
+                       steps.append(GenResourcesSrc(library))
+               steps.append(BuildJavaSource(library))
+
+       # Build main package .java sources
+       if package.hasResources:
+               steps.append(GenResourcesSrc(package))
+       steps.append(BuildJavaSource(package, libraries))
+       steps.append(BuildDex(package, libraries))
+
+       # Build base APK
+       steps.append(BuildBaseAPK(package, libraries))
+       steps.append(AddJavaToAPK(package))
+
+       # Add assets from first ABI
+       steps.append(AddAssetsToAPK(package, abis[0]))
+
+       # Add native libs to APK
+       steps.append(AddNativeLibsToAPK(package, abis))
+
+       # Finalize APK
+       steps.append(CreateKeystore())
+       steps.append(SignAPK(package))
+       steps.append(FinalizeAPK(package))
+
+       return steps
+
+def getPackageAndLibrariesForTarget (target):
+       deqpPackage     = PackageDescription("package", "dEQP")
+       ctsPackage      = PackageDescription("openglcts", "Khronos-CTS", hasResources = False)
+
+       if target == 'deqp':
+               return (deqpPackage, [])
+       elif target == 'openglcts':
+               return (ctsPackage, [deqpPackage])
+       else:
+               raise Exception("Uknown target '%s'" % target)
+
+def findNDK ():
+       ndkBuildPath = which('ndk-build')
+       if ndkBuildPath != None:
+               return os.path.dirname(ndkBuildPath)
+       else:
+               return None
+
+def findSDK ():
+       sdkBuildPath = which('android')
+       if sdkBuildPath != None:
+               return os.path.dirname(os.path.dirname(sdkBuildPath))
+       else:
+               return None
+
+def getDefaultBuildRoot ():
+       return os.path.join(tempfile.gettempdir(), "deqp-android-build")
+
+def parseArgs ():
+       nativeBuildTypes        = ['Release', 'Debug', 'MinSizeRel', 'RelWithAsserts', 'RelWithDebInfo']
+       defaultNDKPath          = findNDK()
+       defaultSDKPath          = findSDK()
+       defaultBuildRoot        = getDefaultBuildRoot()
+
+       parser = argparse.ArgumentParser(os.path.basename(__file__),
+               formatter_class=argparse.ArgumentDefaultsHelpFormatter)
+       parser.add_argument('--native-build-type',
+               dest='nativeBuildType',
+               default="RelWithAsserts",
+               choices=nativeBuildTypes,
+               help="Native code build type")
+       parser.add_argument('--build-root',
+               dest='buildRoot',
+               default=defaultBuildRoot,
+               help="Root build directory")
+       parser.add_argument('--abis',
+               dest='abis',
+               default=",".join(NDKEnv.getKnownAbis()),
+               help="ABIs to build")
+       parser.add_argument('--sdk',
+               dest='sdkPath',
+               default=defaultSDKPath,
+               help="Android SDK path",
+               required=(True if defaultSDKPath == None else False))
+       parser.add_argument('--ndk',
+               dest='ndkPath',
+               default=defaultNDKPath,
+               help="Android NDK path",
+               required=(True if defaultNDKPath == None else False))
+       parser.add_argument('-v', '--verbose',
+               dest='verbose',
+               help="Verbose output",
+               default=False,
+               action='store_true')
+       parser.add_argument('--target',
+               dest='target',
+               help='Build target',
+               choices=['deqp', 'openglcts'],
+               default='deqp')
+       parser.add_argument('--kc-cts-target',
+               dest='gtfTarget',
+               default='gles32',
+               choices=['gles32', 'gles31', 'gles3', 'gles2', 'gl'],
+               help="KC-CTS (GTF) target API (only used in openglcts target)")
+
+       args = parser.parse_args()
+
+       def parseAbis (abisStr):
+               knownAbis       = set(NDKEnv.getKnownAbis())
+               abis            = []
+
+               for abi in abisStr.split(','):
+                       abi = abi.strip()
+                       if not abi in knownAbis:
+                               raise Exception("Unknown ABI: %s" % abi)
+                       abis.append(abi)
+
+               return abis
+
+       # Custom parsing & checks
+       try:
+               args.abis = parseAbis(args.abis)
+               if len(args.abis) == 0:
+                       raise Exception("--abis can't be empty")
+       except Exception as e:
+               print "ERROR: %s" % str(e)
+               parser.print_help()
+               sys.exit(-1)
+
+       return args
+
+if __name__ == "__main__":
+       args            = parseArgs()
+
+       ndk                     = NDKEnv(os.path.realpath(args.ndkPath))
+       sdk                     = SDKEnv(os.path.realpath(args.sdkPath))
+       buildPath       = os.path.realpath(args.buildRoot)
+       env                     = Environment(sdk, ndk)
+       config          = Configuration(env, buildPath, abis=args.abis, nativeBuildType=args.nativeBuildType, gtfTarget=args.gtfTarget, verbose=args.verbose)
+
+       try:
+               config.check()
+       except Exception as e:
+               print "ERROR: %s" % str(e)
+               print ""
+               print "Please check your configuration:"
+               print "  --sdk=%s" % args.sdkPath
+               print "  --ndk=%s" % args.ndkPath
+               sys.exit(-1)
+
+       pkg, libs       = getPackageAndLibrariesForTarget(args.target)
+       steps           = getBuildStepsForPackage(config.abis, pkg, libs)
+
+       executeSteps(config, steps)
+
+       print ""
+       print "Built %s" % os.path.join(buildPath, getBuildRootRelativeAPKPath(pkg))
diff --git a/scripts/android/install_apk.py b/scripts/android/install_apk.py
new file mode 100644 (file)
index 0000000..a3558ee
--- /dev/null
@@ -0,0 +1,248 @@
+# -*- coding: utf-8 -*-
+
+#-------------------------------------------------------------------------
+# drawElements Quality Program utilities
+# --------------------------------------
+#
+# Copyright 2017 The Android Open Source Project
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+#-------------------------------------------------------------------------
+
+import os
+import re
+import sys
+import argparse
+import threading
+import subprocess
+
+from build_apk import findSDK
+from build_apk import getDefaultBuildRoot
+from build_apk import getPackageAndLibrariesForTarget
+from build_apk import getBuildRootRelativeAPKPath
+from build_apk import parsePackageName
+
+# Import from <root>/scripts
+sys.path.append(os.path.join(os.path.dirname(__file__), ".."))
+
+from build.common import *
+
+class Device:
+       def __init__(self, serial, product, model, device):
+               self.serial             = serial
+               self.product    = product
+               self.model              = model
+               self.device             = device
+
+       def __str__ (self):
+               return "%s: {product: %s, model: %s, device: %s}" % (self.serial, self.product, self.model, self.device)
+
+def getDevices (adbPath):
+       proc = subprocess.Popen([adbPath, 'devices', '-l'], stdout=subprocess.PIPE)
+       (stdout, stderr) = proc.communicate()
+
+       if proc.returncode != 0:
+               raise Exception("adb devices -l failed, got %d" % proc.returncode)
+
+       ptrn = re.compile(r'^([a-zA-Z0-9\.:]+)\s+.*product:([^\s]+)\s+model:([^\s]+)\s+device:([^\s]+)')
+       devices = []
+       for line in stdout.splitlines()[1:]:
+               if len(line.strip()) == 0:
+                       continue
+
+               m = ptrn.match(line)
+               if m == None:
+                       print "WARNING: Failed to parse device info '%s'" % line
+                       continue
+
+               devices.append(Device(m.group(1), m.group(2), m.group(3), m.group(4)))
+
+       return devices
+
+def execWithPrintPrefix (args, linePrefix="", failOnNonZeroExit=True):
+
+       def readApplyPrefixAndPrint (source, prefix, sink):
+               while True:
+                       line = source.readline()
+                       if len(line) == 0: # EOF
+                               break;
+                       sink.write(prefix + line)
+
+       process = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+       stdoutJob = threading.Thread(target=readApplyPrefixAndPrint, args=(process.stdout, linePrefix, sys.stdout))
+       stderrJob = threading.Thread(target=readApplyPrefixAndPrint, args=(process.stderr, linePrefix, sys.stderr))
+       stdoutJob.start()
+       stderrJob.start()
+       retcode = process.wait()
+       if failOnNonZeroExit and retcode != 0:
+               raise Exception("Failed to execute '%s', got %d" % (str(args), retcode))
+
+def serialApply (f, argsList):
+       for args in argsList:
+               f(*args)
+
+def parallelApply (f, argsList):
+       class ErrorCode:
+               def __init__ (self):
+                       self.error = None;
+
+       def applyAndCaptureError (func, args, errorCode):
+               try:
+                       func(*args)
+               except:
+                       errorCode.error = sys.exc_info()
+
+       errorCode = ErrorCode()
+       jobs = []
+       for args in argsList:
+               job = threading.Thread(target=applyAndCaptureError, args=(f, args, errorCode))
+               job.start()
+               jobs.append(job)
+
+       for job in jobs:
+               job.join()
+
+       if errorCode.error:
+               raise errorCode.error[0], errorCode.error[1], errorCode.error[2]
+
+def uninstall (adbPath, packageName, extraArgs = [], printPrefix=""):
+       print printPrefix + "Removing existing %s...\n" % packageName,
+       execWithPrintPrefix([adbPath] + extraArgs + [
+                       'uninstall',
+                       packageName
+               ], printPrefix, failOnNonZeroExit=False)
+       print printPrefix + "Remove complete\n",
+
+def install (adbPath, apkPath, extraArgs = [], printPrefix=""):
+       print printPrefix + "Installing %s...\n" % apkPath,
+       execWithPrintPrefix([adbPath] + extraArgs + [
+                       'install',
+                       apkPath
+               ], printPrefix)
+       print printPrefix + "Install complete\n",
+
+def installToDevice (device, adbPath, packageName, apkPath, printPrefix=""):
+       if len(printPrefix) == 0:
+               print "Installing to %s (%s)...\n" % (device.serial, device.model),
+       else:
+               print printPrefix + "Installing to %s\n" % device.serial,
+
+       uninstall(adbPath, packageName, ['-s', device.serial], printPrefix)
+       install(adbPath, apkPath, ['-s', device.serial], printPrefix)
+
+def installToDevices (devices, doParallel, adbPath, packageName, apkPath):
+       padLen = max([len(device.model) for device in devices])+1
+       if doParallel:
+               parallelApply(installToDevice, [(device, adbPath, packageName, apkPath, ("(%s):%s" % (device.model, ' ' * (padLen - len(device.model))))) for device in devices]);
+       else:
+               serialApply(installToDevice, [(device, adbPath, packageName, apkPath) for device in devices]);
+
+def installToAllDevices (doParallel, adbPath, packageName, apkPath):
+       devices = getDevices(adbPath)
+       installToDevices(devices, doParallel, adbPath, packageName, apkPath)
+
+def getAPKPath (buildRootPath, target):
+       package = getPackageAndLibrariesForTarget(target)[0]
+       return os.path.join(buildRootPath, getBuildRootRelativeAPKPath(package))
+
+def getPackageName (target):
+       package                 = getPackageAndLibrariesForTarget(target)[0]
+       manifestPath    = os.path.join(DEQP_DIR, "android", package.appDirName, "AndroidManifest.xml")
+
+       return parsePackageName(manifestPath)
+
+def findADB ():
+       adbInPath = which("adb")
+       if adbInPath != None:
+               return adbInPath
+
+       sdkPath = findSDK()
+       if sdkPath != None:
+               adbInSDK = os.path.join(sdkPath, "platform-tools", "adb")
+               if os.path.isfile(adbInSDK):
+                       return adbInSDK
+
+       return None
+
+def parseArgs ():
+       defaultADBPath          = findADB()
+       defaultBuildRoot        = getDefaultBuildRoot()
+
+       parser = argparse.ArgumentParser(os.path.basename(__file__),
+               formatter_class=argparse.ArgumentDefaultsHelpFormatter)
+       parser.add_argument('--build-root',
+               dest='buildRoot',
+               default=defaultBuildRoot,
+               help="Root build directory")
+       parser.add_argument('--adb',
+               dest='adbPath',
+               default=defaultADBPath,
+               help="ADB binary path",
+               required=(True if defaultADBPath == None else False))
+       parser.add_argument('--target',
+               dest='target',
+               help='Build target',
+               choices=['deqp', 'openglcts'],
+               default='deqp')
+       parser.add_argument('-p', '--parallel',
+               dest='doParallel',
+               action="store_true",
+               help="Install package in parallel")
+       parser.add_argument('-s', '--serial',
+               dest='serial',
+               type=str,
+               nargs='+',
+               help="Install package to device with serial number")
+       parser.add_argument('-a', '--all',
+               dest='all',
+               action="store_true",
+               help="Install to all devices")
+
+       return parser.parse_args()
+
+if __name__ == "__main__":
+       args            = parseArgs()
+       packageName     = getPackageName(args.target)
+       apkPath         = getAPKPath(args.buildRoot, args.target)
+
+       if not os.path.isfile(apkPath):
+               die("%s does not exist" % apkPath)
+
+       if args.all:
+               installToAllDevices(args.doParallel, args.adbPath, packageName, apkPath)
+       else:
+               if args.serial == None:
+                       devices = getDevices(args.adbPath)
+                       if len(devices) == 0:
+                               die('No devices connected')
+                       elif len(devices) == 1:
+                               installToDevice(devices[0], args.adbPath, packageName, apkPath)
+                       else:
+                               print "More than one device connected:"
+                               for i in range(0, len(devices)):
+                                       print "%3d: %16s %s" % ((i+1), devices[i].serial, devices[i].model)
+
+                               deviceNdx = int(raw_input("Choose device (1-%d): " % len(devices)))
+                               installToDevice(devices[deviceNdx-1], args.adbPath, packageName, apkPath)
+               else:
+                       devices = getDevices(args.adbPath)
+
+                       devices = [dev for dev in devices if dev.serial in args.serial]
+                       devSerials = [dev.serial for dev in devices]
+                       notFounds = [serial for serial in args.serial if not serial in devSerials]
+
+                       for notFound in notFounds:
+                               print("Couldn't find device matching serial '%s'" % notFound)
+
+                       installToDevices(devices, args.doParallel, args.adbPath, packageName, apkPath)
index 1f030fa..f6f9074 100644 (file)
@@ -42,7 +42,7 @@ def initBuildDir (config, generator):
        pushWorkingDir(config.getBuildDir())
 
        try:
-               execute(["cmake", config.getSrcPath()] + cfgArgs)
+               execute([config.getCMakePath(), config.getSrcPath()] + cfgArgs)
        finally:
                popWorkingDir()
 
@@ -58,7 +58,7 @@ def build (config, generator, targets = None):
        else:
                initBuildDir(config, generator)
 
-       baseCmd         = ['cmake', '--build', '.']
+       baseCmd         = [config.getCMakePath(), '--build', '.']
        buildArgs       = generator.getBuildArgs(config.getBuildType())
 
        pushWorkingDir(config.getBuildDir())
index 069905e..c1ea6de 100644 (file)
 #-------------------------------------------------------------------------
 
 import os
+import sys
 import shlex
+import platform
 import subprocess
 
 DEQP_DIR = os.path.realpath(os.path.normpath(os.path.join(os.path.dirname(__file__), "..", "..")))
 
+# HostInfo describes properties of the host where these scripts
+# are running on.
+class HostInfo:
+       OS_WINDOWS      = 0
+       OS_LINUX        = 1
+       OS_OSX          = 2
+
+       @staticmethod
+       def getOs ():
+               if sys.platform == 'darwin':
+                       return HostInfo.OS_OSX
+               elif sys.platform == 'win32':
+                       return HostInfo.OS_WINDOWS
+               elif sys.platform.startswith('linux'):
+                       return HostInfo.OS_LINUX
+               else:
+                       raise Exception("Unknown sys.platform '%s'" % sys.platform)
+
+       @staticmethod
+       def getArchBits ():
+               MACHINE_BITS = {
+                       "i386":         32,
+                       "i686":         32,
+                       "x86":          32,
+                       "x86_64":       64,
+                       "AMD64":        64
+               }
+               machine = platform.machine()
+
+               if not machine in MACHINE_BITS:
+                       raise Exception("Unknown platform.machine() '%s'" % machine)
+
+               return MACHINE_BITS[machine]
+
 def die (msg):
        print(msg)
        exit(-1)
@@ -62,11 +98,26 @@ def writeFile (filename, data):
        f.write(data)
        f.close()
 
-def which (binName):
-       for path in os.environ['PATH'].split(os.pathsep):
-               path = path.strip('"')
-               fullPath = os.path.join(path, binName)
-               if os.path.isfile(fullPath) and os.access(fullPath, os.X_OK):
-                       return fullPath
+def which (binName, paths = None):
+       if paths == None:
+               paths = os.environ['PATH'].split(os.pathsep)
+
+       def whichImpl (binWithExt):
+               for path in paths:
+                       path = path.strip('"')
+                       fullPath = os.path.join(path, binWithExt)
+                       if os.path.isfile(fullPath) and os.access(fullPath, os.X_OK):
+                               return fullPath
+
+               return None
+
+       extensions = [""]
+       if HostInfo.getOs() == HostInfo.OS_WINDOWS:
+               extensions += [".exe", ".bat"]
+
+       for extension in extensions:
+               extResult = whichImpl(binName + extension)
+               if extResult != None:
+                       return extResult
 
        return None
index 0639350..c48dfc5 100644 (file)
 import os
 import sys
 import copy
-import platform
 import multiprocessing
 
-from common import which, DEQP_DIR
+from common import which, HostInfo, DEQP_DIR
 
 try:
        import _winreg
@@ -39,6 +38,7 @@ class BuildConfig:
                self.buildDir           = buildDir
                self.buildType          = buildType
                self.args                       = copy.copy(args)
+               self.cmakePath          = BuildConfig.findCMake()
 
        def getSrcPath (self):
                return self.srcPath
@@ -52,6 +52,22 @@ class BuildConfig:
        def getArgs (self):
                return self.args
 
+       def getCMakePath (self):
+               return self.cmakePath
+
+       @staticmethod
+       def findCMake ():
+               if which("cmake") == None:
+                       possiblePaths = [
+                               "/Applications/CMake.app/Contents/bin/cmake"
+                       ]
+                       for path in possiblePaths:
+                               if os.path.exists(path):
+                                       return path
+
+               # Fall back to PATH - may fail later
+               return "cmake"
+
 class CMakeGenerator:
        def __init__ (self, name, isMultiConfig = False, extraBuildArgs = []):
                self.name                       = name
@@ -90,7 +106,7 @@ class NMakeGenerator(CMakeGenerator):
                CMakeGenerator.__init__(self, "NMake Makefiles")
 
        def isAvailable (self):
-               return which('nmake.exe') != None
+               return which('nmake') != None
 
 class NinjaGenerator(CMakeGenerator):
        def __init__(self):
@@ -117,14 +133,14 @@ class VSProjectGenerator(CMakeGenerator):
 
        @staticmethod
        def getNativeArch ():
-               arch = platform.machine().lower()
+               bits = HostInfo.getArchBits()
 
-               if arch == 'x86':
+               if bits == 32:
                        return VSProjectGenerator.ARCH_32BIT
-               elif arch == 'amd64':
+               elif bits == 64:
                        return VSProjectGenerator.ARCH_64BIT
                else:
-                       raise Exception("Unhandled arch '%s'" % arch)
+                       raise Exception("Unhandled bits '%s'" % bits)
 
        @staticmethod
        def registryKeyAvailable (root, arch, name):
diff --git a/targets/android/ndk-r11.cmake b/targets/android/ndk-r11.cmake
new file mode 100644 (file)
index 0000000..4ac30af
--- /dev/null
@@ -0,0 +1,188 @@
+#-------------------------------------------------------------------------
+# drawElements CMake utilities
+# ----------------------------
+#
+# Copyright 2016 The Android Open Source Project
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+#-------------------------------------------------------------------------
+
+# Platform defines.
+set(CMAKE_SYSTEM_NAME Linux)
+
+set_property(GLOBAL PROPERTY TARGET_SUPPORTS_SHARED_LIBS TRUE)
+
+set(CMAKE_CROSSCOMPILING 1)
+
+# NDK installation path
+if (NOT DEFINED ANDROID_NDK_PATH)
+       message(FATAL_ERROR "Please provide ANDROID_NDK_PATH")
+endif ()
+
+# Host os (for toolchain binaries)
+if (NOT DEFINED ANDROID_NDK_HOST_OS)
+       message(FATAL_ERROR "Please provide ANDROID_NDK_HOST_OS")
+endif ()
+
+# Compile target
+set(ANDROID_ABI                        "armeabi-v7a"                   CACHE STRING "Android ABI")
+set(ANDROID_NDK_TARGET "android-${DE_ANDROID_API}")
+
+# dE defines
+set(DE_OS "DE_OS_ANDROID")
+
+if (NOT DEFINED DE_COMPILER)
+       set(DE_COMPILER "DE_COMPILER_CLANG")
+endif ()
+
+if (NOT DEFINED DE_ANDROID_API)
+       set(DE_ANDROID_API 9)
+endif ()
+
+set(COMMON_C_FLAGS             "-D__STDC_INT64__")
+set(COMMON_CXX_FLAGS   "${COMMON_C_FLAGS} -frtti -fexceptions")
+set(COMMON_LINKER_FLAGS        "")
+
+# ABI-dependent bits
+if (ANDROID_ABI STREQUAL "x86")
+       set(DE_CPU                                      "DE_CPU_X86")
+       set(CMAKE_SYSTEM_PROCESSOR      i686-android-linux)
+
+       set(ANDROID_CC_PATH                     "${ANDROID_NDK_PATH}/toolchains/x86-4.9/prebuilt/${ANDROID_NDK_HOST_OS}/")
+       set(CROSS_COMPILE                       "${ANDROID_CC_PATH}bin/i686-linux-android-")
+       set(ANDROID_SYSROOT                     "${ANDROID_NDK_PATH}/platforms/${ANDROID_NDK_TARGET}/arch-x86")
+
+       set(CMAKE_FIND_ROOT_PATH
+               "${ANDROID_CC_PATH}i686-linux-android"
+               "${ANDROID_CC_PATH}lib/gcc/i686-linux-android/4.9"
+               )
+
+       set(TARGET_C_FLAGS                      "-march=i686 -msse3 -mstackrealign -mfpmath=sse")
+       set(TARGET_LINKER_FLAGS         "")
+       set(LLVM_TRIPLE                         "i686-none-linux-android")
+
+elseif (ANDROID_ABI STREQUAL "armeabi" OR
+               ANDROID_ABI STREQUAL "armeabi-v7a")
+       set(DE_CPU                                      "DE_CPU_ARM")
+       set(CMAKE_SYSTEM_PROCESSOR      arm-linux-androideabi)
+
+       set(ANDROID_CC_PATH     "${ANDROID_NDK_PATH}/toolchains/arm-linux-androideabi-4.9/prebuilt/${ANDROID_NDK_HOST_OS}/")
+       set(CROSS_COMPILE       "${ANDROID_CC_PATH}bin/arm-linux-androideabi-")
+       set(ANDROID_SYSROOT     "${ANDROID_NDK_PATH}/platforms/${ANDROID_NDK_TARGET}/arch-arm")
+       set(ARM_C_FLAGS         "-D__ARM_ARCH_5__ -D__ARM_ARCH_5T__ -D__ARM_ARCH_5E__ -D__ARM_ARCH_5TE__ ")
+
+       if (ANDROID_ABI STREQUAL "armeabi-v7a")
+               set(TARGET_C_FLAGS              "${ARM_C_FLAGS} -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp")
+               set(TARGET_LINKER_FLAGS "-Wl,--fix-cortex-a8 -march=armv7-a")
+               set(LLVM_TRIPLE                 "armv7-none-linux-androideabi")
+
+       else () # armeabi
+               set(TARGET_C_FLAGS              "${ARM_C_FLAGS} -march=armv5te -mfloat-abi=softfp")
+               set(TARGET_LINKER_FLAGS "-Wl,--fix-cortex-a8 -march=armv5te")
+               set(LLVM_TRIPLE                 "armv5te-none-linux-androideabi")
+       endif ()
+
+       set(CMAKE_FIND_ROOT_PATH
+               "${ANDROID_CC_PATH}arm-linux-androideabi"
+               )
+
+elseif (ANDROID_ABI STREQUAL "arm64-v8a")
+       set(DE_CPU                                      "DE_CPU_ARM_64")
+       set(CMAKE_SYSTEM_PROCESSOR      aarch64-linux-android)
+       set(CMAKE_SIZEOF_VOID_P         8)
+
+       set(ANDROID_CC_PATH     "${ANDROID_NDK_PATH}/toolchains/aarch64-linux-android-4.9/prebuilt/${ANDROID_NDK_HOST_OS}/")
+       set(CROSS_COMPILE       "${ANDROID_CC_PATH}bin/aarch64-linux-android-")
+       set(ANDROID_SYSROOT     "${ANDROID_NDK_PATH}/platforms/${ANDROID_NDK_TARGET}/arch-arm64")
+
+       set(CMAKE_FIND_ROOT_PATH
+               "${ANDROID_CC_PATH}arm-linux-androideabi"
+               )
+
+       set(TARGET_C_FLAGS              "-march=armv8-a")
+       set(TARGET_LINKER_FLAGS "-Wl,--fix-cortex-a53-835769 -Wl,--fix-cortex-a53-835769 -march=armv8-a")
+       set(LLVM_TRIPLE                 "aarch64-none-linux-android")
+
+       if (DE_COMPILER STREQUAL "DE_COMPILER_GCC")
+               set(TARGET_C_FLAGS "${TARGET_C_FLAGS} -mabi=lp64")
+       endif ()
+
+elseif (ANDROID_ABI STREQUAL "x86_64")
+       set(DE_CPU                                      "DE_CPU_X86_64")
+       set(CMAKE_SYSTEM_PROCESSOR      x86_64-linux-android)
+       set(CMAKE_SIZEOF_VOID_P         8)
+
+       set(CMAKE_LIBRARY_PATH "/usr/lib64")
+
+       set(ANDROID_CC_PATH     "${ANDROID_NDK_PATH}/toolchains/x86_64-4.9/prebuilt/${ANDROID_NDK_HOST_OS}/")
+       set(CROSS_COMPILE       "${ANDROID_CC_PATH}bin/x86_64-linux-android-")
+       set(ANDROID_SYSROOT     "${ANDROID_NDK_PATH}/platforms/${ANDROID_NDK_TARGET}/arch-x86_64")
+
+       set(CMAKE_FIND_ROOT_PATH
+               "${ANDROID_CC_PATH}x86_64-linux-android"
+               )
+
+       set(LLVM_TRIPLE                 "x86_64-none-linux-android")
+
+else ()
+       message(FATAL_ERROR "Unknown ABI \"${ANDROID_ABI}\"")
+endif ()
+
+# Use LLVM libc++ for full C++11 support
+set(ANDROID_CXX_LIBRARY                "${ANDROID_NDK_PATH}/sources/cxx-stl/llvm-libc++/libs/${ANDROID_ABI}/libc++_static.a")
+set(CXX_INCLUDES                       "-I${ANDROID_NDK_PATH}/sources/cxx-stl/llvm-libc++/libcxx/include")
+set(CMAKE_FIND_ROOT_PATH       "" ${CMAKE_FIND_ROOT_PATH})
+
+set(CMAKE_FIND_ROOT_PATH ${CMAKE_FIND_ROOT_PATH} ${ANDROID_SYSROOT})
+
+include(CMakeForceCompiler)
+
+if (ANDROID_NDK_HOST_OS STREQUAL "windows" OR
+       ANDROID_NDK_HOST_OS STREQUAL "windows-x86_64")
+       set(BIN_EXT ".exe")
+else ()
+       set(BIN_EXT "")
+endif ()
+
+if (DE_COMPILER STREQUAL "DE_COMPILER_GCC")
+       set(CMAKE_C_COMPILER    "${CROSS_COMPILE}gcc${BIN_EXT}"         CACHE FILEPATH "C Compiler")
+       set(CMAKE_CXX_COMPILER  "${CROSS_COMPILE}g++${BIN_EXT}"         CACHE FILEPATH "C++ Compiler")
+
+       set(TARGET_C_FLAGS              "-mandroid ${TARGET_C_FLAGS}")
+
+elseif (DE_COMPILER STREQUAL "DE_COMPILER_CLANG")
+       set(LLVM_PATH "${ANDROID_NDK_PATH}/toolchains/llvm/prebuilt/${ANDROID_NDK_HOST_OS}/")
+
+       set(CMAKE_C_COMPILER    "${LLVM_PATH}bin/clang${BIN_EXT}"       CACHE FILEPATH "C Compiler")
+       set(CMAKE_CXX_COMPILER  "${LLVM_PATH}bin/clang++${BIN_EXT}"     CACHE FILEPATH "C++ Compiler")
+       set(CMAKE_AR                    "${CROSS_COMPILE}ar${BIN_EXT}"          CACHE FILEPATH "Archiver")
+       set(CMAKE_RANLIB                "${CROSS_COMPILE}ranlib${BIN_EXT}"      CACHE FILEPATH "Indexer")
+
+       set(TARGET_C_FLAGS              "-target ${LLVM_TRIPLE} -gcc-toolchain ${ANDROID_CC_PATH} ${TARGET_C_FLAGS}")
+       set(TARGET_LINKER_FLAGS "-target ${LLVM_TRIPLE} -gcc-toolchain ${ANDROID_CC_PATH} ${TARGET_LINKER_FLAGS}")
+
+endif ()
+
+set(CMAKE_SHARED_LIBRARY_C_FLAGS       "")
+set(CMAKE_SHARED_LIBRARY_CXX_FLAGS     "")
+
+set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
+set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
+set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
+
+# \note Without CACHE STRING FORCE cmake ignores these.
+set(CMAKE_C_FLAGS                              "--sysroot=${ANDROID_SYSROOT} ${CMAKE_C_FLAGS} ${COMMON_C_FLAGS} ${TARGET_C_FLAGS}" CACHE STRING "" FORCE)
+set(CMAKE_CXX_FLAGS                            "--sysroot=${ANDROID_SYSROOT} ${CMAKE_CXX_FLAGS} ${COMMON_CXX_FLAGS} ${TARGET_C_FLAGS} ${CXX_INCLUDES} -I${ANDROID_NDK_PATH}/sources/android/support/include" CACHE STRING "" FORCE)
+set(CMAKE_SHARED_LINKER_FLAGS  "-nodefaultlibs -Wl,-shared,-Bsymbolic -Wl,--no-undefined ${COMMON_LINKER_FLAGS} ${TARGET_LINKER_FLAGS}" CACHE STRING "" FORCE)
+set(CMAKE_EXE_LINKER_FLAGS             "-nodefaultlibs ${COMMON_LINKER_FLAGS} ${TARGET_LINKER_FLAGS}" CACHE STRING "" FORCE)