Fix possible importing errors in build_libtorch.py (#15471)

Summary:
1. I fixed the importing process, which had some problems
    -  **I think `setup_helpers` should not be imported as the top level module. It can lead to many future errors. For example, what if `setup_helpers` imports another module from the upper level?** So we need to change it.
    - The code is not consistent with other modules in `tools` package. For example, other
    modules in the package imports `from tools.setuptools...` not `from setuptools...`.
    - **It should be able to run with `python -m tools.build_libtorch` command**  because this module is a part of the tools package. Currently, you cannot do that and I think it's simply wrong.

~~2. I Added platform specific warning messages.
    - I constantly forgot that I needed to define some environment variables in advance specific to my platform to build libtorch, especially when I'm working at a non pytorch root directory. So I thought adding warnings for common options would be helpful .~~

~~3. Made the build output path configurable. And a few other changes.~~

orionr  ebetica
Pull Request resolved: https://github.com/pytorch/pytorch/pull/15471

Differential Revision: D13709607

Pulled By: ezyang

fbshipit-source-id: 950d5727aa09f857d973538c50b1ab169d88da38

diff --git a/docs/libtorch.rst b/docs/libtorch.rst
index e11f1a0..6b5c411 100644 (file)
--- a/docs/libtorch.rst
+++ b/docs/libtorch.rst
@@ -1,17 +1,23 @@
 libtorch (C++-only)
 ===================
 
-The core of pytorch can be built and used without Python. A
+The core of pytorch does not depend on Python. A
 CMake-based build system compiles the C++ source code into a shared
 object, libtorch.so.
 
 Building libtorch
 -----------------
 
-There is a script which wraps the CMake build. Invoke it with
+You can use a python script/module located in tools package to build libtorch
+::
+   cd <pytorch_root>
+   # export some required environment variables
+   python -m tools.build_libtorch
+
 
+Alternatively, you can invoke a shell script in the same directory to achieve the same goal
 ::
-   cd pytorch
+   cd <pytorch_root>
    BUILD_TORCH=ON ONNX_NAMESPACE=onnx_torch bash tools/build_pytorch_libs.sh --use-nnpack caffe2
    ls torch/lib/tmp_install # output is produced here
    ls torch/lib/tmp_install/lib/libtorch.so # of particular interest
@@ -20,4 +26,4 @@ To produce libtorch.a rather than libtorch.so, set the environment variable `BUI
 
 To use ninja rather than make, set `CMAKE_GENERATOR="-GNinja" CMAKE_INSTALL="ninja install"`.
 
-Future work will simplify this further.
+Note that we are working on eliminating tools/build_pytorch_libs.sh in favor of a unified cmake build.

diff --git a/tools/build_libtorch.py b/tools/build_libtorch.py
index a03d7e9..eeddd85 100644 (file)
--- a/tools/build_libtorch.py
+++ b/tools/build_libtorch.py
@@ -1,13 +1,20 @@
 import argparse
 import os
+from os.path import dirname, abspath
 import shlex
 import subprocess
 import sys
 
+# By appending pytorch_root to sys.path, this module can import other torch
+# modules even when run as a standalone script. i.e., it's okay either you
+# do `python build_libtorch.py` or `python -m tools.build_libtorch`.
+pytorch_root = dirname(dirname(abspath(__file__)))
+sys.path.append(pytorch_root)
+
 # If you want to modify flags or environmental variables that is set when
 # building torch, you should do it in tools/setup_helpers/configure.py.
 # Please don't add it here unless it's only used in LibTorch.
-from setup_helpers.configure import get_libtorch_env_with_flags
+from tools.setup_helpers.configure import get_libtorch_env_with_flags, IS_WINDOWS
 
 if __name__ == '__main__':
     # Placeholder for future interface. For now just gives a nice -h.
@@ -15,7 +22,7 @@ if __name__ == '__main__':
     options = parser.parse_args()
 
     tools_path = os.path.dirname(os.path.abspath(__file__))
-    if sys.platform == 'win32':
+    if IS_WINDOWS:
         build_pytorch_libs = os.path.join(tools_path, 'build_pytorch_libs.bat')
     else:
         build_pytorch_libs = os.path.join(tools_path, 'build_pytorch_libs.sh')