README.md

   1 # NNStreamer <img align="right" src="Documentation/media/lfaidata-project-badge-incubation-black.svg" width="150"/>
   2
   3 [![Gitter][gitter-image]][gitter-url]
   4 [![DailyBuild](http://ci.nnstreamer.ai/nnstreamer/ci/daily-build/badge/daily_build_badge.svg)](http://ci.nnstreamer.ai/nnstreamer/ci/daily-build/build_result/)
   5 [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/4401/badge)](https://bestpractices.coreinfrastructure.org/projects/4401)
   6 [![Total alerts](https://img.shields.io/lgtm/alerts/g/nnstreamer/nnstreamer.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/nnstreamer/nnstreamer/alerts/)
   7 [![Code Coverage](http://ci.nnstreamer.ai/nnstreamer/ci/badge/codecoverage.svg)](http://ci.nnstreamer.ai/nnstreamer/ci/gcov_html/index.html)
   8 <a href="https://scan.coverity.com/projects/nnstreamer-nnstreamer">
   9 <img alt="Coverity Scan Defect Status" src="https://scan.coverity.com/projects/22501/badge.svg"/>
  10 </a>
  11 ![GitHub repo size](https://img.shields.io/github/repo-size/nnstreamer/nnstreamer)
  12
  13 Neural Network Support as Gstreamer Plugins.
  14
  15 NNStreamer is a set of Gstreamer plugins that allow
  16 Gstreamer developers to adopt neural network models easily and efficiently and
  17 neural network developers to manage neural network pipelines and their filters easily and efficiently.
  18
  19 [Architectural Description](https://github.com/nnstreamer/nnstreamer/wiki/Architectural-Description) (WIP)<br /> <br />
  20
  21 [Toward Among-Device AI from On-Device AI with Stream Pipelines](https://conf.researchr.org/home/icse-2022), IEEE/ACM ICSE 2022 SEIP <br />
  22 [NNStreamer: Efficient and Agile Development of On-Device AI Systems](https://ieeexplore.ieee.org/document/9402062), IEEE/ACM ICSE 2021 SEIP [[media](https://youtu.be/HtNXFReF2GY)]<br />
  23 [NNStreamer: Stream Processing Paradigm for Neural Networks ...](https://arxiv.org/abs/1901.04985) [[pdf/tech report](https://arxiv.org/pdf/1901.04985)]<br />
  24 [GStreamer Conference 2018, NNStreamer](https://gstreamer.freedesktop.org/conference/2018/talks-and-speakers.html#nnstreamer-neural-networks-as-filters) [[media](https://github.com/nnstreamer/nnstreamer/wiki/Gstreamer-Conference-2018-Presentation-Video)] [[pdf/slides](https://github.com/nnstreamer/nnstreamer/wiki/slides/2018_GSTCON_Ham_181026.pdf)]<br />
  25 [Naver Tech Talk (Korean), 2018](https://www.facebook.com/naverengineering/posts/2255360384531425) [[media](https://youtu.be/XvXxcnbRjgU)] [[pdf/slides](https://www.slideshare.net/NaverEngineering/nnstreamer-stream-pipeline-for-arbitrary-neural-networks)]<br />
  26 [Samsung Developer Conference 2019, NNStreamer (media)](https://youtu.be/wVbMbpOjbkw)<br />
  27 [ResearchGate Page of NNStreamer](https://www.researchgate.net/project/Neural-Network-Streamer-nnstreamer)
  28
  29
  30 ## Official Releases
  31
  32 |     | [Tizen](https://download.tizen.org/snapshots/TIZEN/Tizen/Tizen-Unified/latest/repos/standard/packages/) | [Ubuntu](https://launchpad.net/~nnstreamer/+archive/ubuntu/ppa) | Android | [Yocto](https://github.com/nnstreamer/meta-neural-network) | macOS |
  33 | :-- | :--: | :--: | :--: | :--: | :--: |
  34 |     | 5.5M2 and later | ~~16.04~~/18.04/20.04/22.04 | 9/P | Kirkstone |   |
  35 | arm | [![armv7l badge](http://ci.nnstreamer.ai/nnstreamer/ci/daily-build/badge/armv7l_result_badge.svg)](http://ci.nnstreamer.ai/nnstreamer/ci/daily-build/build_result/latest/log/) | Available  | Available| Ready | N/A |
  36 | arm64 |  [![aarch64 badge](http://ci.nnstreamer.ai/nnstreamer/ci/daily-build/badge/aarch64_result_badge.svg)](http://ci.nnstreamer.ai/nnstreamer/ci/daily-build/build_result/latest/log/) | Available  | [![android badge](http://ci.nnstreamer.ai/nnstreamer/ci/daily-build/badge/android_result_badge.svg)](http://ci.nnstreamer.ai/nnstreamer/ci/daily-build/build_result/latest/log/) | [![yocto badge](http://ci.nnstreamer.ai/nnstreamer/ci/daily-build/badge/yocto_result_badge.svg)](http://ci.nnstreamer.ai/nnstreamer/ci/daily-build/build_result/latest/log/) | N/A |
  37 | x64 | [![x64 badge](http://ci.nnstreamer.ai/nnstreamer/ci/daily-build/badge/x86_64_result_badge.svg)](http://ci.nnstreamer.ai/nnstreamer/ci/daily-build/build_result/latest/log/)  | [![ubuntu badge](http://ci.nnstreamer.ai/nnstreamer/ci/daily-build/badge/ubuntu_result_badge.svg)](http://ci.nnstreamer.ai/nnstreamer/ci/daily-build/build_result/latest/log/)  | Ready  | Ready | Available |
  38 | x86 | [![x86 badge](http://ci.nnstreamer.ai/nnstreamer/ci/daily-build/badge/i586_result_badge.svg)](http://ci.nnstreamer.ai/nnstreamer/ci/daily-build/build_result/latest/log/)  | N/A  | N/A  | Ready | N/A |
  39 | Publish | [Tizen Repo](https://download.tizen.org/snapshots/TIZEN/Tizen/Tizen-Unified/latest/repos/standard/packages/) | [PPA](https://launchpad.net/~nnstreamer/+archive/ubuntu/ppa) | [Daily build](http://ci.nnstreamer.ai/nnstreamer/ci/daily-build/build_result/latest/android/) | [Layer](https://github.com/nnstreamer/meta-neural-network) | [Brew Tap](Documentation/getting-started-macos.md) |
  40 | API | C/C# (Official) | C | Java | C  | C  |
  41
  42 - Ready: CI system ensures build-ability and unit-testing. Users may easily build and execute. However, we do not have automated release & deployment system for this instance.
  43 - Available: binary packages are released and deployed automatically and periodically along with CI tests.
  44 - [Daily Release](http://ci.nnstreamer.ai/nnstreamer/ci/daily-build/build_result/latest/)
  45 - SDK Support: Tizen Studio (5.5 M2+) / Android Studio (JCenter, "nnstreamer")
  46 - [Enabled features of official releases](Documentation/features-per-distro.md)
  47
  48
  49
  50 ## Objectives
  51
  52 - Provide neural network framework connectivities (e.g., tensorflow, caffe) for gstreamer streams.
  53   - **Efficient Streaming for AI Projects**: Apply efficient and flexible stream pipeline to neural networks.
  54   - **Intelligent Media Filters!**: Use a neural network model as a media filter / converter.
  55   - **Composite Models!**: Multiple neural network models in a single stream pipeline instance.
  56   - **Multi Modal Intelligence!**: Multiple sources and stream paths for neural network models.
  57
  58 - Provide easy methods to construct media streams with neural network models using the de-facto-standard media stream framework, **GStreamer**.
  59   - Gstreamer users: use neural network models as if they are yet another media filters.
  60   - Neural network developers: manage media streams easily and efficiently.
  61
  62 ## Maintainers
  63 * [MyungJoo Ham](https://github.com/myungjoo/)
  64
  65 ## Committers
  66 * [Jijoong Moon](https://github.com/jijoongmoon)
  67 * [Geunsik Lim](https://github.com/leemgs)
  68 * [Sangjung Woo](https://github.com/again4you)
  69 * [Wook Song](https://github.com/wooksong)
  70 * [Jaeyun Jung](https://github.com/jaeyun-jung)
  71 * [Hyoungjoo Ahn](https://github.com/helloahn)
  72 * [Parichay Kapoor](https://github.com/kparichay)
  73 * [Dongju Chae](https://github.com/dongju-chae)
  74 * [Gichan Jang](https://github.com/gichan-jang)
  75 * [Yongjoo Ahn](https://github.com/anyj0527)
  76 * [Jihoon Lee](https://github.com/zhoonit)
  77
  78 ## Components
  79
  80 Note that this project has just started and many of the components are in design phase.
  81 In [Component Description](Documentation/component-description.md) page, we describe nnstreamer components of the following three categories: data type definitions, gstreamer elements (plugins), and other misc components.
  82
  83 ## Getting Started
  84 For more details, please access the following manuals.
  85 * For Linux-like systems such as Tizen, Debian, and Ubuntu, press [here](Documentation/getting-started.md).
  86 * For macOS systems, press [here](Documentation/getting-started-macos.md).
  87 * To build an API library for Android, press [here](Documentation/getting-started-android.md).
  88
  89 ## Applications
  90
  91
  92 <a href="https://github.com/nnstreamer/nnstreamer-example/blob/master/native/example_pose_estimation_tflite">
  93   <img src="https://raw.githubusercontent.com/nnstreamer/nnstreamer-example/master/native/example_pose_estimation_tflite/yongjoo2.webp" height=180>
  94 </a><a href="https://github.com/nnstreamer/nnstreamer-example/blob/master/native/example_image_classification_tflite">
  95   <img src="https://raw.githubusercontent.com/nnstreamer/nnstreamer-example/master/native/example_image_classification_tflite/image_classification_tflite_demo.webp" height=180>
  96 </a><a href="https://github.com/nnstreamer/nnstreamer-example/blob/master/native/example_object_detection_tensorflow_lite">
  97   <img src="https://raw.githubusercontent.com/nnstreamer/nnstreamer-example/master/native/example_object_detection_tensorflow_lite/object_detection_tflite_demo.webp" height=180>
  98 </a>
  99 <a href="https://github.com/nnstreamer/nnstreamer-example/tree/main/Tizen.platform/Tizen_IoT_face_landmark">
 100   <img src="Documentation/media/face_landmark.webp" height=180>
 101 </a>
 102 <a href="https://fainders.ai">
 103   <img src="Documentation/media/FaindersAI_Store.webp" height=180>
 104 </a>
 105 <a href="https://klleon.io">
 106   <img src="Documentation/media/Klleon_Speech.webp" height=180>
 107 </a>
 108 <a href="https://openncc.com">
 109   <img src="Documentation/media/OpenNCC_Mask.webp" height=180>
 110 </a>
 111 <a href="https://printnanny.ai">
 112   <img src="Documentation/media/PrintNanny.webp" height=180>
 113 </a>
 114 <a href="https://github.com/NXPmicro/nxp-nnstreamer-examples">
 115   <img src="https://github.com/NXPmicro/nxp-nnstreamer-examples/raw/main/segmentation/segmentation_demo.webp" height=180>
 116 </a>
 117 <a href="https://github.com/NXPmicro/nxp-nnstreamer-examples">
 118   <img src="https://github.com/NXPmicro/nxp-nnstreamer-examples/raw/main/pose/pose_demo.webp" height=180>
 119 </a>
 120
 121
 122 - [Edge-AI Examples](Documentation/edge-ai.md)
 123 - [Products with NNStreamer](Documentation/products.md)
 124 - NNStreamer example applications: [Github](https://github.com/nnstreamer/nnstreamer-example) / [Screenshots](https://github.com/nnstreamer/nnstreamer/wiki/usage-examples-screenshots)
 125
 126 ## CI Server
 127
 128 - [CI service status](http://ci.nnstreamer.ai/)
 129 - [TAOS-CI config files for nnstreamer](.TAOS-CI).
 130
 131 ## AI Acceleration Hardware Support
 132
 133 Although a framework may accelerate transparently as Tensorflow-GPU does, nnstreamer provides various hardware acceleration subplugins.
 134 - Movidius-X via ncsdk2 subplugin: Released
 135 - Movidius-X via openVINO subplugin: Released
 136 - Edge-TPU via edgetpu subplugin: Released
 137 - [**ONE**](https://github.com/Samsung/ONE) runtime via nnfw(an old name of **ONE**) subplugin: Released
 138 - ARMNN via armnn subplugin: Released
 139 - Verisilicon-Vivante via vivante subplugin: Released
 140 - Qualcomm SNPE via snpe subplugin: Released
 141 - NVidia via TensorRT subplugin: Released
 142 - TRI-x NPUs: Released
 143 - NXP i.MX series: [via the vendor](https://www.nxp.com/docs/en/user-guide/IMX-MACHINE-LEARNING-UG.pdf)
 144 - Others: TVM, TensorFlow, TensorFlow-lite, PyTorch, Caffe2, SNAP, ...
 145
 146
 147 [gitter-url]: https://gitter.im/nnstreamer/Lobby
 148 [gitter-image]: https://img.shields.io/badge/+%20GITTER-JOIN%20CHAT%20%E2%86%92-1DCE73.svg?style=flat-square
 149
 150
 151 ## Contributing
 152
 153 Contributions are welcome! Please see our [Contributing](Documentation/contributing.md) Guide for more details.
 154
 155 [![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nnstreamer/images/0)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nnstreamer/links/0)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nnstreamer/images/1)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nnstreamer/links/1)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nnstreamer/images/2)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nnstreamer/links/2)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nnstreamer/images/3)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nnstreamer/links/3)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nnstreamer/images/4)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nnstreamer/links/4)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nnstreamer/images/5)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nnstreamer/links/5)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nnstreamer/images/6)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nnstreamer/links/6)[![](https://sourcerer.io/fame/dongju-chae/nnstreamer/nnstreamer/images/7)](https://sourcerer.io/fame/dongju-chae/nnstreamer/nnstreamer/links/7)