Refactor Quantization.md to separate TFLite native quantization scheme from the more...

PiperOrigin-RevId: 241785572

diff --git a/mlir/g3doc/Quantization.md b/mlir/g3doc/Quantization.md
index ad1775c..35e2783 100644 (file)
--- a/mlir/g3doc/Quantization.md
+++ b/mlir/g3doc/Quantization.md
@@ -164,7 +164,8 @@ $$
 
 ## Usage within MLIR {#usage-within-mlir}
 
-There are several components to the quantization system within MLIR:
+There are several components to the quantization system being developed within
+MLIR:
 
 *   *Quantization* dialect containing:
 
@@ -179,27 +180,61 @@ There are several components to the quantization system within MLIR:
         instrumentation points within the computation where runtime statistics
         may help guide the quantization process.
 
-*   *QuantizedMath* dialect containing:
+*   [Integration with simulated quantization at training time](#fake-quant)
+
+*   [TFLite native quantization](#tflite-native-quantization)
+
+    *   The TFLite op-set natively supports uniform-quantized variants.
+    *   Passes and tools exist to convert directly from the *TensorFlow* dialect
+        to the TFLite quantized op-set.
+
+*   [*FxpMath* dialect](#fxp-math-dialect) containing (experimental) generalized
+    representations of fixed-point math ops and conversions:
 
     *   [Real math ops](#real-math-ops) representing common combinations of
         arithmetic operations that closely match corresponding fixed-point math
         concepts (as opposed to being spread across multiple ops as is typical
         in source dialects).
-    *   [Fixed-point math ops](#fixed-point-math-ops) that for carrying out
-        computations on integers, as are typically needed by uniform
-        quantization schemes.
+    *   [Fixed-point math ops](#fxp-math-ops) that for carrying out computations
+        on integers, as are typically needed by uniform quantization schemes.
     *   Passes to lower from real math ops to fixed-point math ops.
 
-*   [Solver tools](#solver-tools) which can generically operate on computations
-    expressed in the *QuantizedMath* dialect in order to convert from floating
-    point types to appropriate *QuantizedTypes*, allowing the computation to be
-    further lowered to integral math ops.
+*   [Solver tools](#solver-tools) which can (experimentally and generically
+    operate on computations expressed in the *FxpMath* dialect in order to
+    convert from floating point types to appropriate *QuantizedTypes*, allowing
+    the computation to be further lowered to integral math ops.
 
 Not every application of quantization will use all facilities. Specifically, the
 TensorFlow to TensorFlow Lite conversion uses the QuantizedTypes but has its own
 ops for type conversion and expression of the backing math.
 
-## Interactions with simulated quantization at training time {#training-time}
+## Quantization Dialect {#quantization-dialect}
+
+### Quantized type {#quantized-type}
+
+TODO : Flesh this section out.
+
+*   QuantizedType base class
+*   UniformQuantizedType
+
+### Quantized type conversion ops {#quantized-type-conversion-ops}
+
+*   qcast : Convert from an expressed type to QuantizedType
+*   dcast : Convert from a QuantizedType to its expressed type
+*   scast : Convert between a QuantizedType and its storage type
+
+### Instrumentation and constraint ops {#instrumentation-ops}
+
+TODO : These ops are not defined yet
+
+*   instrument_stats : Assigns a unique id and signals that statistics should be
+    collected by the runtime when execution passes through this op.
+*   constrain_uniform : Constrains that for uniform quantization, the solver
+    should choose a type with certain characteristics such as the number of
+    fixed-point values, underlying storage type, or whether to constrain to
+    power of two scales.
+
+## Integration with simulated quantization at training time {#fake-quant}
 
 TensorFlow has historically used the
 [tf.quantization.fake_quant_\*](https://www.tensorflow.org/api_docs/python/tf/quantization/fake_quant_with_min_max_args)
@@ -214,44 +249,36 @@ judgment about how to convert to use kernels from its quantized ops subset.
 
 In MLIR-based quantization, fake_quant_\* ops are handled by converting them to
 a sequence of *qcast* (quantize) followed by *dcast* (dequantize) with an
-appropriate *UniformQuantizedType* as the target of the qbarrier operation.
+appropriate *UniformQuantizedType* as the target of the qcast operation.
 
 This allows subsequent compiler passes to preserve the knowledge that
 quantization was simulated in a certain way while giving the compiler
-flexibility to move the barriers as it simplifies the computation and converts
-it to a form based on integral arithmetic.
+flexibility to move the casts as it simplifies the computation and converts it
+to a form based on integral arithmetic.
 
 This scheme also naturally allows computations that are *partially quantized*
 where the parts which could not be reduced to integral ops are still carried out
 in floating point with appropriate conversions at the boundaries.
 
-## Quantization Dialect
-
-### Quantized type {#quantized-type}
-
-TODO : Flesh this section out.
-
-*   QuantizedType base class
-*   UniformQuantizedType
-
-### Quantized type conversion ops {#quantized-type-conversion-ops}
-
-*   qcast : Convert from an expressed type to QuantizedType
-*   dcast : Convert from a QuantizedType to its expressed type
-*   scast : Convert between a QuantizedType and its storage type
+## TFLite Native Quantization {#tflite-native-quantization}
 
-### Instrumentation and constraint ops {#instrumentation-ops}
+TODO : Flesh this out
 
-TODO : These ops are not defined yet
+### General algorithm
 
-*   instrument_stats : Assigns a unique id and signals that statistics should be
-    collected by the runtime when execution passes through this op.
-*   constrain_uniform : Constrains that for uniform quantization, the solver
-    should choose a type with certain characteristics such as the number of
-    fixed-point values, underlying storage type, or whether to constrain to
-    power of two scales.
+1.  Take input min/max information and set the ArrayInfo (which really is
+    InputOrOutputArrayInfo.
+1.  In LegalizeTF, convert ArrayInfo min/max to tf.Quantize and tf.Dequantize
+    nodes. (or tf.FakeQuant) Convert all constant FakeQuants to (tf.FQ -> tfl.Q
+    -> tfl.DQ).
+1.  Hardcode logic/propagation needs to happen here.
+1.  Run TF constant folding.
+1.  In PrepareTFL, convert all tf.FQ to (tfl.Q -> tfl.DQ).
+1.  Run quantization pass that take (tfl.DQ (for both input and weights) -> op
+    -> tfl.Q) and replaces with (op). Also replace (constant_float -> tfl.Q)
+    with (constant_quant).
 
-## QuantizedMath Dialect
+## FxpMath Dialect {#fxp-math-dialect}
 
 ### Real math ops {#real-math-ops}
 
@@ -283,7 +310,7 @@ TODO: This op set is still evolving and needs to be completed.
     *   CMPLZ
     *   CMPGZ
 
-### Fixed-point math ops {#fixed-point-math-ops}
+### Fixed-point math ops {#fxp-math-ops}
 
 TODO: This op set only has enough ops to lower a simple power-of-two
 RealAddEwOp.