From d8a173ddc7331075c3e25afa97f85321fca9ebcf Mon Sep 17 00:00:00 2001 From: Wouter van Oortmerssen Date: Thu, 31 Jul 2014 11:39:40 -0700 Subject: [PATCH] A few document clarifications for Java & Internals. Change-Id: I770b53cf7d82c860422c1fe6193fb597d9c9495c --- docs/html/md__internals.html | 2 +- docs/html/md__java_usage.html | 11 +++++++---- docs/html/md__schemas.html | 2 +- docs/source/Internals.md | 4 +++- docs/source/JavaUsage.md | 22 +++++++++++++++------- 5 files changed, 27 insertions(+), 14 deletions(-) diff --git a/docs/html/md__internals.html b/docs/html/md__internals.html index 49f8287..e277beb 100644 --- a/docs/html/md__internals.html +++ b/docs/html/md__internals.html @@ -56,7 +56,7 @@ $(document).ready(function(){initNavTree('md__internals.html','');});

This section is entirely optional for the use of FlatBuffers. In normal usage, you should never need the information contained herein. If you're interested however, it should give you more of an appreciation of why FlatBuffers is both efficient and convenient.

Format components

A FlatBuffer is a binary file and in-memory format consisting mostly of scalars of various sizes, all aligned to their own size. Each scalar is also always represented in little-endian format, as this corresponds to all commonly used CPUs today. FlatBuffers will also work on big-endian machines, but will be slightly slower because of additional byte-swap intrinsics.

-

On purpose, the format leaves a lot of details about where exactly things live in memory undefined, e.g. fields in a table can have any order, and objects to some extend can be stored in many orders. This is because the format doesn't need this information to be efficient, and it leaves room for optimization and extension (for example, fields can be packed in a way that is most compact). Instead, the format is defined in terms of offsets and adjacency only.

+

On purpose, the format leaves a lot of details about where exactly things live in memory undefined, e.g. fields in a table can have any order, and objects to some extend can be stored in many orders. This is because the format doesn't need this information to be efficient, and it leaves room for optimization and extension (for example, fields can be packed in a way that is most compact). Instead, the format is defined in terms of offsets and adjacency only. This may mean two different implementations may produce different binaries given the same input values, and this is perfectly valid.

Format identification

The format also doesn't contain information for format identification and versioning, which is also by design. FlatBuffers is a statically typed system, meaning the user of a buffer needs to know what kind of buffer it is. FlatBuffers can of course be wrapped inside other containers where needed, or you can use its union feature to dynamically identify multiple possible sub-objects stored. Additionally, it can be used together with the schema parser if full reflective capabilities are desired.

Versioning is something that is intrinsically part of the format (the optionality / extensibility of fields), so the format itself does not need a version number (it's a meta-format, in a sense). We're hoping that this format can accommodate all data needed. If format breaking changes are ever necessary, it would become a new kind of format rather than just a variation.

diff --git a/docs/html/md__java_usage.html b/docs/html/md__java_usage.html index 44e30e5..99b9230 100644 --- a/docs/html/md__java_usage.html +++ b/docs/html/md__java_usage.html @@ -53,15 +53,15 @@ $(document).ready(function(){initNavTree('md__java_usage.html','');});
Use in Java
-

There's experimental support for reading FlatBuffers in Java. Generate code for Java with the -j option to flatc.

-

See javaTest.java for an example. Essentially, you read a FlatBuffer binary file into a byte[], which you then turn into a ByteBuffer, which you pass to the getRootAsMonster function: 

ByteBuffer bb = ByteBuffer.wrap(data);
+
FlatBuffers supports reading and writing binary FlatBuffers in Java. Generate code for Java with the -j option to flatc.

+
See javaTest.java for an example. Essentially, you read a FlatBuffer binary file into a byte[], which you then turn into a ByteBuffer, which you pass to the getRootAsMyRootType function: 
ByteBuffer bb = ByteBuffer.wrap(data);
 Monster monster = Monster.getRootAsMonster(bb);
 
Now you can access values much like C++: 
short hp = monster.hp();
 Vec3 pos = monster.pos();
 
Note that whenever you access a new object like in the pos example above, a new temporary accessor object gets created. If your code is very performance sensitive (you iterate through a lot of objects), there's a second pos() method to which you can pass a Vec3 object you've already created. This allows you to reuse it across many calls and reduce the amount of object allocation (and thus garbage collection) your program does.

 
Java does not support unsigned scalars. This means that any unsigned types you use in your schema will actually be represented as a signed value. This means all bits are still present, but may represent a negative value when used. For example, to read a byte b as an unsigned number, you can do: (short)(b & 0xFF)

 
Sadly the string accessors currently always create a new string when accessed, since FlatBuffer's UTF-8 strings can't be read in-place by Java.

-
Vector access is also a bit different from C++: you pass an extra index to the vector field accessor. Then a second method with the same name suffixed by _length let's you know the number of elements you can access: 
for (int i = 0; i < monster.inventory_length(); i++)
+
Vector access is also a bit different from C++: you pass an extra index to the vector field accessor. Then a second method with the same name suffixed by Length let's you know the number of elements you can access: 
for (int i = 0; i < monster.inventoryLength(); i++)
     monster.inventory(i); // do something here
 
You can also construct these buffers in Java using the static methods found in the generated code, and the FlatBufferBuilder class: 
FlatBufferBuilder fbb = new FlatBufferBuilder();
 
Create strings: 
int str = fbb.createString("MyMonster");
@@ -74,7 +74,10 @@ Monster.addTest_type(fbb, (byte)1);
 Monster.addTest(fbb, mon2);
 Monster.addTest4(fbb, test4s);
 int mon = Monster.endMonster(fbb);
-
As you can see, the Java code for tables does not use a convenient createMonster call like the C++ code. This is to create the buffer without using temporary object allocation (since the Vec3 is an inline component of Monster, it has to be created right where it is added, whereas the name and the inventory are not inline). Structs do have convenient methods that even have arguments for nested structs.

+
As you can see, the Java code for tables does not use a convenient createMonster call like the C++ code. This is to create the buffer without using temporary object allocation.

+
It's important to understand that fields that are structs are inline (like Vec3 above), and MUST thus be created between the start and end calls of a table. Everything else (other tables, strings, vectors) MUST be created before the start of the table they are referenced in.

+
Structs do have convenient methods that even have arguments for nested structs.

+
As you can see, references to other objects (e.g. the string above) are simple ints, and thus do not have the type-safety of the Offset type in C++. Extra case must thus be taken that you set the right offset on the right field.

 
Vectors also use this start/end pattern to allow vectors of both scalar types and structs: 
Monster.startInventoryVector(fbb, 5);
 for (byte i = 4; i >=0; i--) fbb.addByte(i);
 int inv = fbb.endVector();
diff --git a/docs/html/md__schemas.html b/docs/html/md__schemas.html
index 0ddf7e1..93bf4c3 100644
--- a/docs/html/md__schemas.html
+++ b/docs/html/md__schemas.html
@@ -126,7 +126,7 @@ root_type Monster;
 
  • bit_flags (on an enum): the values of this field indicate bits, meaning that any value N specified in the schema will end up representing 1<<N, or if you don't specify values at all, you'll get the sequence 1, 2, 4, 8, ...
    • 
 
 
    JSON Parsing
    
-
    The same parser that is parsing the schema declarations above is also able to parse JSON objects that conform to this schema. So, unlike other JSON parsers, this parser is strongly typed, and parses directly into a FlatBuffer (see the compiler documentation on how to do this from the command line, or the C++ documentation on how to do this at runtime).
    
+
    The same parser that parses the schema declarations above is also able to parse JSON objects that conform to this schema. So, unlike other JSON parsers, this parser is strongly typed, and parses directly into a FlatBuffer (see the compiler documentation on how to do this from the command line, or the C++ documentation on how to do this at runtime).
    
 
    Besides needing a schema, there are a few other changes to how it parses JSON:
    
 
      
 
    • It accepts field names with and without quotes, like many JSON parsers already do. It outputs them without quotes as well, though can be made to output them using the strict_json flag.
      • 
diff --git a/docs/source/Internals.md b/docs/source/Internals.md
index b0fbbdc..afe994f 100755
--- a/docs/source/Internals.md
+++ b/docs/source/Internals.md
@@ -20,7 +20,9 @@ order, and objects to some extend can be stored in many orders. This is
 because the format doesn't need this information to be efficient, and it
 leaves room for optimization and extension (for example, fields can be
 packed in a way that is most compact). Instead, the format is defined in
-terms of offsets and adjacency only.
+terms of offsets and adjacency only. This may mean two different
+implementations may produce different binaries given the same input
+values, and this is perfectly valid.
 
 ### Format identification
 
diff --git a/docs/source/JavaUsage.md b/docs/source/JavaUsage.md
index f231a2f..4e2fe8b 100755
--- a/docs/source/JavaUsage.md
+++ b/docs/source/JavaUsage.md
@@ -1,11 +1,11 @@
 # Use in Java
 
-There's experimental support for reading FlatBuffers in Java. Generate code
+FlatBuffers supports reading and writing binary FlatBuffers in Java. Generate code
 for Java with the `-j` option to `flatc`.
 
 See `javaTest.java` for an example. Essentially, you read a FlatBuffer binary
 file into a `byte[]`, which you then turn into a `ByteBuffer`, which you pass to
-the `getRootAsMonster` function:
+the `getRootAsMyRootType` function:
 
     ByteBuffer bb = ByteBuffer.wrap(data);
     Monster monster = Monster.getRootAsMonster(bb);
@@ -33,9 +33,9 @@ since FlatBuffer's UTF-8 strings can't be read in-place by Java.
 
 Vector access is also a bit different from C++: you pass an extra index
 to the vector field accessor. Then a second method with the same name
-suffixed by `_length` let's you know the number of elements you can access:
+suffixed by `Length` let's you know the number of elements you can access:
 
-    for (int i = 0; i < monster.inventory_length(); i++)
+    for (int i = 0; i < monster.inventoryLength(); i++)
         monster.inventory(i); // do something here
 
 You can also construct these buffers in Java using the static methods found
@@ -61,11 +61,19 @@ Create a table with a struct contained therein:
 
 As you can see, the Java code for tables does not use a convenient
 `createMonster` call like the C++ code. This is to create the buffer without
-using temporary object allocation (since the `Vec3` is an inline component of
-`Monster`, it has to be created right where it is added, whereas the name and
-the inventory are not inline).
+using temporary object allocation.
+
+It's important to understand that fields that are structs are inline (like
+`Vec3` above), and MUST thus be created between the start and end calls of
+a table. Everything else (other tables, strings, vectors) MUST be created
+before the start of the table they are referenced in.
+
 Structs do have convenient methods that even have arguments for nested structs.
 
+As you can see, references to other objects (e.g. the string above) are simple
+ints, and thus do not have the type-safety of the Offset type in C++. Extra
+case must thus be taken that you set the right offset on the right field.
+
 Vectors also use this start/end pattern to allow vectors of both scalar types
 and structs:
 
-- 
2.7.4