projects / platform / upstream / tensorflow.git / commit
? search:
summary | shortlog | log | commit | commitdiff | tree
(parent: fc252eb) | patch
Add reservoir sampling to DB summary writer
authorJustine Tunney <jart@google.com>
Fri, 12 Jan 2018 00:08:50 +0000 (16:08 -0800)
committerTensorFlower Gardener <gardener@tensorflow.org>
Fri, 12 Jan 2018 00:12:46 +0000 (16:12 -0800)
commitfebdd26ae594133d24f82544706b1e012a5cf1ea
treedd325008019ab10ce35f98368bf392ce4a118ec9tree | snapshot
parentfc252eb976c98c95a625ea6e6a0486334d3c5b6ecommit | diff
Add reservoir sampling to DB summary writer

This thing is kind of cool. It's able to turn a 350mB event log into a
35mB SQLite file at 80mBps with one Macbook core. Best of all, this was
accomplished using a normalized schema without the embedded protos.

PiperOrigin-RevId: 181676380
12 files changed:
tensorflow/contrib/summary/summary_ops_test.py diff | blob | history
tensorflow/contrib/tensorboard/db/BUILD diff | blob | history
tensorflow/contrib/tensorboard/db/schema.cc diff | blob | history
tensorflow/contrib/tensorboard/db/summary_db_writer.cc diff | blob | history
tensorflow/contrib/tensorboard/db/summary_db_writer.h diff | blob | history
tensorflow/contrib/tensorboard/db/summary_db_writer_test.cc diff | blob | history
tensorflow/core/kernels/BUILD diff | blob | history
tensorflow/core/kernels/data/sql/sqlite_query_connection.cc diff | blob | history
tensorflow/core/kernels/summary_kernels.cc diff | blob | history
tensorflow/core/lib/db/BUILD diff | blob | history
tensorflow/core/lib/db/sqlite.cc diff | blob | history
tensorflow/core/lib/db/sqlite.h diff | blob | history
Domain: Machine Learning / ML Framework;