Code

Record hashing perf#781

Code

Merged

johnkerl merged 31 commits intomainjohnkerl/miller:mainfrom record-hashing-perfjohnkerl/miller:record-hashing-perfCopy head branch name to clipboardDec 15, 2021

+67-8

Commits

Commits on Dec 13, 2021

• todo

  johnkerl

  committed
  02557d0

  Copy full SHA for 02557d0

• Merge branch 'main' of git+ssh://github.com/johnkerl/miller

  johnkerl

  committed
  bfefc8d

  Copy full SHA for bfefc8d

• Rename inputChannel,outputChannel to readerChannel,writerChannel

  johnkerl

  committed
  03cd9e0

  Copy full SHA for 03cd9e0

• Rename inputChannel,outputChannel to readerChannel,writerChannel (#772)

  johnkerl

  committed
  8abd334

  Copy full SHA for 8abd334

• Start batched-reader API mods

  johnkerl

  committed
  2f8d2b1

  Copy full SHA for 2f8d2b1

• Singleton-list step for reader-batching at input

  johnkerl

  committed
  f2879ae

  Copy full SHA for f2879ae

• CLI options for records-per-batch and hash-records

  johnkerl

  committed
  5c02275

  Copy full SHA for 5c02275

• Push channelized-reader logic into DKVP reader

  johnkerl

  committed
  aaf0c27

  Copy full SHA for aaf0c27

• Push batching logic into chain-transformer, transformers, and channel-writer

  johnkerl

  committed
  9bdc53d

  Copy full SHA for 9bdc53d

• foo

  johnkerl

  committed
  0335c4e

  Copy full SHA for 0335c4e

• cmd/mprof and cmd/mprof2

  johnkerl

  committed
  7f1aced

  Copy full SHA for 7f1aced

• cmd/mprof3 and cmd/mprof4

  johnkerl

  committed
  6ad475b

  Copy full SHA for 6ad475b

• narrowed in on regexp-splitting on IFS/IPS as perf-hit

  johnkerl

  committed
  b6ab5d9

  Copy full SHA for b6ab5d9

• neaten

  johnkerl

  committed
  3faf66a

  Copy full SHA for 3faf66a

• channelize nidx

  johnkerl

  committed
  5666b59

  Copy full SHA for 5666b59

• cmd/mprof5

  johnkerl

  committed
  79bc7fd

  Copy full SHA for 79bc7fd

• channelize CSV reader

  johnkerl

  committed
  d183d11

  Copy full SHA for d183d11

• channelize NIDX reader

  johnkerl

  committed
  2618d63

  Copy full SHA for 2618d63

• Dedupe DKVP-reader and NIDX-reader source files

  johnkerl

  committed
  4df477f

  Copy full SHA for 4df477f

• channelize CSV-lite reader

  johnkerl

  committed
  be3286f

  Copy full SHA for be3286f

• channelize XTAB reader

  johnkerl

  committed
  8403549

  Copy full SHA for 8403549

• batchify JSON reader

  johnkerl

  committed
  f47b2dd

  Copy full SHA for f47b2dd

• channelize GEN pseudo-reader

  johnkerl

  committed
  e24a36d

  Copy full SHA for e24a36d

• scripts for perf-testing on larger files

  johnkerl

  committed
  34627e7

  Copy full SHA for 34627e7

• merge with main for #776

  johnkerl

  committed
  4aab213

  Copy full SHA for 4aab213

• Fix record-batching for join and repl

  johnkerl

  committed
  056b8e9

  Copy full SHA for 056b8e9

• Fix comment-handling in channelized XTAB reader

  johnkerl

  committed
  8cea6ae

  Copy full SHA for 8cea6ae

• Fix bug found in positional-rename

  johnkerl

  committed
  fe6d6cf

  Copy full SHA for fe6d6cf

• merge

  johnkerl

  committed
  641ecbd

  Copy full SHA for 641ecbd

• merge

  johnkerl

  committed
  ba6286c

  Copy full SHA for ba6286c

Commits on Dec 15, 2021

• Use --no-hash-records by default

  johnkerl

  committed
  447f139

  Copy full SHA for 447f139