Streamline memory-bundle loads

[jerinphilip]

Provides an additional constructor which takes care of the bundle loading inside the boundary of the source here, when a configuration file is supplied from a client like translateLocally or python bindings. Once the config file is read, we have access to the information required to construct the MemoryBundle.

• The command-line application supplied from here, app/bergamot is configured to use the fast-load path now.
• Changes to binary-loading additionally revealed a bug in the example-run script used in docs and tied to CI and the fix is included.
• Shortlist is made optional in the memory-bundle, making changes to getModelMemoryFromConfig.

Fixes #304.
Fixes #306.
See also: XapaJIaMnu/translateLocally#82.

PS: There is a need to refactor of byte_array_utils, aggressive reduction of paths and branches and consolidation of existing fragmented functions into manageable classes, but keeping this PR minimal to simplify review.

[jerinphilip]

The following are from jerinphilip/lemonade#43, would appreciate if someone tested with translateLocally and let know if things work.

With shortlist:

 [jerin@eltbk lemonade]$ bergamot translate -m en-de-tiny --html=1 --log-level=info <<< 'The quick brown <i id="fox">fox</i> jumps <u>over</u> the lazy <i id="dog">dog</i>.' # Forward translation
[2022-01-19 15:08:58] [data] Loading SentencePiece vocabulary from file /home/jerin/.local/share/lemonade/models/ende.student.tiny11/vocab.deen.spm
[2022-01-19 15:08:59] Missing list of protected prefixes for sentence splitting. Set with --ssplit-prefix-file.
[2022-01-19 15:08:59] [data] Loading binary shortlist as /home/jerin/.local/share/lemonade/models/ende.student.tiny11/lex.s2t.bin false
[2022-01-19 15:08:59] [data] Lexical short list firstNum 50 and bestNum 50
[2022-01-19 15:08:59] [memory] Extending reserved space to 128 MB (device cpu0)
[2022-01-19 15:08:59] Loaded model config
[2022-01-19 15:08:59] Loading scorer of type transformer as feature F0
[2022-01-19 15:08:59] [memory] Reserving 31 MB, device cpu0
[2022-01-19 15:08:59] [memory] Reserving 8 MB, device cpu0
Der schnelle braune <i id="fox">Fuchs</i> springt <u>über</u> den faulen <i id="dog">Hund.</i>

Shortlist entries commented:

(env) [jerin@eltbk lemonade]$ bergamot translate -m en-de-tiny --html=1 --log-level=info <<< 'The quick brown <i id="fox">fox</i> jumps <u>over</u> the lazy <i id="dog">dog</i>.' # Forward translation
[2022-01-19 15:10:24] [data] Loading SentencePiece vocabulary from file /home/jerin/.local/share/lemonade/models/ende.student.tiny11/vocab.deen.spm
[2022-01-19 15:10:24] Missing list of protected prefixes for sentence splitting. Set with --ssplit-prefix-file.
[2022-01-19 15:10:24] [memory] Extending reserved space to 128 MB (device cpu0)
[2022-01-19 15:10:24] Loaded model config
[2022-01-19 15:10:24] Loading scorer of type transformer as feature F0
[2022-01-19 15:10:24] [memory] Reserving 31 MB, device cpu0
[2022-01-19 15:10:24] [memory] Reserving 8 MB, device cpu0
Der schnelle braune <i id="fox">Fuchs</i> springt <u>über</u> den faulen <i id="dog">Hund.</i>

[jelmervdl]

Tested it from TranlateLocally while removing the shortlist from a model. Translation still loads and works afterwards. Any other (invalid) changes to the model config still cause an error to popup, so that's good.