Self-descriptive object

[hornc]

My idea for this year is to (try to) create a self-descriptive object. It will be a sort of 2nd person mostly linear choose your own adventure / text based walking sim where the world is the book...

The narration will be third person, but the reader is "you". I want some Stanley Parable-esque conflict between what the narrative says and what the reader chooses to do. There will be page jumps, and splitting of the character, I expect to have to track multiple versions of "the reader", offset in time and compliant vs. non-compliant readers. I still need to figure this out though.

There will be a lot of self-referencial description, and the code will generate this and keep things consistent. If the idea works well, it'll be a mystery walking-sim in book form. Having a physical copy printed would be kinda fun.

[hornc]

Repo up here: https://github.com/hornc/nanogenmo2022-albatross

After a first session (it's November in my timezone) I have a basic seed text start paragraph, and the super rough bones of a minimal 'reader' that reads it to generate more text:

The reader sits down in a comfortable spot and holds up the book in front of them.
It is a strange book.
The reader opens the book to the first page and begins to read:

The first letter encountered on the page is a capital 'T'. The next letter encountered is a lowercase 'h'. The next letter encountered is a lowercase 'e'. Then there is a space. The next letter encountered is a lowercase 'r'. The next letter encountered is a lowercase 'e'. The next letter encountered is a lowercase 'a'. The next letter encountered is a lowercase 'd'. The next letter encountered is a lowercase 'e'. The next letter encountered is a lowercase 'r'. Then there is a space. The next letter encountered is a lowercase 's'. The next letter encountered is a lowercase 't'. The next letter encountered is a lowercase 's'. Then there is a space. The next letter encountered is a lowercase 'd'.
...
The next letter encountered is a lowercase 't'. The next letter encountered is a lowercase 'h'. The next letter encountered is a lowercase 'e'. The next letter encountered is a lowercase 'm'. Then there is a period, ending the current sentence.

Next steps are to break up the descriptions to mix describing words and letters, add variations, probably with a Tracery grammar, and being a little more context aware, to make the thing more accurate, and slightly less tedious.

Then I'll need to make the description page and chapter division aware, and work on displaying the layout nicely.

Then I'll need to figure out how long a reader might realistically keep reading such a text, and describe what happens next... which is how the plot will evolve.

The current code outputs:

• The seed text
• A letter-by-letter reading of the start text.
• A letter-by-letter reading of the letter-by-letter reading of the start text.

Just these three blocks produce 49,468 words, 532 shy of the target, which is pretty good for the minimal effort so far.

./albatross.py data/start.txt | wc -w
49468

I only need 532 more words over the rest of the month to be done!

Going breadth first with NaNoGenMo seems a good strategy -- get the word count, then work on making those words incrementally better over November.

I'm considering this a MVP 'undercoat' for the project, or very low-res background texture for the 'world', which will hopefully get some interesting detail and more structure as the month progresses.

Session1 tag: https://github.com/hornc/nanogenmo2022-albatross/tree/session1

[hornc]

Session2: https://github.com/hornc/nanogenmo2022-albatross/tree/session2
I'm using git tags to track progress.

I've added some more sophistication to the reader, but reduced the word count to 11605 in the process. The sample gives a better idea of what I'm aiming for though. I've scrawled out a lot of narration examples and variations on paper, and I'm trying to code them all up so they occur automatically, and fit together semi-decently.

The reader sits down in a comfortable spot and holds up the book in front of them.
It is a strange book.
The reader opens the book to the first page and begins to read:

The first letter encountered on the page is a capital 'T', followed by a lowercase 'h', then an 'e'. The reader knows this word well. Next there is a space followed by a new word: 'r' - 'e' - 'a' - 'd' - 'e' - 'r' - The reader nods at the reference to themself. The next word is a verb: 'sits'. The reader remembers taking this action very recently. It just happened seconds ago. The next word is 'down'. The reader nods again, and continues to read. The next letter encountered is a lowercase 'i', then an 'n'. The reader knows this word well. Next there is a space followed by a new word: 'a' - The next letter encountered is a lowercase 'c', followed by an 'o', followed by an 'm', followed by an 'f', followed by an 'o', followed by an 'r', followed by a 't', followed by an 'a', followed by a 'b', followed by an 'l', then an 'e'. The next word is 'spot'. The reader nods again, and continues to read. The next word is a conjunction: 'and'. Next there is a space followed by a new word: 'h' - 'o' - 'l' - 'd' - 's' - The next letter encountered is a lowercase 'u', then a 'p'. The next word is 'the'. The reader knows this word well. The next letter encountered is a lowercase 'b', followed by an 'o', followed by an 'o', then a 'k'. , this book. Next there is a space followed by a new word: 'i' - 'n' - The reader knows this word well. The next letter encountered is a lowercase 'f', followed by an 'r', followed by an 'o', followed by an 'n', then a 't'. The next word is 'of'. The reader nods again, and continues to read. The next letter encountered is a lowercase 't', followed by an 'h', followed by an 'e', followed by an 'm', then there is a period, ending the current sentence.

I'm not using Tracery yet, and I realised I had wanted to make this completely deterministic, without any random or psuedo-random choices at all. All decisions should be made by the reader character based on how often or how recently they had done something, and I guess I'll have to model those factors, and that might not play neatly with Tracery. Not sure yet.

This session was a bit rushed, trying things out quickly. I think I need to impose some more structure on the text reading module before it gets out of hand.

[ikarth]

I really like the self-descriptive object part of this: I tend to think that one of the best measures of an artistically successful NaNoGenMo project (in my personal estimation) is having a strong concept paired with an implementation that conveys that concept to the reader. A self-described object fulfills the first half, so now you just need to actually make it.

[hornc]

Thanks @ikarth.

... an implementation that conveys that concept to the reader.

I totally agree with the importance of this. I had been reviewing one of my efforts from last year, and was a bit disappointed that it wasn't as good as I remembered. Technically I had implemented most of the things I intended, but none of that was particularly obvious from reading the output one year later. I had been too deep in the code, and making things happen technically. Hopefully I will do better this year! I'm a little worried I'm going to tie myself in knots with the self-reference, but it'll be an adventure :)

[hornc]

Session 3 and 4:
Added pdf output generation using markdown (with some Latex) > pdf with pandoc.
In order to get accurate page numbers and breaks I realised I'd have to generate intermediate pdfs, and am parsing them with pdftotext. Pandoc apparently doesn't model page breaks, and it'd be silly to try and re-implement a complex page break algorithm for the output chain, so I'm just generating the output, and reading it in again as needed.

To verify the accuracy I added some "tests" as part of the novel.

One chapter begins with the tests, and another chapter is a commentary on those tests:

CHAPTER 6
From flicking through the pages earlier, the reader had noticed that chapter 9
starts with a number of statements about specific letters located throughout
the book. The first was that “The first letter on page 1 of this book is ‘T’.”.
This appears to be correct. Then “The first letter on page 2 of this book is ‘s’.”.
Glancing at the facing page, this too is verified as correct. “The fifth letter of
the second line on page 7 is ‘x’.” Flicking forward 6 pages, the reader finds that
the fifth letter of the second line is indeed ‘x’.
...

CHAPTER 9
The first letter on page 1 of this book is ‘T’. The first letter on page 2 of this
book is ‘s’. The fifth letter of the second line on page 7 is ‘x’. The last letter
of the last line on page 7 is ‘n’. The second to last letter of the last line on
page 7 is ‘a’. The last letter of the second to last line on page 7 is ‘d’. The last
letter of the first line on page 7 is ‘d’. The tenth letter of the tenth line on page
10 is ‘CURRENTLY EMPTY’. The tenth letter of the tenth line on page 11 is
‘CURRENTLY EMPTY’. The twentieth letter of the twentieth line on page 4
is ‘r’. The twentieth letter of the twentieth line on page 5 is ‘h’. The twentieth
letter of the twentieth line on page 6 is ‘CURRENTLY EMPTY’. The twentieth
letter of the twentieth line on page 7 is ‘h’. The last letter of the last line on
page 8 is ‘e’.

The realisation here is that the 'world' has both a logical structure (chapters, and sentences) , and a physical structure (pages, and lines), and it can be navigated using either. I have direct control over the logical structure, but the physical structure is determined more by the pdf / layout tools.

Another thought: any musings I have on what this novel is supposed to be, and on the role of reader and narrator can be written into the text as I try to figure out what is supposed to be going on.

[verachell]

I love this wonderfully original idea. I can't wait to see how the final version turns out! The previews alone are fascinating.

[hornc]

I've neglected my dev diary -- I'm made some sporadic progress over the last week, but all sessions have felt a bit rushed.

Here is a summary of what's been happening, and roughly what's next:

session3: Nov 02

Got pdf output working with markdown to pdf using pandoc, which is needed early since the output pdf is an input to the story generation process, using pdftotext.
Very rough coding, trying things out quickly as I'm not sure how it will all sit together yet.
Spent a bit of time trying to calculate final paging, but it seems to difficult to guess the paging algorithm. Generating the pdf and figuring out pagination from the the text conversion seems to be the most reliable.

session4: Nov 03

Brief session; adding some plot elements to the 'plot.py' module.
Added some tests to track wordcount progress, and how the pre-composed 'plot' compares to the generated content, so I can tell where I am throughout the month.

session5: Nov 04 / 06

Two brief sessions exploring the same ideas spread over a couple of days. Structuring the code a fraction. Got a rough placeholder for a module to describe letter-shapes, and added a test to see how words were distributed over the multiple chapters.
This caused me to reduce the number of chapters from 12 to 10.
Added output/output.md to track the current markdown output as the code progresses.

session6: Nov 07

Fill in the letter-form describing module with a pytracery grammar.

Soon these too lose their meaning as symbols and there are just the shapes, printed in black ink on the paper: First there appears a long vertical taking up the full height of the line with a horizontal cross bar at the very top. This is followed by a long vertical taking up the full height of the line with a curved hook protudence to the lower right. Then a compact curve with a small closed loop where the end of the curve joins back onto itself forming an eye. Then there is a gap. Then a short vertical stroke with a small curved hook at the top-right. Then a compact curve with a small closed loop where the end of the curve joins back onto itself forming an eye. Then a loop closing in on itself in a clockwise direction. Then a closed curve attached to a long vertical stroke. Then a compact curve with a small closed loop where the end of the curve joins back onto itself forming an eye. Next, a short vertical stroke with a small curved hook at the top-right. Then there is a gap. Next, a long vertical stroke with a curved hook protudence to the lower right. Then a loop closing in on itself in a clockwise direction. Next, a curvaceous symbol, with symmetrical bends one above the other. Then there is a gap. Next, a curved stroke, descending below the line, with a dot above it. Then a small curved symbol open at the top with a slightly straighter right side. Then a curvaceous symbol, with symmetrical bends one above the other. Then a long vertical taking up the full height of the line, slightly curved at the bottom with a crossbar.

session7 Nov 08

Push to get my initial ideas for chapters 1, 2, and the beginning of chapter 3 all in place.

• Descriptions are still rough -- but they exist as placeholders
• Most references to other points in the book are currently hard coded, or fudged. This will have to improve.
• The first plot structure points are in place and set the scene (hopefully!)

Next steps: Work on the next bits of plot development. I have some ideas of the content, but am not sure quite where they will fit into the existing structure, or how best to link them.
I think I'll have to simply write them down as separate blocks, then put them in some sequence, and adjust exact lengths and positions, or prune based on what seems to work well.

Progress output:

1:	 4237	 84.74%
2:	 7792	155.84%
3:	 1407	 28.14%
4:	    0	  0.00%
5:	    0	  0.00%
6:	   99	  1.98%
7:	11851	237.02%
8:	    0	  0.00%
9:	  233	  4.66%
10:	    0	  0.00%
.Plot words: 661
Generated percentage: 97%

....
E       AssertionError: 51.234000% complete!
E       assert 25617 > 50000

[hornc]

Well, it's the last day of November and I'm not close to finishing my NaNoGenMo 2022 project to the level I had hoped.

I still like the idea, and want to do it justice.

From a little bit of self-reflection I think my usual problem with these projects is setting out to write in one month:

1. an entire new text generation framework (experimenting and testing it as I go, and then having to learn how to use it effectively all in the same month)
2. a 50k word novel using that framework

Observing projects that get completed, they tend to use existing tools and frameworks which the authors are generally familiar with to create a 50k word novel. The best ones tend to be where the creator understood the framework used well, and was able to push it to create something impressive, using their knowledge and skill.

Trying to create both (a new framework, and a decent novel) leads to resource conflict, and the two tasks compete with each other, leading to both feeling half done at the end.

I think that was my problem last year, I technically made a framework that hit all of (most of?) the design goals I had intended, but none of that was particularly obvious from the final novel output -- it would require a better write up and technical docs to explain. The final novel didn't stand up as well as I would have liked on its own.

On the one hand, focusing on 1. is clearly what I like doing, and it seems to be a reasonable part of what NaNoGenMo is about, but on the other, working solely on a framework without producing a decent output feels like it misses the point too. The output at the end of the month is supposed to be the 50k words (with code and dev diary (the most overlooked part?) as secondary outputs).

Trying to fit everything in one month is a recipe for a rushed framework, and a rushed output (i.e. mediocre output, regardless of how good the idea was -- the idea can suffer for being crammed into one month).

I am going to try an pull something together to get a finished product over the line, but it won't be the novel I intended to write, unfortunately.

[hornc]

Last day:

Rushed PR to get something in over the word count: hornc/nanogenmo2022-albatross#1

I ran out of time to do most of the things I wanted, after a promising start to the month.

I embraced the self-description and went with a description of a relatively standard over-ambitious and not-quite-there NaNoGenMo novel :

https://raw.githubusercontent.com/hornc/nanogenmo2022-albatross/lastday/output/unfinished.pdf

The last chapter is almost a work on its own... apologies for having to rush it.

I may still work on this to make myself a proper finished physical book and work on completing the self-descriptive model.

I hope someone enjoys the output, despite the rough edges.

Generating novels is hard.

[hugovk]

I like this, good work!

[verachell]

This is AWESOME! I love the novel with the final output and the original idea leading to it.