Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: fix content layer types #11527

Merged
merged 4 commits into from
Jul 23, 2024
Merged

chore: fix content layer types #11527

merged 4 commits into from
Jul 23, 2024

Conversation

ascorbic
Copy link
Contributor

@ascorbic ascorbic commented Jul 22, 2024

Creates a separate experimental_content type alongside experimental_data to allow the render() and rendered properties to be added to the types. Makes the data store types generic and fixes the collection definition types so that types are enforced better for manually-defined loaders.

Copy link

changeset-bot bot commented Jul 22, 2024

⚠️ No Changeset found

Latest commit: ea67f96

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@github-actions github-actions bot added the pkg: astro Related to the core `astro` package (scope) label Jul 22, 2024

This comment was marked as outdated.

@ascorbic ascorbic changed the title Add experimental_content type Fixes to content layer typegen Jul 22, 2024

This comment was marked as outdated.

@ascorbic ascorbic changed the title Fixes to content layer typegen chore: fix content layer types Jul 22, 2024
@ascorbic
Copy link
Contributor Author

!preview contentlayer

Copy link
Contributor

Snapshots have been released for the following packages:

  • astro@experimental--contentlayer
Publish Log
🦋  warn ===============================IMPORTANT!===============================
🦋  warn Packages will be released under the experimental--contentlayer tag
🦋  warn ----------------------------------------------------------------------
🦋  info npm info astro
🦋  info npm info @astrojs/prism
🦋  info npm info @astrojs/rss
🦋  info npm info create-astro
🦋  info npm info @astrojs/db
🦋  info npm info @astrojs/alpinejs
🦋  info npm info @astrojs/lit
🦋  info npm info @astrojs/markdoc
🦋  info npm info @astrojs/mdx
🦋  info npm info @astrojs/node
🦋  info npm info @astrojs/partytown
🦋  info npm info @astrojs/preact
🦋  info npm info @astrojs/react
🦋  info npm info @astrojs/sitemap
🦋  info npm info @astrojs/solid-js
🦋  info npm info @astrojs/svelte
🦋  info npm info @astrojs/tailwind
🦋  info npm info @astrojs/vercel
🦋  info npm info @astrojs/vue
🦋  info npm info @astrojs/web-vitals
🦋  info npm info @astrojs/internal-helpers
🦋  info npm info @astrojs/markdown-remark
🦋  info npm info @astrojs/studio
🦋  info npm info @astrojs/telemetry
🦋  info npm info @astrojs/underscore-redirects
🦋  info npm info @astrojs/upgrade
🦋  info astro is being published because our local version (0.0.0-contentlayer-20240722131255) has not been published on npm
🦋  warn @astrojs/prism is not being published because version 3.1.0 is already published on npm
🦋  warn @astrojs/rss is not being published because version 4.0.7 is already published on npm
🦋  warn create-astro is not being published because version 4.8.0 is already published on npm
🦋  warn @astrojs/db is not being published because version 0.12.0 is already published on npm
🦋  warn @astrojs/alpinejs is not being published because version 0.4.0 is already published on npm
🦋  warn @astrojs/lit is not being published because version 4.3.0 is already published on npm
🦋  warn @astrojs/markdoc is not being published because version 0.11.3 is already published on npm
🦋  warn @astrojs/mdx is not being published because version 3.1.3 is already published on npm
🦋  warn @astrojs/node is not being published because version 8.3.2 is already published on npm
🦋  warn @astrojs/partytown is not being published because version 2.1.1 is already published on npm
🦋  warn @astrojs/preact is not being published because version 3.5.1 is already published on npm
🦋  warn @astrojs/react is not being published because version 3.6.0 is already published on npm
🦋  warn @astrojs/sitemap is not being published because version 3.1.6 is already published on npm
🦋  warn @astrojs/solid-js is not being published because version 4.4.0 is already published on npm
🦋  warn @astrojs/svelte is not being published because version 5.7.0 is already published on npm
🦋  warn @astrojs/tailwind is not being published because version 5.1.0 is already published on npm
🦋  warn @astrojs/vercel is not being published because version 7.7.2 is already published on npm
🦋  warn @astrojs/vue is not being published because version 4.5.0 is already published on npm
🦋  warn @astrojs/web-vitals is not being published because version 1.0.0 is already published on npm
🦋  warn @astrojs/internal-helpers is not being published because version 0.4.1 is already published on npm
🦋  warn @astrojs/markdown-remark is not being published because version 5.2.0 is already published on npm
🦋  warn @astrojs/studio is not being published because version 0.1.1 is already published on npm
🦋  warn @astrojs/telemetry is not being published because version 3.1.0 is already published on npm
🦋  warn @astrojs/underscore-redirects is not being published because version 0.3.4 is already published on npm
🦋  warn @astrojs/upgrade is not being published because version 0.3.1 is already published on npm
🦋  info Publishing "astro" at "0.0.0-contentlayer-20240722131255"
🦋  success packages published successfully:
🦋  [email protected]
🦋  Creating git tag...
🦋  New tag:  [email protected]
Build Log

> [email protected] build /home/runner/work/astro/astro
> turbo run build --filter=astro --filter=create-astro --filter="@astrojs/*" --filter="@benchmark/*"

• Packages in scope: @astrojs/alpinejs, @astrojs/cloudflare, @astrojs/db, @astrojs/internal-helpers, @astrojs/lit, @astrojs/markdoc, @astrojs/markdown-remark, @astrojs/mdx, @astrojs/netlify, @astrojs/node, @astrojs/partytown, @astrojs/preact, @astrojs/prism, @astrojs/react, @astrojs/rss, @astrojs/sitemap, @astrojs/solid-js, @astrojs/studio, @astrojs/svelte, @astrojs/tailwind, @astrojs/telemetry, @astrojs/underscore-redirects, @astrojs/upgrade, @astrojs/vercel, @astrojs/vue, @astrojs/web-vitals, @benchmark/timer, astro, create-astro
• Running build in 29 packages
• Remote caching enabled
::group::@astrojs/prism:build
cache hit, suppressing logs 8a40705ceb3325a8
::endgroup::
::group::@astrojs/upgrade:build
cache hit, suppressing logs 2b0b579e27f20608
::endgroup::
::group::@astrojs/internal-helpers:build
cache hit, suppressing logs 1de66db24dbb21b4
::endgroup::
::group::create-astro:build
cache hit, suppressing logs 16143e853a29e0a4
::endgroup::
::group::@astrojs/telemetry:build
cache hit, suppressing logs 4d6b9ab62f27eef8
::endgroup::
::group::@astrojs/markdown-remark:build
cache hit, suppressing logs f41824c56ce07c90
::endgroup::
::group::astro:build
cache miss, executing fa0d2d7ef9780a38

> [email protected] build /home/runner/work/astro/astro/packages/astro
> pnpm run prebuild && astro-scripts build "src/**/*.{ts,js}" && tsc && pnpm run postbuild


> [email protected] prebuild /home/runner/work/astro/astro/packages/astro
> astro-scripts prebuild --to-string "src/runtime/server/astro-island.ts" "src/runtime/client/{idle,load,media,only,visible}.ts"


> [email protected] postbuild /home/runner/work/astro/astro/packages/astro
> astro-scripts copy "src/**/*.astro" && astro-scripts copy "src/**/*.wasm"

::endgroup::
::group::@astrojs/studio:build
cache miss, executing 0fcc2adc8ed72a66

> @astrojs/[email protected] build /home/runner/work/astro/astro/packages/studio
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@benchmark/timer:build
cache miss, executing dfc5568977629113

> @benchmark/[email protected] build /home/runner/work/astro/astro/benchmark/packages/timer
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/react:build
cache miss, executing 296b2cb662b9b86b

> @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/react
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/vercel:build
cache miss, executing cfbde04bb6cf4bd8

> @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/vercel
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/underscore-redirects:build
cache miss, executing 3cb48b7bfcbe9e1c

> @astrojs/[email protected] build /home/runner/work/astro/astro/packages/underscore-redirects
> astro-scripts build "src/**/*.ts" && tsc -p tsconfig.json


> @astrojs/[email protected] postbuild /home/runner/work/astro/astro/packages/underscore-redirects
> astro-scripts copy "src/**/*.js"

::endgroup::
::group::@astrojs/partytown:build
cache miss, executing dfda2faf80e2ebc8

> @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/partytown
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/markdoc:build
cache miss, executing e344746a4d0ffbae

> @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/markdoc
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/preact:build
cache miss, executing 829ec376a2060f58

> @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/preact
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/svelte:build
cache miss, executing e3ddaef4bdebe8ff

> @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/svelte
> astro-scripts build "src/index.ts" && astro-scripts build "src/editor.cts" --force-cjs --no-clean-dist && tsc

::endgroup::
::group::@astrojs/solid-js:build
cache miss, executing cf33effdf226abda

> @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/solid
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/lit:build
cache miss, executing 622fd080d4de710f

> @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/lit
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/rss:build
cache miss, executing b581f03bf44fc798

> @astrojs/[email protected] build /home/runner/work/astro/astro/packages/astro-rss
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/alpinejs:build
cache miss, executing c093990958cb4fdd

> @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/alpinejs
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/tailwind:build
cache miss, executing a3f071c36a6608b7

> @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/tailwind
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/node:build
cache miss, executing 355dc4a0ea184477

> @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/node
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/vue:build
cache miss, executing 9f136a76355641ce

> @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/vue
> astro-scripts build "src/index.ts" && astro-scripts build "src/editor.cts" --force-cjs --no-clean-dist && tsc

::endgroup::
::group::@astrojs/mdx:build
cache miss, executing f20c3fbe7d8cefb3

> @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/mdx
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/sitemap:build
cache miss, executing ffb662cf1b3a394b

> @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/sitemap
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::
::group::@astrojs/db:build
cache miss, executing 4ed6800f5b290117

> @astrojs/[email protected] build /home/runner/work/astro/astro/packages/db
> astro-scripts build "src/**/*.ts" && tsc && pnpm types:virtual


> @astrojs/[email protected] types:virtual /home/runner/work/astro/astro/packages/db
> tsc -p ./tsconfig.virtual.json

::endgroup::
::group::@astrojs/web-vitals:build
cache miss, executing d04b102fe0ad6142

> @astrojs/[email protected] build /home/runner/work/astro/astro/packages/integrations/web-vitals
> astro-scripts build "src/**/*.ts" && tsc

::endgroup::

 Tasks:    27 successful, 27 total
Cached:    6 cached, 27 total
  Time:    50.221s 

@ascorbic ascorbic marked this pull request as ready for review July 22, 2024 13:20
@withastro withastro deleted a comment from github-actions bot Jul 22, 2024
@ascorbic ascorbic merged commit 4230c11 into content-layer Jul 23, 2024
13 checks passed
@ascorbic ascorbic deleted the add-exp-content branch July 23, 2024 09:45
ascorbic added a commit that referenced this pull request Aug 14, 2024
* Empty commit

* Changeset

* feat: add Content Layer loader (#11334)

* wip

* wip

* wip

* Update demo

* Add meta

* wip

* Add file loader

* Add schema validation

* Remove log

* Changeset

* Format

* Lockfile

* Fix type

* Handle loading for data store JSON

* Use rollup util to import JSON

* Fix types

* Format

* Add tests

* Changes from review

* fix: sync content layer in dev (#11365)

* wip

* wip

* wip

* Update demo

* Add meta

* wip

* Add file loader

* Add schema validation

* Remove log

* Changeset

* Format

* Lockfile

* Fix type

* Handle loading for data store JSON

* Use rollup util to import JSON

* Fix types

* Format

* Add tests

* Changes from review

* Sync content layer in dev

* feat: add typegen for loaders (#11358)

* fix: watch for content layer changes (#11371)

* fix: watch for content layer changes

* Add test

* feat: adds simple loader (#11386)

* wip

* Add simple loader

* Fix type guard

* Tighten loader schema

* Add loader function to type

* Reinstall vitest

* feat: add glob loader (#11398)

* feat: add glob loader

* Enable watching and fix paths

* Store the full entry object, not just data

* Add generateId support

* Fix test

* Rename loaders to sync

* Refacctor imports

* Use getEntry

* Format

* Fix import

* Remove type from output

* Windows path

* Add test for absolute path

* Update lockfile

* Debugging windows

* Allow file URL for base dir

* Reset time limit

* feat: add markdown rendering to content layer (#11440)

* feat: add glob loader

* Enable watching and fix paths

* Store the full entry object, not just data

* Add generateId support

* Fix test

* Rename loaders to sync

* Refacctor imports

* Use getEntry

* Format

* Fix import

* Remove type from output

* Windows path

* Add test for absolute path

* Update lockfile

* Debugging windows

* Allow file URL for base dir

* Reset time limit

* wip: add markdown rendering to content layer

* use cached entries

* CLean up types

* Instrument more of the build

* Add digest helper

* Add comments

* Make image extraction work

* feat: image support for content layer (#11469)

* wip

* wip

* Add image to benchmark

* Stub assets if missing

* Resolve assets in data

* Ignore virtual module

* Format

* rm log

* Handle images when using cached data

* Fix CCC

* Add a comment

* Changes from review

* Format

* Use relative paths for asset files

* Pass all md props to getImage

* Ensure dotastro dir exists

* Fix tests

* Changes from review

* Don't use temp array in getcollection

* Add error handling

* Format

* Handle paths that are already relative

* Dedupe sync runs

* Fix syncing in dev

* Changes from review

* Windows paths ftw

* feat(content-layer): support references in content layer (#11494)

* Support references in content layer

* Fix utf8 rendering

* Warn for invalid entries

* Fix test

* lol windows paths

* Remove assertion

* chore: fix content layer types (#11527)

* Add experimental_content type

* Fix import

* Make data store methods generic

* fix loader types

* Lockfile

* Clean content layer with `--force` (#11541)

* Clearn content layer with `--force`

* Add tests

* Document --force flag

* Fixes to content layer render types (#11558)

* Lockfile

* feat: use devalue to serialize content layer data (#11562)

* feat: use devalue to serialize content layer data

* Fix import

* Use devalue stringify

* Unused import

* Propagate error messages correctly

* Support --force flag in sync and dev (#11581)

* Support --force flag in sync and dev

* Fix test

* Separate render function and merge content layer types (#11579)

* Separate render function and merge content layer types

* Changes from review

* fix: clear content layer cache if config has changed (#11591)

* fix: clear content layer cache if config has changed

* Add test

* Watch config

* Change from review

* fix: skip glob files in content dir (#11622)

* fix: skip glob files in content dir

* Changes from review

* Log pattern

* Refactor content layer into shared instance (#11625)

* Refactor content layer into shared instance

* Clean up when testing

* Handle cleanup

* fix: support filters in content layer getCollection (#11631)

* Throw when using deprecated getEntryByX functions with content layer (#11637)

* Updates to content layer types and jsdocs (#11643)

* Add hot key to reload content layer (#11626)

* Add hot key to reload content layer

* Fix filename

* Remove cli message

* Update example

* Change key to "s"

* feat: handle simple mdx rendering (#11633)

* feat: handle simple mdx rendering

* cleanup

* feedback

* fix regression

* remove log

* flip condition

* update tests

* log collections to understand the error

* let's try this alternative

* try parallel test to understand the issue

* chore: use a new fixture to fix tests

* rebase and docs

* fix regressions

* remove old code

* address feedback

* rename param

* log error

* rebase

* chore: try a different cache dir to solve the error test

* fix invalidation of the module when there's no store available

* address suggestion

* run formatter

* update lock file

* Lint

* Add experimental content layer flag (#11652)

* Add experimental content layer flag

* Syntax and format

* Aside

* Format

* Reset content config between runs

* Update fixture

* Update terminology

* Lint

* wut

* Normalize render function return value (#11663)

* Add markdoc support to content layer (#11664)

* Add markdoc support to content layer

* Switch test to cheerio

* Update benchmarks

* update lock file

* Update content layer flag docs (#11682)

* Update content layer flag docs

* Apply suggestions from code review

Co-authored-by: Sarah Rainsberger <[email protected]>

* More markdoc

* Typo

* Apply suggestions from code review

Co-authored-by: Sarah Rainsberger <[email protected]>

* Update

---------

Co-authored-by: Sarah Rainsberger <[email protected]>

* Add changeset for content layer experimental release (#11644)

* Add changeset for content layer experimental release

* Update changeset

* Update .changeset/smooth-chicken-wash.md

Co-authored-by: Sarah Rainsberger <[email protected]>

---------

Co-authored-by: Sarah Rainsberger <[email protected]>

* feat: injectTypes (#11551)

* feat: make inline config 1st arg

* fix: run config done in sync

* feat: start working on injectTypes

* feat: write files

* feat: adapt core features

* feat: migrate db to injectTypes

* feat: special db handling

* feat: update settings instead of workarounds

* fix: create dotAstroDir

* feat: refactor sync tests

* fix: path

* fix: paths

* chore: add comments

* feat: overwrite content file if exists

* chore: remove unused db env related code

* feat: use dotAstroDir for settings

* chore: simplify astro env sync

* feat: use dotAstroDir for preferences

* feat: handle db in integration api

* chore: reorganize

* feat: format

* feat: add test

* Discard changes to examples/basics/astro.config.mjs

* Discard changes to examples/basics/package.json

* Discard changes to pnpm-lock.yaml

* chore: remove test files

* feat: update examples dts

* fix: dts

* chore: changesets

* fix: indentation

* Apply suggestions from code review

Co-authored-by: Chris Swithinbank <[email protected]>

* Apply suggestions from code review

Co-authored-by: Chris Swithinbank <[email protected]>

* chore: format

* Update packages/astro/src/integrations/hooks.ts

* Update .changeset/mean-horses-kiss.md

* feat: remove formatting

* feat: handle fs errors

* feat: remove astro:db special path handling

* feat: add fs error

* Update packages/astro/src/content/types-generator.ts

* Update .changeset/mean-horses-kiss.md

* Update errors-data.ts

* Update .changeset/mean-horses-kiss.md

Co-authored-by: Sarah Rainsberger <[email protected]>

* Update .changeset/mean-horses-kiss.md

Co-authored-by: Sarah Rainsberger <[email protected]>

---------

Co-authored-by: Chris Swithinbank <[email protected]>
Co-authored-by: Sarah Rainsberger <[email protected]>
Co-authored-by: Erika <[email protected]>

* Add file generation and flag for content intellisense (#11639)

* feat: add type to infer input type of collection

* refactor:

* feat: generate json schema for content too

* feat: generate a manifest of all the collections

* refactor: unnecessary type

* fix: only add content collections to manifest

* chore: changeset

* fix: generate file URLs

* fix: flag it properly

* fix: save in lower case

* docs: add jsdoc to experimental option

* nit: move function out

* fix: match vscode flag name

* Update packages/astro/src/@types/astro.ts

Co-authored-by: Sarah Rainsberger <[email protected]>

* Update packages/astro/src/@types/astro.ts

Co-authored-by: Sarah Rainsberger <[email protected]>

* Update serious-pumas-run.md

* test: add tests

* Add content layer support

* Apply suggestions from code review

* fix: test

* Update .changeset/serious-pumas-run.md

Co-authored-by: Sarah Rainsberger <[email protected]>

* Apply suggestions from code review

* Remove check for json

---------

Co-authored-by: Matt Kane <[email protected]>
Co-authored-by: Sarah Rainsberger <[email protected]>

* nit: use same filesystem error as injectTypes

* fix: code component was missing support for meta string (#11605)

* fix: code component was missing support for meta string

Fixed #11604

* Create odd-buttons-pay.md

* <Code>: add reference link for meta prop

* Apply suggestions from code review

Co-authored-by: Sarah Rainsberger <[email protected]>

* Update .changeset/odd-buttons-pay.md

* Update .changeset/odd-buttons-pay.md

Co-authored-by: Sarah Rainsberger <[email protected]>

---------

Co-authored-by: Erika <[email protected]>
Co-authored-by: Sarah Rainsberger <[email protected]>
Co-authored-by: Emanuele Stoppa <[email protected]>

* Deprecates exporting prerender with dynamic values (#11657)

* wip

* done i think

* Add changeset

* Use hook instead

* Reorder hooks [skip ci]

* Update .changeset/eleven-pens-glow.md

Co-authored-by: Sarah Rainsberger <[email protected]>

* Fix run

* Fix link

* Add link

Co-authored-by: Sarah Rainsberger <[email protected]>

* More accurate migration [skip ci]

---------

Co-authored-by: Sarah Rainsberger <[email protected]>
Co-authored-by: Sarah Rainsberger <[email protected]>

* Use node parseArgs instead of yargs-parser and arg (#11645)

* wip

* done

* Add changeset

* Format

* Update

* Fix houston

* Fix test

* Fix test

* [ci] format

* resolve conflict

---------

Co-authored-by: Emanuele Stoppa <[email protected]>
Co-authored-by: Sarah Rainsberger <[email protected]>
Co-authored-by: Florian Lefebvre <[email protected]>
Co-authored-by: Chris Swithinbank <[email protected]>
Co-authored-by: Erika <[email protected]>
Co-authored-by: Julien Cayzac <[email protected]>
Co-authored-by: Bjorn Lu <[email protected]>
Co-authored-by: Sarah Rainsberger <[email protected]>
Co-authored-by: Bjorn Lu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pkg: astro Related to the core `astro` package (scope)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants