Issues with East Asian Language Tags with Hyperlinks and Quotes in DOCX Output #9909

TomBener · 2024-06-24T00:56:25Z

As have discussed in #9817, there are issues with East Asian Language font hints, specifically:

---
title: Test eastAsia document
reference-section-title: References
csl: test.csl
references:
- author:
  - family: Ma
    given: Jinguan
    suffix: 马經觀
  container-title: 國聞周報
  id: majingguan1949
  issued: 1949
  original-title: China's tomorrow and the day after tomorrow
  page: 1-2
  publisher-place: Shanghai
  title: 中國的明天和後天
  type: article-newspaper
  doi: 10.1234/5678
- author:
  - family: Ma
    given: Jinguan
    suffix: 马經觀
  container-title: 國聞周報
  id: majingguan1949a
  issued: 1949
  original-title: China's tomorrow and the day after tomorrow
  page: 1
  publisher-place: Shanghai
  title: ‘中國的明天’和後天
  type: article-newspaper
- author:
  - family: He
    given: Fangchuan
    suffix: 何芳川
  container-title: 北京大学学报
  id: hefangchuan1998
  issue: 6
  issued: 1998
  original-title: On "Hua-Yi order"
  page: 30-45
  title: "\"华夷秩序\"论"
  type: article-journal
---

This is a test document for CJK in docx with Pandoc 3.2 later
[@majingguan1949; @majingguan1949a; @hefangchuan1998].

中文“汉字”测试。

The first citation contained a DOI and was rendered as a hyperlink with test.csl, but the quotation mark was not double-width for Chinese texts
The title of the third citation contains a quote at the first place, leading to the open quotation mark not inclosed with East Asian font hints.

Related XML

<w:body>
  <w:p>
    <w:pPr>
      <w:pStyle w:val="Title" />
    </w:pPr>
    <w:r>
      <w:t xml:space="preserve">Test eastAsia document</w:t>
    </w:r>
  </w:p>
  <w:p>
    <w:pPr>
      <w:pStyle w:val="FirstParagraph" />
    </w:pPr>
    <w:r>
      <w:t xml:space="preserve">This is a test document for CJK in docx with Pandoc 3.2 later</w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve"></w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve">(Ma, 1949b, 1949a; He, 1998)</w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve">.</w:t>
    </w:r>
  </w:p>
  <w:p>
    <w:pPr>
      <w:pStyle w:val="BodyText" />
    </w:pPr>
    <w:r>
      <w:rPr>
        <w:rFonts w:hint="eastAsia" />
      </w:rPr>
      <w:t xml:space="preserve">中文“汉字”测试。</w:t>
    </w:r>
  </w:p>
  <w:bookmarkStart w:id="25" w:name="bibliography" />
  <w:p>
    <w:pPr>
      <w:pStyle w:val="Heading1" />
    </w:pPr>
    <w:r>
      <w:t xml:space="preserve">References</w:t>
    </w:r>
  </w:p>
  <w:bookmarkStart w:id="24" w:name="refs" />
  <w:bookmarkStart w:id="20" w:name="ref-hefangchuan1998" />
  <w:p>
    <w:pPr>
      <w:pStyle w:val="Bibliography" />
    </w:pPr>
    <w:r>
      <w:t xml:space="preserve">HE FANGCHUAN </w:t>
    </w:r>
    <w:r>
      <w:rPr>
        <w:rFonts w:hint="eastAsia" />
      </w:rPr>
      <w:t xml:space="preserve">何芳川</w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve"> (1998)</w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve"></w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve">“</w:t>
    </w:r>
    <w:r>
      <w:rPr>
        <w:rFonts w:hint="eastAsia" />
      </w:rPr>
      <w:t xml:space="preserve">‘华夷秩序’</w:t>
    </w:r>
    <w:r>
      <w:rPr>
        <w:rFonts w:hint="eastAsia" />
      </w:rPr>
      <w:t xml:space="preserve">论”</w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve"></w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve">(On</w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve"></w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve">“Hua-Yi order”</w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve">). </w:t>
    </w:r>
    <w:r>
      <w:rPr>
        <w:rFonts w:hint="eastAsia" />
      </w:rPr>
      <w:t xml:space="preserve">北京大学学报</w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve"> 6: 30–45.</w:t>
    </w:r>
  </w:p>
  <w:bookmarkEnd w:id="20" />
  <w:bookmarkStart w:id="21" w:name="ref-majingguan1949a" />
  <w:p>
    <w:pPr>
      <w:pStyle w:val="Bibliography" />
    </w:pPr>
    <w:r>
      <w:t xml:space="preserve">MA JINGUAN </w:t>
    </w:r>
    <w:r>
      <w:rPr>
        <w:rFonts w:hint="eastAsia" />
      </w:rPr>
      <w:t xml:space="preserve">马經觀</w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve"> (1949a)</w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve"></w:t>
    </w:r>
    <w:r>
      <w:rPr>
        <w:rFonts w:hint="eastAsia" />
      </w:rPr>
      <w:t xml:space="preserve">“’中國的明天’和後天”</w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve"></w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve">(China’s tomorrow and the day after tomorrow). </w:t>
    </w:r>
    <w:r>
      <w:rPr>
        <w:rFonts w:hint="eastAsia" />
      </w:rPr>
      <w:t xml:space="preserve">國聞周報:</w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve"> 1.</w:t>
    </w:r>
  </w:p>
  <w:bookmarkEnd w:id="21" />
  <w:bookmarkStart w:id="23" w:name="ref-majingguan1949" />
  <w:p>
    <w:pPr>
      <w:pStyle w:val="Bibliography" />
    </w:pPr>
    <w:r>
      <w:t xml:space="preserve">MA JINGUAN </w:t>
    </w:r>
    <w:r>
      <w:rPr>
        <w:rFonts w:hint="eastAsia" />
      </w:rPr>
      <w:t xml:space="preserve">马經觀</w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve"> (1949b)</w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve"></w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve">“</w:t>
    </w:r>
    <w:hyperlink r:id="rId22">
      <w:r>
        <w:rPr>
          <w:rStyle w:val="Hyperlink" />
          <w:rFonts w:hint="eastAsia" />
        </w:rPr>
        <w:t xml:space="preserve">中國的明天和後天</w:t>
      </w:r>
    </w:hyperlink>
    <w:r>
      <w:t xml:space="preserve">”</w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve"></w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve">(China’s tomorrow and the day after tomorrow). </w:t>
    </w:r>
    <w:r>
      <w:rPr>
        <w:rFonts w:hint="eastAsia" />
      </w:rPr>
      <w:t xml:space="preserve">國聞周報:</w:t>
    </w:r>
    <w:r>
      <w:t xml:space="preserve"> 1–2.</w:t>
    </w:r>
  </w:p>
  <w:bookmarkEnd w:id="23" />
  <w:bookmarkEnd w:id="24" />
  <w:bookmarkEnd w:id="25" />
  <w:sectPr w:rsidR="00D3414C">
    <w:pgSz w:h="16840" w:w="11900" />
    <w:pgMar w:bottom="1440" w:footer="720" w:gutter="0" w:header="720" w:left="1440" w:right="1440" w:top="1440" />
    <w:cols w:space="720" />
    <w:docGrid w:linePitch="360" />
  </w:sectPr>
</w:body>

Please update to fix the issue. Thanks for your great work!

TomBener · 2024-06-26T03:04:04Z

Update: The East Asian language font hints has no issue with main body

---
title: Test eastAsia document
reference-section-title: References
csl: test.csl
references:
- author:
  - family: Ma
    given: Jinguan
    suffix: 马經觀
  container-title: 國聞周報
  id: majingguan1949
  issued: 1949
  original-title: China's tomorrow and the day after tomorrow
  page: 1-2
  publisher-place: Shanghai
  title: 中國的明天和後天
  type: article-newspaper
  doi: 10.1234/5678
- author:
  - family: Ma
    given: Jinguan
    suffix: 马經觀
  container-title: 國聞周報
  id: majingguan1949a
  issued: 1949
  original-title: China's tomorrow and the day after tomorrow
  page: 1
  publisher-place: Shanghai
  title: ‘中國的明天’和後天
  type: article-newspaper
- author:
  - family: He
    given: Fangchuan
    suffix: 何芳川
  container-title: 北京大学学报
  id: hefangchuan1998
  issue: 6
  issued: 1998
  original-title: On "Hua-Yi order"
  page: 30-45
  title: "\"华夷秩序\"论"
  type: article-journal
---

This is a test document for CJK in docx with Pandoc 3.2 later
[@majingguan1949; @majingguan1949a; @hefangchuan1998].

中文“汉字”测试 and "English text" test。
中文“‘汉字’”测试 and "English text" test。
中文“[汉字](https://pandoc.org/custom-writers.html)”测试and "English text" test.

The generated result in document.xml:

<w:p>
  <w:pPr>
    <w:pStyle w:val="Title" />
  </w:pPr>
  <w:r>
    <w:t xml:space="preserve">Test eastAsia document</w:t>
  </w:r>
</w:p>
<w:p>
  <w:pPr>
    <w:pStyle w:val="FirstParagraph" />
  </w:pPr>
  <w:r>
    <w:t xml:space="preserve">This is a test document for CJK in docx with Pandoc 3.2 later</w:t>
  </w:r>
  <w:r>
    <w:t xml:space="preserve"></w:t>
  </w:r>
  <w:r>
    <w:t xml:space="preserve">(Ma, 1949b, 1949a; He, 1998)</w:t>
  </w:r>
  <w:r>
    <w:t xml:space="preserve">.</w:t>
  </w:r>
</w:p>
<w:p>
  <w:pPr>
    <w:pStyle w:val="BodyText" />
  </w:pPr>
  <w:r>
    <w:rPr>
      <w:rFonts w:hint="eastAsia" />
    </w:rPr>
    <w:t xml:space="preserve">中文“汉字”测试</w:t>
  </w:r>
  <w:r>
    <w:t xml:space="preserve"> and</w:t>
  </w:r>
  <w:r>
    <w:t xml:space="preserve"></w:t>
  </w:r>
  <w:r>
    <w:t xml:space="preserve">“English text”</w:t>
  </w:r>
  <w:r>
    <w:t xml:space="preserve"></w:t>
  </w:r>
  <w:r>
    <w:t xml:space="preserve">test。</w:t>
  </w:r>
  <w:r>
    <w:t xml:space="preserve"></w:t>
  </w:r>
  <w:r>
    <w:rPr>
      <w:rFonts w:hint="eastAsia" />
    </w:rPr>
    <w:t xml:space="preserve">中文“</w:t>
  </w:r>
  <w:r>
    <w:rPr>
      <w:rFonts w:hint="eastAsia" />
    </w:rPr>
    <w:t xml:space="preserve">‘汉字’</w:t>
  </w:r>
  <w:r>
    <w:rPr>
      <w:rFonts w:hint="eastAsia" />
    </w:rPr>
    <w:t xml:space="preserve">”测试</w:t>
  </w:r>
  <w:r>
    <w:t xml:space="preserve"> and</w:t>
  </w:r>
  <w:r>
    <w:t xml:space="preserve"></w:t>
  </w:r>
  <w:r>
    <w:t xml:space="preserve">“English text”</w:t>
  </w:r>
  <w:r>
    <w:t xml:space="preserve"></w:t>
  </w:r>
  <w:r>
    <w:t xml:space="preserve">test。</w:t>
  </w:r>
  <w:r>
    <w:t xml:space="preserve"></w:t>
  </w:r>
  <w:r>
    <w:rPr>
      <w:rFonts w:hint="eastAsia" />
    </w:rPr>
    <w:t xml:space="preserve">中文“</w:t>
  </w:r>
  <w:hyperlink r:id="rId20">
    <w:r>
      <w:rPr>
        <w:rStyle w:val="Hyperlink" />
        <w:rFonts w:hint="eastAsia" />
      </w:rPr>
      <w:t xml:space="preserve">汉字</w:t>
    </w:r>
  </w:hyperlink>
  <w:r>
    <w:rPr>
      <w:rFonts w:hint="eastAsia" />
    </w:rPr>
    <w:t xml:space="preserve">”测试and</w:t>
  </w:r>
  <w:r>
    <w:t xml:space="preserve"></w:t>
  </w:r>
  <w:r>
    <w:t xml:space="preserve">“English text”</w:t>
  </w:r>
  <w:r>
    <w:t xml:space="preserve"></w:t>
  </w:r>
  <w:r>
    <w:t xml:space="preserve">test.</w:t>
  </w:r>
</w:p>

We can notice that all text fragments with Chinese were enclosed with <w:rFonts w:hint="eastAsia" />, which worked as expected.

So the issue only happen in the bibliography generated by citeproc. Given that Chinese bibliographies generally don't need to enclose the title with quotation marks, and the hyperlink is also not preferred, so I think the issue is not a big deal. If one want to correct the Chinese quotation mark font hints in the bibliography, I believe a Lua filter can work with that.

jgm · 2024-06-26T04:14:45Z

The issue will arise (even in the main body) whenever you have a structure like:
QuotationMark, Link (Chinese text), Quotation Mark
because in this case the Chinese text and the quotation marks won't be in the same docx "run."

It would not arise for
Link (QuotationMark, Chinese text, Quotation Mark)

TomBener added the bug label Jun 24, 2024

TomBener changed the title ~~Issues with East Asian Language Tags with Hyperlinks in DOCX Output~~ Issues with East Asian Language Tags with Hyperlinks and Quotes in DOCX Output Jun 24, 2024

TomBener closed this as completed Jun 26, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issues with East Asian Language Tags with Hyperlinks and Quotes in DOCX Output #9909

Issues with East Asian Language Tags with Hyperlinks and Quotes in DOCX Output #9909

TomBener commented Jun 24, 2024 •

edited

Loading

TomBener commented Jun 26, 2024

jgm commented Jun 26, 2024

Issues with East Asian Language Tags with Hyperlinks and Quotes in DOCX Output #9909

Issues with East Asian Language Tags with Hyperlinks and Quotes in DOCX Output #9909

Comments

TomBener commented Jun 24, 2024 • edited Loading

TomBener commented Jun 26, 2024

Update: The East Asian language font hints has no issue with main body

jgm commented Jun 26, 2024

TomBener commented Jun 24, 2024 •

edited

Loading