Repository navigation
Regression: since 0.28.0, marked filter does not render markdown in HTML blocks #310
Description
I first hit this in angular/material#11881, which ended up with the project pinning the dependency to 0.27.5 and not being able to upgrade since then. At that time, early 2020, I was quite unfamiliar with Marked and how it was used in dgeni-packages. So I was hoping that someone else would report this and it would get fixed. Unfortunately, that hasn't happened.
I have dug deeper recently in angular/angular.js#17141 and this seems to be related to 16ceb9c from PR #281, released in 0.28.0.
There is a breaking changes note:
There are a few relevant breaking changes with this latest version of `marked`.
This only affects usage of the `renderMarkdown()` service and the `marked`
nunjucks filter. Take a look through the
[marked release notes](https://github.com/markedjs/marked/releases) and
check if this affects you.
renderMarkdown()is not used by AngularJS or AngularJS Material.- They both use
require('dgeni-packages/nunjucks')and themarkedfilter, i.e.{$ doc.description | marked $}
I read through the Marked release notes, but being unfamiliar with the library, I didn't really see anything related or of value to fixing this.
I read through this project's README.md but didn't see any references on how to use the marked filter.
It looks like the code for the filter is here:
dgeni-packages/nunjucks/rendering/filters/marked.js
Lines 1 to 13 in 0fe303e
| /** | |
| * @dgRenderFilter marked | |
| * @description Convert the value, as markdown, into HTML using the marked library | |
| */ | |
| module.exports = function markedNunjucksFilter(renderMarkdown) { | |
| return { | |
| name: 'marked', | |
| process(str) { | |
| const output = str && renderMarkdown(str); | |
| return output; | |
| } | |
| }; | |
| }; |
I don't see how a change to any
marked APIs could cause issues with this simple filter.
Except that it seems to call renderMarkdown()
dgeni-packages/nunjucks/services/renderMarkdown.js
Lines 1 to 28 in 0fe303e
| const marked = require('marked'); | |
| /** | |
| * @dgService renderMarkdown | |
| * @description | |
| * Render the markdown in the given string as HTML. | |
| */ | |
| module.exports = function renderMarkdown(trimIndentation) { | |
| const renderer = new marked.Renderer(); | |
| // Remove the leading whitespace from the code block before it gets to the | |
| // markdown code render function | |
| renderer.code = (code, string, language) => { | |
| const trimmedCode = trimIndentation(code); | |
| let renderedCode = marked.Renderer.prototype.code.call(renderer, trimmedCode, string, language); | |
| // Bug in marked - forgets to add a final newline sometimes | |
| if ( !/\n$/.test(renderedCode) ) { | |
| renderedCode += '\n'; | |
| } | |
| return renderedCode; | |
| }; | |
| return content => marked(content, { renderer: renderer }); | |
| }; |
Which does import
marked and use its APIs.
There is perhaps something that I'm missing here where this can be fixed in the AngularJS and AngularJS Material projects, but from what I can tell, with limited knowledge of this library, the fix needs to be made here.
Activity
Splaktar commentedon Jun 6, 2021
I looked into the fact that the arguments for
code()are(code, infostring, escaped)(whereinfostringhas thelangandescapedis aboolean) but therenderMarkdown()arguments are(code, string, language). I triedBut it didn't help at all.
When I dug into this deeper, the| markedfilters don't seem to be running at all. I put logs in here forcodeandrenderedCodeand some of the problematic text like[Find out more](misc/version-support-status)never came through at all.Doh, because the problematic markdown is not code... 🤦🏼♂️
Splaktar commentedon Jun 6, 2021
OK, so this doesn't parse properly:
But this does
Splaktar commentedon Jun 6, 2021
Failing test
Passing test
Splaktar commentedon Jun 6, 2021
OK, this is a bug or breaking change by design in
marked. Here's a minimal demo withmarked@2.0.7.It works with
marked@0.3.19(demo) and breaks in0.4.0.Splaktar commentedon Jun 6, 2021
I've opened markedjs/marked#2085 for this. It looks like this breaking change might be Working as Intended, but I hope that there is some way that I can monkey patch the parser/renderer to fix this w/o having to add a ton of blank line feeds throughout the old documentation.
Splaktar commentedon Jun 6, 2021
So the response from the
markedteam was extremely short, but it basically amounted toThis kind of custom HTML block renderer could live in dgeni-packages, but it's not clear to me that it's viable or worth the effort. The Angular project has already moved on to
remark, so there would be no benefit there.That said, it's not ideal to leave the AngularJS and AngularJS Material projects on an old version of
markedthat has a huge list of major vulnerabilities. Though it could be argued that those vulnerabilities could only be triggered via commits to the repos, which are extremely limited at this time.(╯°□°)╯︵ ┻━┻
chore(dgeni): fix rendering of docs markdown
[-]Regression: since 0.28.0, marked filter does not render markdown[/-][+]Regression: since 0.28.0, marked filter does not render markdown in HTML blocks[/+]petebacondarwin commentedon Jun 7, 2021
The CommonMark specification is what we should be following in our use of markdown (IMO). If the only problems with upgrading to the new version of dgeni-packages is that angular.js docs need to be updated to include blank lines to trigger the switch from HTML mode to markdown mode then that is the most appropriate fix.
Note that remark (used in the Angular project) also conforms to this CommonMark specfiication.
Closing as this is working as expected in dgeni-packages.