Related to the GSoC '25 project.

This issue will serve as the central discussion for integrating the work completed in my fork. We need to discuss the steps required to merge the new Javadoc grammar and its related changes into the main repository. We also need to make sure that we outline any changes to google style.

Project Final Report: https://mahfouz72.github.io/gsoc-report/2025

Key Changes

• Implemented a new Javadoc lexer and parser in ANTLR with full coverage of the Javadoc Specification
• Added a visitor-based AST construction layer to map ANTLR parse trees into Checkstyle’s internal AST.
• Integrated the new parser into Checkstyle’s APIs.
• Updated all AbstractJavadocCheck subclasses and Javadoc-related checks to work with the new AST.
• Added extensive AST tests and regression tests to ensure correctness and stability.

Impact

• Improves maintainability and long-term stability of Javadoc parsing.
• Provides a cleaner and more consistent AST structure for checks.
• Backward compatibility preserved, existing checks continue to work after migration.
• Expected minor differences in parsing behavior compared to the old parser (verified through regressions).

Testing

• Unit tests cover the full grammar and AST conversion layer.
• Regression testing performed on large projects confirmed correctness and identified only expected differences.

Issue includes:

• Create a new Javadoc parser and lexer
• Add new javadoc grammar and build the AST using visitors
• Update JavadocNodeImpl
• Update javadoc tags ast
• Replace WS token with empty TEXT for consistency
• Update MissingDeprecatedCheck
• Update SingleLineJavadocCheck
• Fix column number bug when line has no leading asterisk
• Field name and field type should be optional in @serialField
• Add support for html comment parsing
• Update JavadocBlockTagLocationCheck
• @hidden should be of type HIDDEN_BLOCK_TAG
• Allow dashes in systemProperty inline tag
• Fix grammar bugs
• Fix type parameter in @see description is parsed as HTML tag
• Fix line number and column number bug by refactoring modes
• Update AtclauseOrderCheck
• Convert Javadoc lexer token recognition errors into violations
• Update NonEmptyAtclauseDescriptionCheck
• Remove unused code in JavadocDetailNodeParser and the related references
• Update RequireEmptyLineBeforeBlockTagGroupCheck
• Update MissingWhitespaceAfterAsteriskCheck
• Update AbstractJavadocTest and Connet Unclosed html message to the new parser
• Update JavadocMissingLeadingAsteriskCheck
• Update JavadocPropertiesGenerator and enable TokenUtilTest, TokenTypesTest
• Enable MainTest.java
• Update JavadocLeadingAsteriskAlignCheck
• Update JavadocTagContinuationIndentationCheck
• Update SummaryJavadocCheck
• Update JavadocMetadataScraperUtil and ModuleJavadocParsingUtil
• Update ParseTreeTablePresentation and ParseTreeTableModel Tests
• Update ClassAndPropertiesSettersJavadocScraper, JavadocCommentsTokenTypesTest and JavadocNodeImplTest
• Update SiteUtilTest
• Update JavadocUtilTest
• Update JavadocNodeImpl and CodeSelectorPresentation
• Update JavadocParagraph and remove JavadocTokenTypes
• Remove old parser files
• Run XdocsPagesTest
• Fix ImmutabilityTest
• Fix checkstyle violations
• Fix inspection violations
• Fix CI issues
• Update pitest-profiles
• Fix false positive in JavadocParagraph

Migration Notes

1. NonEmptyAtclauseDescriptionCheck

The following token names have been updated to better reflect their purpose:
XXXXX_LITERAL → XXXXX_BLOCK_TAG.

If your configuration currently uses the old token names:

<module name="Checker">
  <module name="TreeWalker">
    <module name="NonEmptyAtclauseDescription">
      <property name="javadocTokens" value="PARAM_LITERAL,THROWS_LITERAL"/>
    </module>
  </module>
</module>

You should update it to use the new token names:

<module name="Checker">
  <module name="TreeWalker">
    <module name="NonEmptyAtclauseDescription">
      <property name="javadocTokens" value="PARAM_BLOCK_TAG,THROWS_BLOCK_TAG"/>
    </module>
  </module>
</module>

2. Custom Javadoc Checks

All custom javadoc checks should be reimplemented because the AST structure and token names have changed

Examples:

• Updare AtclauseOrderCheck checkstyle-GSoC25/checkstyle#42
• Update JavadocMissingLeadingAsteriskCheck checkstyle-GSoC25/checkstyle#65
• Update SummaryJavadocCheck checkstyle-GSoC25/checkstyle#78