Updates to the HTML API in 6.7 – Make WordPress Core Skip to content Log In Register WordPress.org Showcase Plugins Themes Hosting News Resources Learn WordPress Documentation Education Forums Developers Blocks Patterns Photos Openverse ↗︎ WordPress.tv ↗︎ About About WordPress Make WordPress Events Five for the Future Enterprise Gutenberg ↗︎ Job Board ↗︎ Swag Store ↗︎ Get WordPress Search in WordPress.org Get WordPress WordPress.org Make WordPress Core People Tickets Components Handbook Browse Source Trac Timeline Log in People Tickets Components Handbook Browse Source Trac Timeline Log in Menu Hide welcome box Welcome! The WordPress coreCore Core is the set of software required to run WordPress. The Core Development Team builds WordPress. development team builds WordPress! Follow this site for general updates, status reports, and the occasional code debate. There’s lots of ways to contribute: Found a bugbug A bug is an error or unexpected result. Performance improvements, code optimization, and are considered enhancements, not defects. After feature freeze, only bugs are dealt with, with regressions (adverse changes from the previous version) being the highest priority.? Create a ticket in the bug tracker. Want to contribute? Get started quickly with tickets marked as good first bugs for new contributors or join a bug scrub. There’s more on the reports page, like patches needing testing, and on feature projects page. Other questions? Here is a detailed handbook for contributors, complete with tutorials. Communication Core uses Slack for real-time communication. Contributors live all over the world, so there are discussions happening at all hours of the day. Core development meetings are every Wednesday at 15:00 UTC in the #core channel on Slack. Anyone can join and participate or listen in! Dennis Snell 10:27 am on October 17, 2024 Tags: dev-notes ( 622 ), dev-notes-6-7 ( 15 ), html-api ( 6 )    Updates to the HTML API in 6.7 After important internal changes to the HTML API in 6.6, WordPress 6.7 brings forward a major leap in support with the HTMLHTML HyperText Markup Language. The semantic scripting language primarily used for outputting content in web browsers. Processor. Major Updates Full HTML Support Ability to modify most text in a document A full-parser mode in the HTML Processor Normalization of input documents Thorough review and alignment of CSS-related behaviors Important changes for existing code. Enhancements Bug Fixes Major Updates Full HTML Support After a long time of only supporting a subset of HTML, the HTML Processor now supports practically all HTML and all HTML tags1. The internal work during the past few releases made it possible to add the remaining support needed to tackle all the colorful varieties of HTML that exist. Only in the rarest situations will the HTML Processor now give up, and in none of these cases is it because the processor doesn’t know how to handle a given tagtag A directory in Subversion. WordPress uses tags to store a single snapshot of a version (3.6, 3.6.1, etc.), the common convention of tags in version control systems. (Not to be confused with post tags.). Effectively, the HTML APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. is entering a new phase, where the rules for understanding HTML are finally baked in, and the API can start to proliferate into new developer APIs and replace older and buggier HTML processing code. Ability to modify most text in a document The first of a new set of modification methods has been added to the HTML API: set_modifiable_text(). When paused on a text node, a special self-contained element like SCRIPT, STYLE, or TITLE, or on a comment, it’s possible to replace the text content of that node. while ( $processor->next_tag( 'SCRIPT' ) ) { $existing_script = $processor->get_modifiable_text(); $prefix = "// Careful, this is executable!\n"; $processor->set_modifiable_text( "{$prefix}{$existing_script}" ); } This method will handle the appropriate escaping for the kind of node it is. For example, it will escape the text for a SCRIPT element in a different way than for a STYLE element or a raw text node. This is important, because it means you don’t have to think about what kind of text it is, but it will remain reliable. It’s important to understand here that this operates on the level of individual text nodes. set_modifiable_text() is different than set_inner_text(). A given element may contain multiple text nodes which are separated by things like HTML comments, specific byte sequences, and invalidinvalid A resolution on the bug tracker (and generally common in software development, sometimes also notabug) that indicates the ticket is not a bug, is a support request, or is generally invalid. markup which is interpreted as a comment. For set_inner_text() it will be best to wait until the HTML API itself supports these operations. A full-parser mode in the HTML Processor Until now, the HTML Processor has provided a slightly unexpected interface. Creating a processor involved a call to create_fragment(), which creates something called a “fragment parser.” There was a good reason for this: the fragment parser is the version that’s used in JavaScriptJavaScript JavaScript or JS is an object-oriented computer programming language commonly used to create interactive effects within web browsers. WordPress makes extensive use of JS for a better user experience. While PHP is executed on the server, JS executes within a user’s browser. https://www.javascript.com when assigning node.innerHTML = newHtml. Unfortunately, the fragment parser cannot ingest an entire document from start to finish. If you worked with the HTML Processor you would have noticed that it gave up as soon as it found the starting <!DOCTYPE html> or <html> tokens. In WordPress 6.7 the HTML Processor offers create_full_parser() to fill this need. Thanks to completing support for all of the remaining contexts in a document, the HTML Processor can now start with arbitrary HTML and understand how it turns into a DOM tree. While possibly unexpected, the default implementation with the fragment processor is still the best option in most cases. If you’ve ever had to deal with HTML parsing libraries adding <!DOCTYPE html><html><head> (and other tags) to the start of every snippet of HTML that they process, it’s because these tools are performing full parses and full serialization. When working with anything other than a complete HTML document, the fragment parser remains the most appropriate tool. Process fragments of HTML with the fragment parser, and full documents with the full parser. Normalization of input documents With the HTML specification rules in place it’s possible to start building higher-level tools and helpers without having to worry about the complexities of HTML. The first of these is WP_HTML_Processor::normalize(), which gives us the HTML we always wanted. Normalizing is the process of taking any HTML input and returning a fully well-formed equivalent document. Since a picture is worth a thousand words: $html = '</p><p class=wp class=duplicate><[GETTING[started]]><p><?not a tag?></p done=yet?><script><!--<script></script>'; echo WP_HTML_Processor::normalize( $html ); // <p></p><p class="wp">&lt;[GETTING[started]]></p><p><!--?not a tag?--></p> This little function is packed with utility. It interprets the input HTML the way a browser would, meaning that it applies all of the complicated parsing rules, and it prints out a new serialization of that parsed structure. Attribute quoting, mismatched tags, missing tags, invalid syntax, unexpected whitespace, comments, and more – it prints out tags without duplicates, where each value is double-quoted, where all relevant syntax characters are encoded, and it even removes partial tokens at the end of the input. That’s right, normalize() won’t leave you hanging – literally! $html = 'Just a <em>sneaky but <!-- foiled attempt to hide the rest of the page'; echo WP_HTML_Processor::normalize( $html ); // Just a <em>sneaky but </em> Significantly, there’s something extremely valuable in the output of this function: once HTML has been normalized, it’s considerably safer to process with naive parsing tools. There’s a guarantee on how the text it produces will appear, so string-based and regular-expression-based tools will instantly become more reliable; the edge cases they fail to handle will never make it out of normalize(). Thorough review and alignment of CSSCSS Cascading Style Sheets.-related behaviors CSS selectors and HTML interact via the document mode, which is either in standards mode or quirks mode. In standards mode, CSS selectors match classes and IDs in a byte-for-byte match. Otherwise they are matched in an ASCII case-insensitive manner. Confusing! Thankfully, the entire HTML API was audited to ensure compliance even in how CSS class selectors work, and the behavior of methods like has_class(), add_class(), and remove_class() will now respect each document’s mode. For almost every case this should be the “standards mode” and so both of the HTML API processors default to it, matching class names in a byte-for-byte manner. When creating an HTML Processor though with the full parser, since it’s able to determine the document mode from the input HTML, it will make the same choice a browser makes, and when appropriate, will match case-insensitively. Important changes for existing code. There should be no significant changes required to existing code written with previous versions of the HTML API. While almost all HTML documents are now supported, it’s still required to verify if the HTML Processor bailed while attempting to process unsupported input. There were several changes to the way null bytes and whitespace are handled in specific circumstances. These cases represent invalid sections of documents and normal HTML inputs aren’t affected by them. Since changes were also made with the handling of ASCII casing in CSS class names, there may be cases where a class name previously matched and now won’t. For example, a browser will not match the CSS class name selector for full-width with an element containing FULL-WIDTH in its class attribute unless the document is parsed in quirks-mode. The HTML API is now processing its inputs in standards mode, matching the behavior of a browser, unless created with a full parser containing the appropriate DOCTYPE to indicate quirks mode. Code that relied on has_class() need not change, but the results in WordPress 6.7 may be different at times because it has become more reliable to the proper handling of these class names. Enhancements Low-level parsing updates improved the scanning speed of the Tag Processor by 3.5-7.5%. [Core-61545] When removing class names, leading whitespace in front of the class name is now removed. [Core-61531] CSS class names and class selectors behave according to the document mode for the input document, or standards mode if none can be directly inferred. [Core-61531] Improved spec-compliance by handling remaining HTML tags and insertion modes, including SVG, TABLE, TEMPLATE, and more. [Core-61576] Added WP_HTML_Processor::create_full_parser() to process entire HTML documents from start to finish. [Core-61576] Added get_qualified_tag_name() and get_qualified_attribute_name() to return case-variants for certain tags inside foreign content where the HTML rules specify. [Core-61576] It’s now possible to replace the value of modifiable text. [Core-61617] Added get_unsupported_exception() to provide more debugging information when the HTML Processor bails on unsupported input. [Core-61646] set_attribute() returns false when WordPress rejects the update. [Core-61719] Added subdivide_text_appropriately() to aid in algorithms wanting to skip null byte or whitespace-only prefixes of text nodes. [Core-61974] Added normalize() and serialize() to return a normalized representation of parsed input HTML. [Core-62036] Bug Fixes class_list() now replaces null bytes with the Unicode replacement character. [Core-61531] The HTML Processor now properly generates implied end tags. Previously, in some cases, it would generate the end tag but leave the element open on the stack of open elements, leading to mistakes in how it handled later tags in the document. [Core-61576] The HTML Processor now respects the array form of a next_tag() query when specifying the tag_name. [Core-61581] The HTML API no longer hangs when fed documents ending with open SCRIPT elements and which end in one of the - or < characters. [Core-61810] The Tag Processor now reports all un-closed funky comments as having paused at incomplete input. Previously, when the document ended immediately after opening the funky comment, e.g. </# with no more characters, it failed to report the incomplete token. [Core-61831] There are some unusual situations in HTML which are not supported. They relate to what HTML calls “fostering” and “adoption,” where a node’s placement in the document tree does not correspond to where it appears in the HTML source. This arises in roughly 0.5% of all websites on the web and isn’t currently supported in the HTML API. The deprecated PLAINTEXT element is also unsupported. ↩︎ Props @jonsurrell and @fabiankaegy for review. #dev-notes, #dev-notes-6-7, #html-api Share this: Share on Threads (Opens in new window) Threads Share on Mastodon (Opens in new window) Mastodon Share on Bluesky (Opens in new window) Bluesky Share on X (Opens in new window) X Share on Facebook (Opens in new window) Facebook Share on LinkedIn (Opens in new window) LinkedIn Miguel Fonseca 12:49 pm on October 17, 2024 Nice work! Russell Heimlich 10:42 pm on October 17, 2024 This is awesome. I love these updates! Austin 2:20 pm on October 18, 2024 Nice enhancements! !Benni 1:51 pm on October 22, 2024 Really nice work! Thanks @dmsnell Maybe GutenbergGutenberg The Gutenberg project is the new Editor Interface for WordPress. The editor improves the process and experience of creating new content, making writing rich content much simpler. It uses ‘blocks’ to add richness rather than shortcodes, custom HTML etc. https://wordpress.org/gutenberg/ could implement a Table of Contents now, since it’s now easier than ever to dynamically find headings in the content? Weston Ruter 7:13 pm on October 22, 2024 Would HTMLHTML HyperText Markup Language. The semantic scripting language primarily used for outputting content in web browsers. parsing be needed here given that the heading structure should already be identifiable by blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. parsing? !Benni 8:09 am on October 23, 2024 Hey @westonruter I’m not sure. For example, if a user inserts a heading directly within an HTMLHTML HyperText Markup Language. The semantic scripting language primarily used for outputting content in web browsers. blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience., would block parsing be able to detect that? Or in cases where a user uses alternative heading blocks from plugins or a pluginPlugin A plugin is a piece of software containing a group of functions that can be added to a WordPress website. They can extend functionality or add new features to your WordPress websites. WordPress plugins are written in the PHP programming language and integrate seamlessly with WordPress. These can be free in the WordPress.org Plugin Directory https://wordpress.org/plugins/ or can be cost-based plugin from a third-party. modifies headings dynamically—like replacing placeholders (e.g., price_for_something) with actual values (e.g., $XX.XX) using the the_content filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output.—could block parsing detect these? Weston Ruter 4:55 pm on October 23, 2024 Headings to a Custom HTMLHTML HyperText Markup Language. The semantic scripting language primarily used for outputting content in web browsers. blockBlock Block is the abstract term used to describe units of markup that, composed together, form the content or layout of a webpage using the WordPress editor. The idea combines concepts of what in the past may have achieved with shortcodes, custom HTML, and embed discovery into a single consistent API and user experience. wouldn’t be detected, no. But that isn’t common, right? Plugins that filterFilter Filters are one of the two types of Hooks https://codex.wordpress.org/Plugin_API/Hooks. They provide a way for functions to modify data of other functions. They are the counterpart to Actions. Unlike Actions, filters are meant to work in an isolated manner, and should never have side effects such as affecting global variables and output. placeholders in Heading block markup would work if the_content filters run before do_blocks at priority 9. In the end, using the HTML APIAPI An API or Application Programming Interface is a software intermediary that allows programs to interact with each other and share data in limited, clearly defined ways. would handle all of the edge cases. But I don’t think the HTML Processor would strictly be required. Using the more basic HTML Tag Processor should also be able to do the trick. digitalapps 4:14 am on October 28, 2024 tasty 😋 Well done team! Comments are closed. Post navigation ← Agenda, Dev Chat, Oct 16, 2024 Roster of design tools per block (WordPress 6.7 edition) → Site resources Search Email Updates Enter your email address to subscribe to this blog and receive notifications of new posts by email. Type your email… Subscribe Join 6,812 other subscribers Recent Updates Recent Comments No RepliesRecent UpdatesRecent CommentsNo Replies Current Release The current release in progress is WordPress 7.1. Planned future releases are listed on the Project Roadmap. Feature projects not tied to specific releases can be found on the Features page. Regular Chats Note: All chats happen on Slack. Weekly Developer Meetings: [time relative]Wednesday 15:00 UTC[/time] in #core About the Dev Chat Agendas | Summaries Performance Weekly Chat [time relative]Tuesday 15:00 UTC[/time] in #core-performance New Contributors Chat The 2nd and 4th Wednesday of every month at 17:00 UTC in #core See all meetings → Recent Posts and Comments Team Pledges 2779 people have pledged time to contribute to Core Team efforts! When looking for help on a project or program, try starting by reaching out to them! About News Hosting Privacy Showcase Themes Plugins Patterns Learn Documentation Developers WordPress.tv ↗ Get Involved Events Donate ↗ Five for the Future WordPress.com ↗ Matt ↗ bbPress ↗ BuddyPress ↗ WordPress.org WordPress.org Visit our X (formerly Twitter) account Visit our Bluesky account Visit our Mastodon account Visit our Threads account Visit our Facebook page Visit our Instagram account Visit our LinkedIn account Visit our TikTok account Visit our YouTube channel Visit our Tumblr account The WordPress® trademark is the intellectual property of the WordPress Foundation. ssearch ccompose new post r reply e edit t go to top j go to the next post or comment k go to the previous post or comment o toggle comment visibility esc cancel edit post or comment Please enable JavaScript to view this page properly.