Expand ExtractorHTML to extract html from conditional comments

Description

Currently all html comments are filtered out of the extraction, but conditional comments may contain scripts, css, or other elements specific to certain versions of IE.

Conditional comments are used by some versions of Internet Explorer to allow html specific to a browser version to be ignored by other browsers/versions

Typical HTML Comment:
<!-- Comment content -->

Conditional HTML comment:
<

[endif]-->

This technique is commonly used to reference CSS & JavaScript that are designed to deal with the quirks of a specific version of Internet Explorer.

There is another syntax for conditional comments which already gets parsed by the regex since it doesn't start with "<!--"
<

[endif]>

https://en.wikipedia.org/wiki/Conditional_Comments

Environment

None

Status

Assignee

Unassigned

Reporter

Adam Miller

Labels

None

Group Assignee

None

ZendeskID

None

Estimated Difficulty

None

Actual Difficulty

None

Affects versions

Heritrix 3.1.0

Priority

Major
Configure