We're updating the issue view to help you get more done. 

Heritrix ignores robots.txt

Description

Environment

None

Status

Assignee

Unassigned

Reporter

Robert Jäschke

Labels

Group Assignee

None

ZendeskID

None

Estimated Difficulty

None

Actual Difficulty

None

Affects versions

Heritrix 3.2.0

Priority

Minor