IllegalStateException "got suspicious value" in IpAddressSetDecideRule when

Description

Occured when using the IpAddressSetDecideRule with DecideRuledSheetAssociation.

Stacktrace:
2014-10-02 21:47:45.615 SEVERE thread-72 org.archive.crawler.spring.SheetOverlaysManager.applyOverlaysTo() problem determining whether to apply overlays, so not applying [superPolite] to http://fasteignasala.is/
java.lang.IllegalStateException: got suspicious value 0.0.0.0 for fasteignasala.is
at org.archive.modules.fetcher.DefaultServerCache.getHostFor(DefaultServerCache.java:112)
at org.archive.modules.net.ServerCache.getHostFor(ServerCache.java:64)
at org.archive.modules.deciderules.IpAddressSetDecideRule.getHostAddress(IpAddressSetDecideRule.java:85)
at org.archive.modules.deciderules.IpAddressSetDecideRule.evaluate(IpAddressSetDecideRule.java:57)
at org.archive.modules.deciderules.PredicatedDecideRule.innerDecide(PredicatedDecideRule.java:47)
at org.archive.modules.deciderules.DecideRule.decisionFor(DecideRule.java:60)
at org.archive.modules.deciderules.DecideRule.accepts(DecideRule.java:77)
at org.archive.crawler.spring.SheetOverlaysManager.applyOverlaysTo(SheetOverlaysManager.java:331)
at org.archive.crawler.frontier.WorkQueueFrontier.findEligibleURI(WorkQueueFrontier.java:632)
at org.archive.crawler.frontier.AbstractFrontier.next(AbstractFrontier.java:455)
at org.archive.crawler.framework.ToeThread.run(ToeThread.java:134)

Suggested fix, catch exception in IpAddressSetDecideRule and log it and return 'no match' when it occurs. Pull request on GitHub: https://github.com/internetarchive/heritrix3/pull/97

Environment

None

Activity

Show:
Kristinn Sigurðsson
October 3, 2014, 11:50 AM

It would probably be better if DefaultServerCache.getHostFor() would throw a more specific exception as well.

Noah Levitt
October 4, 2014, 1:49 AM

Assignee

Kristinn Sigurðsson

Reporter

Kristinn Sigurðsson

Labels

None

Issue Category

None

Group Assignee

None

ZendeskID

None

Estimated Difficulty

None

Actual Difficulty

None

Fix versions

Priority

Major
Configure