ToeThread Fatal Exception: "kryo.SerializationException: Buffer limit exceeded" in BdbMultipleWorkQueues.get

Description

Several reports on list of ToeThreads dying with a stack like so:

SEVERE:
Fatal exception in ToeThread #5: (in thread 'ToeThread #5: ')
com.esotericsoftware.kryo.SerializationException: Buffer limit exceeded reading object of type: org.archive.modules.CrawlURI
at com.esotericsoftware.kryo.Kryo.readObjectData(Kryo.java:516)
at com.esotericsoftware.kryo.ObjectBuffer.readObjectData(ObjectBuffer.java:212)
at org.archive.bdb.KryoBinding.entryToObject(KryoBinding.java:87)
at org.archive.crawler.frontier.BdbMultipleWorkQueues.get(BdbMultipleWorkQueues.java:271)
at org.archive.crawler.frontier.BdbWorkQueue.peekItem(BdbWorkQueue.java:103)
at org.archive.crawler.frontier.WorkQueue.peek(WorkQueue.java:173)
at org.archive.crawler.frontier.WorkQueueFrontier.findEligibleURI(WorkQueueFrontier.java:651)
at org.archive.crawler.frontier.AbstractFrontier.next(AbstractFrontier.java:452)
at org.archive.crawler.framework.ToeThread.run(ToeThread.java:133)
Caused by: java.nio.BufferUnderflowException
at java.nio.Buffer.nextGetIndex(Buffer.java:474)
at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:117)
at com.esotericsoftware.kryo.Serializer.readObject(Serializer.java:54)
at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:206)
at com.esotericsoftware.kryo.serialize.FieldSerializer.readObjectData(FieldSerializer.java:178)
at com.esotericsoftware.kryo.Kryo.readObjectData(Kryo.java:512)
... 8 more

Debug logging captured by David Pane suggests the problem is actually a zero-length item where a CrawlURI is expected, probably the 'queue cap' of the next-in-order queue.

Environment

None

Status

Assignee

Unassigned

Reporter

Gordon Mohr

Labels

None

Group Assignee

None

ZendeskID

None

Estimated Difficulty

None

Actual Difficulty

None

Affects versions

Heritrix 3.1.0

Priority

Major
Configure