Job is Finished but cannot terminate - no reports generated

Description

After a 2 1/2 day crawl, the job shows FINISHED_TIME_LIMIT. The terminate button in the web interface is disabled, but selecting teardown causes pink message at top of window "waiting for job to finish" I continued several attempts over a time frame of 1 hr to no avail. Attempting to get reports generated, since non were generated yet, I selected the report links on the web interface. Mimetypes report generated fine. Crawl Summary will not generate a report.

Stack dump:

2011-11-10 12:01:01
Full thread dump Java HotSpot(TM) 64-Bit Server VM (20.4-b02 mixed mode):

"Attach Listener" daemon prio=10 tid=0x00002aaad05e9800 nid=0x76d7 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"2108783494@qtp0-83" prio=10 tid=0x00002aaabc425000 nid=0x741b waiting on condition [0x0000000040fc9000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000301795a50> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178)
    at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:807)
    at com.sleepycat.je.latch.SharedLatch.acquireExclusive(SharedLatch.java:95)
    at com.sleepycat.je.tree.IN.latch(IN.java:406)
    at com.sleepycat.je.dbi.CursorImpl.latchBIN(CursorImpl.java:554)
    at com.sleepycat.je.dbi.CursorImpl.latchBINs(CursorImpl.java:573)
    at com.sleepycat.je.dbi.CursorImpl.cloneCursor(CursorImpl.java:347)
    at com.sleepycat.je.Cursor.beginMoveCursor(Cursor.java:2732)
    at com.sleepycat.je.Cursor.beginMoveCursor(Cursor.java:2739)
    at com.sleepycat.je.Cursor.retrieveNextAllowPhantoms(Cursor.java:2481)
    at com.sleepycat.je.Cursor.retrieveNext(Cursor.java:2304)
    at com.sleepycat.je.Cursor.getNextNoDup(Cursor.java:1129)
    at com.sleepycat.util.keyrange.RangeCursor.doGetNextNoDup(RangeCursor.java:920)
    at com.sleepycat.util.keyrange.RangeCursor.getNextNoDup(RangeCursor.java:475)
    at com.sleepycat.collections.DataCursor.getNextNoDup(DataCursor.java:456)
    at com.sleepycat.collections.StoredCollection.size(StoredCollection.java:597)
    at org.archive.crawler.reporting.CrawlSummaryReport.write(CrawlSummaryReport.java:50)
    at org.archive.crawler.reporting.StatisticsTracker.writeReportFile(StatisticsTracker.java:909)
    at org.archive.crawler.reporting.StatisticsTracker.writeReportFile(StatisticsTracker.java:888)
    at org.archive.crawler.restlet.ReportGenResource.represent(ReportGenResource.java:50)
    at org.restlet.resource.Resource.getRepresentation(Resource.java:302)
    at org.restlet.resource.Resource.handleGet(Resource.java:464)
    at org.restlet.Finder.handle(Finder.java:353)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Router.handle(Router.java:504)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at com.noelios.restlet.StatusFilter.doHandle(StatusFilter.java:130)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Filter.handle(Filter.java:195)
    at com.noelios.restlet.ChainHelper.handle(ChainHelper.java:124)
    at com.noelios.restlet.application.ApplicationHelper.handle(ApplicationHelper.java:112)
    at org.restlet.Application.handle(Application.java:341)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Guard.accept(Guard.java:198)
    at org.restlet.Guard.doHandle(Guard.java:354)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Router.handle(Router.java:504)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Router.handle(Router.java:504)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at com.noelios.restlet.StatusFilter.doHandle(StatusFilter.java:130)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Filter.handle(Filter.java:195)
    at com.noelios.restlet.ChainHelper.handle(ChainHelper.java:124)
    at org.restlet.Component.handle(Component.java:673)
    at org.restlet.Server.handle(Server.java:331)
    at com.noelios.restlet.ServerHelper.handle(ServerHelper.java:68)
    at com.noelios.restlet.http.HttpServerHelper.handle(HttpServerHelper.java:147)
    at com.noelios.restlet.ext.jetty.JettyServerHelper$WrappedServer.handle(JettyServerHelper.java:174)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
    at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
    at org.mortbay.jetty.security.SslSocketConnector$SslConnection.run(SslSocketConnector.java:636)
    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)

"1473873093@qtp0-76" prio=10 tid=0x00002aaabc0bb800 nid=0x72ed waiting on condition [0x0000000041887000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000301795a50> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178)
    at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:807)
    at com.sleepycat.je.latch.SharedLatch.acquireExclusive(SharedLatch.java:95)
    at com.sleepycat.je.tree.IN.latch(IN.java:406)
    at com.sleepycat.je.dbi.CursorImpl.latchBIN(CursorImpl.java:554)
    at com.sleepycat.je.dbi.CursorImpl.removeCursorBIN(CursorImpl.java:657)
    at com.sleepycat.je.dbi.CursorImpl.removeCursor(CursorImpl.java:650)
    at com.sleepycat.je.dbi.CursorImpl.close(CursorImpl.java:760)
    at com.sleepycat.je.Cursor.endMoveCursor(Cursor.java:2771)
    at com.sleepycat.je.Cursor.retrieveNextAllowPhantoms(Cursor.java:2515)
    at com.sleepycat.je.Cursor.retrieveNext(Cursor.java:2304)
    at com.sleepycat.je.Cursor.getNextNoDup(Cursor.java:1129)
    at com.sleepycat.util.keyrange.RangeCursor.doGetNextNoDup(RangeCursor.java:920)
    at com.sleepycat.util.keyrange.RangeCursor.getNextNoDup(RangeCursor.java:475)
    at com.sleepycat.collections.DataCursor.getNextNoDup(DataCursor.java:456)
    at com.sleepycat.collections.StoredCollection.size(StoredCollection.java:597)
    at org.archive.crawler.reporting.CrawlSummaryReport.write(CrawlSummaryReport.java:50)
    at org.archive.crawler.reporting.StatisticsTracker.writeReportFile(StatisticsTracker.java:909)
    at org.archive.crawler.reporting.StatisticsTracker.writeReportFile(StatisticsTracker.java:888)
    at org.archive.crawler.restlet.ReportGenResource.represent(ReportGenResource.java:50)
    at org.restlet.resource.Resource.getRepresentation(Resource.java:302)
    at org.restlet.resource.Resource.handleGet(Resource.java:464)
    at org.restlet.Finder.handle(Finder.java:353)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Router.handle(Router.java:504)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at com.noelios.restlet.StatusFilter.doHandle(StatusFilter.java:130)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Filter.handle(Filter.java:195)
    at com.noelios.restlet.ChainHelper.handle(ChainHelper.java:124)
    at com.noelios.restlet.application.ApplicationHelper.handle(ApplicationHelper.java:112)
    at org.restlet.Application.handle(Application.java:341)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Guard.accept(Guard.java:198)
    at org.restlet.Guard.doHandle(Guard.java:354)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Router.handle(Router.java:504)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Router.handle(Router.java:504)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at com.noelios.restlet.StatusFilter.doHandle(StatusFilter.java:130)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Filter.handle(Filter.java:195)
    at com.noelios.restlet.ChainHelper.handle(ChainHelper.java:124)
    at org.restlet.Component.handle(Component.java:673)
    at org.restlet.Server.handle(Server.java:331)
    at com.noelios.restlet.ServerHelper.handle(ServerHelper.java:68)
    at com.noelios.restlet.http.HttpServerHelper.handle(HttpServerHelper.java:147)
    at com.noelios.restlet.ext.jetty.JettyServerHelper$WrappedServer.handle(JettyServerHelper.java:174)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
    at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
    at org.mortbay.jetty.security.SslSocketConnector$SslConnection.run(SslSocketConnector.java:636)
    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)

"1395984998@qtp0-73" prio=10 tid=0x00002aaabcd1d800 nid=0x72a9 waiting on condition [0x00002aaada9ab000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000301795a50> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178)
    at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:807)
    at com.sleepycat.je.latch.SharedLatch.acquireExclusive(SharedLatch.java:95)
    at com.sleepycat.je.tree.IN.latch(IN.java:406)
    at com.sleepycat.je.tree.Tree.getNextBinInternal(Tree.java:1359)
    at com.sleepycat.je.tree.Tree.getNextBin(Tree.java:1221)
    at com.sleepycat.je.dbi.CursorImpl.getNextWithKeyChangeStatus(CursorImpl.java:1749)
    at com.sleepycat.je.dbi.CursorImpl.getNext(CursorImpl.java:1617)
    at com.sleepycat.je.dbi.CursorImpl.getNextNoDup(CursorImpl.java:1809)
    at com.sleepycat.je.Cursor.retrieveNextAllowPhantoms(Cursor.java:2497)
    at com.sleepycat.je.Cursor.retrieveNext(Cursor.java:2304)
    at com.sleepycat.je.Cursor.getNextNoDup(Cursor.java:1129)
    at com.sleepycat.util.keyrange.RangeCursor.doGetNextNoDup(RangeCursor.java:920)
    at com.sleepycat.util.keyrange.RangeCursor.getNextNoDup(RangeCursor.java:475)
    at com.sleepycat.collections.DataCursor.getNextNoDup(DataCursor.java:456)
    at com.sleepycat.collections.BlockIterator.hasNext(BlockIterator.java:355)
    at org.archive.modules.fetcher.DefaultServerCache.forAllHostsDo(DefaultServerCache.java:152)
    at org.archive.crawler.reporting.StatisticsTracker.calcReverseSortedHostsDistribution(StatisticsTracker.java:875)
    at org.archive.crawler.reporting.HostsReport.write(HostsReport.java:54)
    at org.archive.crawler.reporting.StatisticsTracker.writeReportFile(StatisticsTracker.java:909)
    at org.archive.crawler.reporting.StatisticsTracker.writeReportFile(StatisticsTracker.java:888)
    at org.archive.crawler.restlet.ReportGenResource.represent(ReportGenResource.java:50)
    at org.restlet.resource.Resource.getRepresentation(Resource.java:302)
    at org.restlet.resource.Resource.handleGet(Resource.java:464)
    at org.restlet.Finder.handle(Finder.java:353)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Router.handle(Router.java:504)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at com.noelios.restlet.StatusFilter.doHandle(StatusFilter.java:130)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Filter.handle(Filter.java:195)
    at com.noelios.restlet.ChainHelper.handle(ChainHelper.java:124)
    at com.noelios.restlet.application.ApplicationHelper.handle(ApplicationHelper.java:112)
    at org.restlet.Application.handle(Application.java:341)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Guard.accept(Guard.java:198)
    at org.restlet.Guard.doHandle(Guard.java:354)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Router.handle(Router.java:504)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Router.handle(Router.java:504)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at com.noelios.restlet.StatusFilter.doHandle(StatusFilter.java:130)
    at org.restlet.Filter.handle(Filter.java:195)
    at org.restlet.Filter.doHandle(Filter.java:150)
    at org.restlet.Filter.handle(Filter.java:195)
    at com.noelios.restlet.ChainHelper.handle(ChainHelper.java:124)
    at org.restlet.Component.handle(Component.java:673)
    at org.restlet.Server.handle(Server.java:331)
    at com.noelios.restlet.ServerHelper.handle(ServerHelper.java:68)
    at com.noelios.restlet.http.HttpServerHelper.handle(HttpServerHelper.java:147)
    at com.noelios.restlet.ext.jetty.JettyServerHelper$WrappedServer.handle(JettyServerHelper.java:174)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
    at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
    at org.mortbay.jetty.security.SslSocketConnector$SslConnection.run(SslSocketConnector.java:636)
    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)

"JEEvictor" daemon prio=10 tid=0x00002aaadc2c2800 nid=0x77a4 waiting on condition [0x00002aaad14bc000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000265a729b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
    at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:342)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
    at java.lang.Thread.run(Thread.java:662)

"JEEvictor" daemon prio=10 tid=0x00002aaadc6ca800 nid=0x77a3 waiting on condition [0x00002aaacf6ba000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000265a729b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
    at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:342)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
    at java.lang.Thread.run(Thread.java:662)

"JEEvictor" daemon prio=10 tid=0x00002aaabc960800 nid=0x77a2 waiting on condition [0x00002aaad3523000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000265a729b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
    at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:342)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
    at java.lang.Thread.run(Thread.java:662)

"JEEvictor" daemon prio=10 tid=0x00002aaad47b0800 nid=0x77a1 waiting on condition [0x00002aaacf0d3000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000265a729b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
    at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:342)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
    at java.lang.Thread.run(Thread.java:662)

"JEEvictor" daemon prio=10 tid=0x00002aaabc480000 nid=0x77a0 waiting on condition [0x00002aaadadb0000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000265a729b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
    at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:342)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
    at java.lang.Thread.run(Thread.java:662)

"JEEvictor" daemon prio=10 tid=0x00002aaad4f8b000 nid=0x779f waiting on condition [0x00002aaad086a000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000265a729b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
    at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:342)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
    at java.lang.Thread.run(Thread.java:662)

"JEEvictor" daemon prio=10 tid=0x00002aaabd392800 nid=0x779e waiting on condition [0x00002aaada5a8000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000265a729b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
    at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:342)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
    at java.lang.Thread.run(Thread.java:662)

"JEEvictor" daemon prio=10 tid=0x00002aaadc2c3800 nid=0x779d waiting on condition [0x00002aaad8f92000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000265a729b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
    at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:342)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
    at java.lang.Thread.run(Thread.java:662)

"JEEvictor" daemon prio=10 tid=0x0000000058708000 nid=0x779c waiting on condition [0x00002aaada7aa000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000265a729b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
    at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:342)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
    at java.lang.Thread.run(Thread.java:662)

"JEEvictor" daemon prio=10 tid=0x00002aaadc1fd000 nid=0x779b waiting on condition [0x00002aaad15bd000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000265a729b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
    at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:342)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
    at java.lang.Thread.run(Thread.java:662)

"com.google.common.base.internal.Finalizer" daemon prio=10 tid=0x00002aaabc2fc000 nid=0x62d0 in Object.wait() [0x00002aaace2e0000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)

  • locked <0x0000000267daf620> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
    at com.google.common.base.internal.Finalizer.run(Finalizer.java:127)

"org.archive.crawler.frontier.BdbFrontier@7ee41d4a.managerThread" prio=10 tid=0x00002aaabc1c5000 nid=0x5dea runnable [0x0000000041787000]
java.lang.Thread.State: RUNNABLE
at java.io.RandomAccessFile.readBytes(Native Method)
at java.io.RandomAccessFile.read(RandomAccessFile.java:338)
at com.sleepycat.je.log.FileManager.readFromFileInternal(FileManager.java:1609)

  • locked <0x000000061fd18a98> (a com.sleepycat.je.log.FileManager$1)
    at com.sleepycat.je.log.FileManager.readFromFile(FileManager.java:1560)
    at com.sleepycat.je.log.FileManager.readFromFile(FileManager.java:1498)
    at com.sleepycat.je.log.FileSource.getBytes(FileSource.java:56)
    at com.sleepycat.je.log.LogManager.getLogEntryFromLogSource(LogManager.java:852)
    at com.sleepycat.je.log.LogManager.getLogEntry(LogManager.java:781)
    at com.sleepycat.je.log.LogManager.getLogEntryAllowInvisibleAtRecovery(LogManager.java:742)
    at com.sleepycat.je.tree.IN.fetchTarget(IN.java:1315)
    at com.sleepycat.je.tree.BIN.fetchTarget(BIN.java:1367)
    at com.sleepycat.je.dbi.CursorImpl.fetchCurrent(CursorImpl.java:2499)
    at com.sleepycat.je.dbi.CursorImpl.getCurrentAlreadyLatched(CursorImpl.java:1545)
    at com.sleepycat.je.dbi.CursorImpl.getNextWithKeyChangeStatus(CursorImpl.java:1692)
    at com.sleepycat.je.dbi.CursorImpl.getNext(CursorImpl.java:1617)
    at com.sleepycat.je.dbi.CursorImpl.getNextNoDup(CursorImpl.java:1809)
    at com.sleepycat.je.Cursor.retrieveNextAllowPhantoms(Cursor.java:2497)
    at com.sleepycat.je.Cursor.retrieveNext(Cursor.java:2304)
    at com.sleepycat.je.Cursor.getNextNoDup(Cursor.java:1129)
    at com.sleepycat.util.keyrange.RangeCursor.doGetNextNoDup(RangeCursor.java:920)
    at com.sleepycat.util.keyrange.RangeCursor.getNextNoDup(RangeCursor.java:475)
    at com.sleepycat.collections.DataCursor.getNextNoDup(DataCursor.java:456)
    at com.sleepycat.collections.StoredCollection.size(StoredCollection.java:597)
    at org.archive.crawler.reporting.CrawlSummaryReport.write(CrawlSummaryReport.java:50)
    at org.archive.crawler.reporting.StatisticsTracker.writeReportFile(StatisticsTracker.java:909)
    at org.archive.crawler.reporting.StatisticsTracker.dumpReports(StatisticsTracker.java:937)
    at org.archive.crawler.reporting.StatisticsTracker.crawlEnded(StatisticsTracker.java:565)
    at org.archive.crawler.reporting.StatisticsTracker.onApplicationEvent(StatisticsTracker.java:985)
    at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:97)
    at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:303)
    at org.archive.crawler.framework.CrawlController.sendCrawlStateChangeEvent(CrawlController.java:326)
    at org.archive.crawler.framework.CrawlController.completeStop(CrawlController.java:391)
    at org.archive.crawler.framework.CrawlController.noteFrontierState(CrawlController.java:656)
    at org.archive.crawler.frontier.AbstractFrontier.reachedState(AbstractFrontier.java:439)
    at org.archive.crawler.frontier.AbstractFrontier.managementTasks(AbstractFrontier.java:397)
    at org.archive.crawler.frontier.AbstractFrontier$1.run(AbstractFrontier.java:315)

"Checkpointer" daemon prio=10 tid=0x00002aaabc03f000 nid=0x5de0 in Object.wait() [0x0000000041585000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.sleepycat.je.utilint.DaemonThread.run(DaemonThread.java:174)

  • locked <0x00000002670dcfd8> (a java.lang.Object)
    at java.lang.Thread.run(Thread.java:662)

"Cleaner-1" daemon prio=10 tid=0x00002aaabc41a800 nid=0x5ddf in Object.wait() [0x00000000412de000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.sleepycat.je.utilint.DaemonThread.run(DaemonThread.java:174)

  • locked <0x00000002670b3c98> (a java.lang.Object)
    at java.lang.Thread.run(Thread.java:662)

"INCompressor" daemon prio=10 tid=0x00002aaabc41f000 nid=0x5dde in Object.wait() [0x0000000040d1d000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at com.sleepycat.je.utilint.DaemonThread.run(DaemonThread.java:176)

  • locked <0x00000002670dd120> (a java.lang.Object)
    at java.lang.Thread.run(Thread.java:662)

"DestroyJavaVM" prio=10 tid=0x00002aaab43e7000 nid=0x5d78 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"1853955208@qtp0-0 - Acceptor0 SslSocketConnector@10.1.1.249:8443" prio=10 tid=0x00002aaabc05b000 nid=0x5d8c runnable [0x0000000040e32000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)

  • locked <0x0000000266f803e0> (a java.net.SocksSocketImpl)
    at java.net.ServerSocket.implAccept(ServerSocket.java:462)
    at com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:261)
    at org.mortbay.jetty.security.SslSocketConnector.accept(SslSocketConnector.java:172)
    at org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:707)
    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)

"Low Memory Detector" daemon prio=10 tid=0x00002aaab4006800 nid=0x5d87 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"C2 CompilerThread1" daemon prio=10 tid=0x00002aaab4004000 nid=0x5d86 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" daemon prio=10 tid=0x00002aaab4001000 nid=0x5d85 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=10 tid=0x0000000057e6b000 nid=0x5d84 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=10 tid=0x0000000057e4e000 nid=0x5d83 in Object.wait() [0x00000000428df000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)

  • locked <0x0000000266f41a10> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
    at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=10 tid=0x0000000057e46800 nid=0x5d82 in Object.wait() [0x00000000427de000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)

  • locked <0x00000002670d1590> (a java.lang.ref.Reference$Lock)

"VM Thread" prio=10 tid=0x0000000057e40000 nid=0x5d81 runnable

"GC task thread#0 (ParallelGC)" prio=10 tid=0x0000000057ded800 nid=0x5d79 runnable

"GC task thread#1 (ParallelGC)" prio=10 tid=0x0000000057def000 nid=0x5d7a runnable

"GC task thread#2 (ParallelGC)" prio=10 tid=0x0000000057df1000 nid=0x5d7b runnable

"GC task thread#3 (ParallelGC)" prio=10 tid=0x0000000057df3000 nid=0x5d7c runnable

"GC task thread#4 (ParallelGC)" prio=10 tid=0x0000000057df4800 nid=0x5d7d runnable

"GC task thread#5 (ParallelGC)" prio=10 tid=0x0000000057df6800 nid=0x5d7e runnable

"GC task thread#6 (ParallelGC)" prio=10 tid=0x0000000057df8800 nid=0x5d7f runnable

"GC task thread#7 (ParallelGC)" prio=10 tid=0x0000000057dfa800 nid=0x5d80 runnable

"VM Periodic Task Thread" prio=10 tid=0x00002aaab4011000 nid=0x5d88 waiting on condition

JNI global references: 1307

Environment

CentOS 2.6.18-194.17.4.el5
java version "1.6.0_29"

Status

Assignee

Unassigned

Reporter

David Pane

Labels

None

Group Assignee

None

ZendeskID

None

Estimated Difficulty

None

Actual Difficulty

None

Affects versions

Heritrix 3.1.0

Priority

Major
Configure