Problem Note 53653: The IBM WebSphere Java virtual machine (JVM) SASServer1 stops responding during invocation
 |  |  |  |
The SASServer1 JVM, stops responding during invocation when the SAS® Content Server Repository task is initializing. Based on analysis of the Java thread dump and the WebSphere SystemOut.log file, the failure occurs specifically when the PDFBox Extractor CMapParser task runs.
The contents of the SystemOut.log file and the JVM thread dump are shown below.
SystemOut.log File Contents
[6/6/14 2:02:59:996 GMT+05:30] 0000001e ThreadMonitor W WSVR0605W: Thread "server.startup : 1" (00000010) has been active for 710105 milliseconds and may be hung. There is/are 2 thread(s) in total in the server that may be hung. at java.util.HashMap.put(HashMap.java:607) at org.pdfbox.cmaptypes.CMap.addMapping(CMap.java:126) at org.pdfbox.cmapparser.CMapParser.parse(CMapParser.java:105) at org.pdfbox.pdmodel.font.PDFont.encode(PDFont.java:326) at org.pdfbox.util.PDFStreamEngine.showString(PDFStreamEngine.java:174) at org.pdfbox.util.PDFTextStripper.showString(PDFTextStripper.java:461) at org.pdfbox.util.PDFStreamEngine.processOperator(PDFStreamEngine.java:722) at org.pdfbox.util.PDFStreamEngine.processStream(PDFStreamEngine.java:128) at org.pdfbox.util.PDFTextStripper.processPage(PDFTextStripper.java:268) at org.pdfbox.util.PDFTextStripper.processPages(PDFTextStripper.java:230) at org.pdfbox.util.PDFTextStripper.writeText(PDFTextStripper.java:172) at org.apache.jackrabbit.extractor.PdfTextExtractor.extractText(PdfTextExtractor.java:75) at org.apache.jackrabbit.extractor.CompositeTextExtractor.extractText(CompositeTextExtractor.java:90) at org.apache.jackrabbit.core.query.lucene.JackrabbitTextExtractor.extractText(JackrabbitTextExtractor.java:195) at org.apache.jackrabbit.core.query.lucene.TextExtractorJob$1.call(TextExtractorJob.java:91) at EDU.oswego.cs.dl.util.concurrent.FutureResult$1.run(FutureResult.java:75) at org.apache.jackrabbit.core.query.lucene.TextExtractorJob.run(TextExtractorJob.java:170) at EDU.oswego.cs.dl.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at org.apache.jackrabbit.core.query.lucene.TextExtractorReader.read(TextExtractorReader.java:107) at java.io.Reader.read(Reader.java:134) at org.apache.jackrabbit.core.query.lucene.NodeIndexer.createFulltextField(NodeIndexer.java:671) at org.apache.jackrabbit.core.query.lucene.NodeIndexer.addBinaryValue(NodeIndexer.java:394) at org.apache.jackrabbit.core.query.lucene.NodeIndexer.addValue(NodeIndexer.java:282) at org.apache.jackrabbit.core.query.lucene.NodeIndexer.createDoc(NodeIndexer.java:221) at org.apache.jackrabbit.core.query.lucene.SearchIndex.createDocument(SearchIndex.java:898) at org.apache.jackrabbit.core.query.lucene.MultiIndex.createDocument(MultiIndex.java:817) at org.apache.jackrabbit.core.query.lucene.MultiIndex.createDocument(MultiIndex.java:832) at org.apache.jackrabbit.core.query.lucene.MultiIndex$AddNode.execute(MultiIndex.java:1562) at org.apache.jackrabbit.core.query.lucene.Recovery.run(Recovery.java:167) at org.apache.jackrabbit.core.query.lucene.Recovery.run(Recovery.java:85) at org.apache.jackrabbit.core.query.lucene.MultiIndex.(MultiIndex.java:315) at org.apache.jackrabbit.core.query.lucene.SearchIndex.doInit(SearchIndex.java:453) at org.apache.jackrabbit.core.query.AbstractQueryHandler.init(AbstractQueryHandler.java:53) at org.apache.jackrabbit.core.SearchManager.initializeQueryHandler(SearchManager.java:582) at org.apache.jackrabbit.core.SearchManager. (SearchManager.java:264) at org.apache.jackrabbit.core.RepositoryImpl$WorkspaceInfo.getSearchManager(RepositoryImpl.java:1652) at org.apache.jackrabbit.core.RepositoryImpl.initWorkspace(RepositoryImpl.java:639) at org.apache.jackrabbit.core.RepositoryImpl.initStartupWorkspace
The JVM Thread-Dump Contents
3LKMONOBJECT com/ibm/ws/webcontainer/component/WebContainerImpl@0x0000000486841488/0x0000000486841494: Flat locked by "server.startup : 2" (0x0000000031C1BA00), entry count 1
3LKWAITERQ Waiting to enter:
3LKWAITER "server.startup : 0" (0x00000000317A3800)
3LKWAITER "server.startup : 1" (0x0000000031C1AA00)
3XMTHREADINFO "server.startup : 2" J9VMThread:0x0000000031C1BA00, j9thread_t:0x000000012ACD3220, java/lang/Thread:0x0000000487F5A0E0, state:CW, prio=5
3XMTHREADINFO1 (native thread ID:0x148022B, native priority:0x5, native policy:UNKNOWN)
3XMTHREADINFO3 Java callstack:
4XESTACKTRACE at java/lang/String.(String.java:265(Compiled Code))
4XESTACKTRACE at org/pdfbox/cmapparser/CMapParser.parse(CMapParser.java:95(Compiled Code))
4XESTACKTRACE at org/pdfbox/pdmodel/font/PDFont.encode(PDFont.java:326)
4XESTACKTRACE at org/pdfbox/util/PDFStreamEngine.showString(PDFStreamEngine.java:174)
4XESTACKTRACE at org/pdfbox/util/PDFTextStripper.showString(PDFTextStripper.java:461)
4XESTACKTRACE at org/pdfbox/util/PDFStreamEngine.processOperator(PDFStreamEngine.java:722)
4XESTACKTRACE at org/pdfbox/util/PDFStreamEngine.processStream(PDFStreamEngine.java:128)
4XESTACKTRACE at org/pdfbox/util/PDFTextStripper.processPage(PDFTextStripper.java:268)
4XESTACKTRACE at org/pdfbox/util/PDFTextStripper.processPages(PDFTextStripper.java:230)
4XESTACKTRACE at org/pdfbox/util/PDFTextStripper.writeText(PDFTextStripper.java:172)
4XESTACKTRACE at org/apache/jackrabbit/extractor/PdfTextExtractor.extractText(PdfTextExtractor.java:75)
4XESTACKTRACE at org/apache/jackrabbit/extractor/CompositeTextExtractor.extractText(CompositeTextExtractor.java:90)
4XESTACKTRACE at org/apache/jackrabbit/core/query/lucene/JackrabbitTextExtractor.extractText(JackrabbitTextExtractor.java:195)
4XESTACKTRACE at org/apache/jackrabbit/core/query/lucene/TextExtractorJob$1.call(TextExtractorJob.java:91)
4XESTACKTRACE at EDU/oswego/cs/dl/util/concurrent/FutureResult$1.run(FutureResult.java:75)
4XESTACKTRACE at org/apache/jackrabbit/core/query/lucene/TextExtractorJob.run(TextExtractorJob.java:170)
4XESTACKTRACE at EDU/oswego/cs/dl/util/concurrent/DirectExecutor.execute(DirectExecutor.java:30)
4XESTACKTRACE at org/apache/jackrabbit/core/query/lucene/TextExtractorReader.read(TextExtractorReader.java:107)
4XESTACKTRACE at java/io/Reader.read(Reader.java:134(Compiled Code))
4XESTACKTRACE at org/apache/jackrabbit/core/query/lucene/NodeIndexer.createFulltextField(NodeIndexer.java:671)
4XESTACKTRACE at org/apache/jackrabbit/core/query/lucene/NodeIndexer.addBinaryValue(NodeIndexer.java:394)
4XESTACKTRACE at org/apache/jackrabbit/core/query/lucene/NodeIndexer.addValue(NodeIndexer.java:282)
4XESTACKTRACE at org/apache/jackrabbit/core/query/lucene/NodeIndexer.createDoc(NodeIndexer.java:221)
4XESTACKTRACE at org/apache/jackrabbit/core/query/lucene/SearchIndex.createDocument(SearchIndex.java:898)
4XESTACKTRACE at org/apache/jackrabbit/core/query/lucene/MultiIndex.createDocument(MultiIndex.java:817)
4XESTACKTRACE at org/apache/jackrabbit/core/query/lucene/MultiIndex.createDocument(MultiIndex.java:832)
4XESTACKTRACE at org/apache/jackrabbit/core/query/lucene/MultiIndex$AddNode.execute(MultiIndex.java:1562)
4XESTACKTRACE at org/apache/jackrabbit/core/query/lucene/Recovery.run(Recovery.java:167)
4XESTACKTRACE at org/apache/jackrabbit/core/query/lucene/Recovery.run(Recovery.java:85)
4XESTACKTRACE at org/apache/jackrabbit/core/query/lucene/MultiIndex.(MultiIndex.java:315)
4XESTACKTRACE at org/apache/jackrabbit/core/query/lucene/SearchIndex.doInit(SearchIndex.java:453)
4XESTACKTRACE at org/apache/jackrabbit/core/query/AbstractQueryHandler.init(AbstractQueryHandler.java:53)
4XESTACKTRACE at org/apache/jackrabbit/core/SearchManager.initializeQueryHandler(SearchManager.java:582)
4XESTACKTRACE at org/apache/jackrabbit/core/SearchManager.(SearchManager.java:264)
4XESTACKTRACE at org/apache/jackrabbit/core/RepositoryImpl$WorkspaceInfo.getSearchManager(RepositoryImpl.java:1652)
4XESTACKTRACE at org/apache/jackrabbit/core/RepositoryImpl.initWorkspace(RepositoryImpl.java:639)
4XESTACKTRACE at org/apache/jackrabbit/core/RepositoryImpl.initStartupWorkspaces(RepositoryImpl.java:448)
4XESTACKTRACE at org/apache/jackrabbit/core/RepositoryImpl.(RepositoryImpl.java:338)
4XESTACKTRACE at org/apache/jackrabbit/core/RepositoryImpl.create(RepositoryImpl.java:590)
4XESTACKTRACE at org/apache/jackrabbit/j2ee/RepositoryStartupServlet.createRepository(RepositoryStartupServlet.java:450)
4XESTACKTRACE at org/apache/jackrabbit/j2ee/RepositoryStartupServlet.initRepository(RepositoryStartupServlet.java:418)
4XESTACKTRACE at org/apache/jackrabbit/j2ee/RepositoryStartupServlet.startup(RepositoryStartupServlet.java:238)
4XESTACKTRACE at org/apache/jackrabbit/j2ee/RepositoryStartupServlet.init(RepositoryStartupServlet.java:211)
4XESTACKTRACE at javax/servlet/GenericServlet.init(GenericServlet.java:241)
4XESTACKTRACE at com/ibm/ws/webcontainer/servlet/ServletWrapper.init(ServletWrapper.java:358)
4XESTACKTRACE at com/ibm/ws/webcontainer/servlet/ServletWrapperImpl.init(ServletWrapperImpl.java:169)
4XESTACKTRACE at com/ibm/ws/webcontainer/servlet/ServletWrapper.initialize(ServletWrapper.java:1809)
4XESTACKTRACE at com/ibm/wsspi/webcontainer/extension/WebExtensionProcessor.createServletWrapper(WebExtensionProcessor.java:98)
4XESTACKTRACE at com/ibm/ws/webcontainer/webapp/WebApp.getServletWrapper(WebApp.java:1038)
4XESTACKTRACE at com/ibm/ws/webcontainer/webapp/WebApp.getServletWrapper(WebApp.java:959)
4XESTACKTRACE at com/ibm/ws/webcontainer/webapp/WebApp.initializeTargetMappings(WebApp.java:638)
4XESTACKTRACE at com/ibm/ws/webcontainer/webapp/WebApp.commonInitializationFinally(WebApp.java:436)
4XESTACKTRACE at com/ibm/ws/webcontainer/webapp/WebAppImpl.initialize(WebAppImpl.java:304)
Workaround
To solve this problem:
- Make a backup copy of the repository.xml file that resides in the folder SAS–configuration–directory/Lev1/AppData/SASContentServer/Repository/. Be sure to rename the backup copy to something other than repository.xml.
- Locate the following lines in the repository.xml file, as shown below, and remove org.apache.jackrabbit.extractor.PdfTextExtractor, from those lines:
- Make a backup copy of the workspace.xml file that resides in the folder SAS–configuration–directory/Lev1/AppData/SASContentServer/Repository/workspaces/default/. Be sure to rename the backup copy to something other than workspace.xml.
- Locate the following lines in the workspace.xml file, as shown below, and remove org.apache.jackrabbit.extractor.PdfTextExtractor, from those lines:
- Restart the SAS middle-tier services.
param name="textFilterClasses" value="org.apache.jackrabbit.extractor.PlainTextExtractor,org.apache.jackrabbit.extractor.MsWordTextExtractor,org.apache.jackrabbit.extractor.MsExcelTextExtractor,org.apache.jackrabbit.extractor.MsPowerPointTextExtractor,org.apache.jackrabbit.extractor.PdfTextExtractor,org.apache.jackrabbit.extractor.OpenOfficeTextExtractor,org.apache.jackrabbit.extractor.RTFTextExtractor,org.apache.jackrabbit.extractor.HTMLTextExtractor,org.apache.jackrabbit.extractor.XMLTextExtractor"param name="textFilterClasses" value="org.apache.jackrabbit.extractor.PlainTextExtractor,org.apache.jackrabbit.extractor.MsWordTextExtractor,org.apache.jackrabbit.extractor.MsExcelTextExtractor,org.apache.jackrabbit.extractor.MsPowerPointTextExtractor,org.apache.jackrabbit.extractor.PdfTextExtractor,org.apache.jackrabbit.extractor.OpenOfficeTextExtractor,org.apache.jackrabbit.extractor.RTFTextExtractor,org.apache.jackrabbit.extractor.HTMLTextExtractor,org.apache.jackrabbit.extractor.XMLTextExtractor"Operating System and Release Information
| Product Family | Product | System | Product Release | SAS Release | ||
| Reported | Fixed* | Reported | Fixed* | |||
| SAS System | SAS Enterprise BI Server | 64-bit Enabled AIX | 9.3 | 9.3 TS1M2 | ||
| Type: | Problem Note |
| Priority: | medium |
| Date Modified: | 2014-07-22 09:31:27 |
| Date Created: | 2014-07-18 13:46:43 |