Menu

#14 Muti threaded stack overflow error

v1.0 (example)
pending
1
2018-03-18
2015-08-31
No

Hi,

We are using cb2xml tool for transforming fixed length character to xml and vice versa. In a single threaded environment, cb2xml tool is working fine, Whereas, in a multithreaded environment, when we increase the number of threads from 1 to 5. we are getting stack overflow error i.e Caused by: java.lang.NullPointerException
at net.sf.cb2xml.convert.MainframeToXml.convertNode(Unknown Source). We feel that this error occurs due to recursion used in this code to achieve this functionality. Can you please provide a solution to this issue?

Best Regards
S.Mouttou

Discussion

1 2 > >> (Page 1 of 2)
  • Bruce Martin

    Bruce Martin - 2015-08-31
    • assigned_to: Bruce Martin

    A stack trace would be useful ????.

    Broadly speaking:

    • The Cobol-Copybook --> Document an is not thread safe or possible to make thread save (it uses generated code). A solution is to Convert the Cobol to Xml (when the cobol changes) then load the Xml in your program
    • For the MainframeToXml step I would suggest using one MainframeToXml object per thread (if you are not already doing so).
     

    Last edit: Bruce Martin 2015-08-31
  • Mouttou Coumara Vadivel

    Hi,

    We have tried with synchronized block also in the below code

    new MainframeToXml().convert(content, copyBookXml)

    still, we face the same error.

    stack trace

    at net.sf.cb2xml.convert.MainframeToXml.convertNode(Unknown Source)
    at net.sf.cb2xml.convert.MainframeToXml.convertNode(Unknown Source)
    at net.sf.cb2xml.convert.MainframeToXml.convertNode(Unknown Source)
    at net.sf.cb2xml.convert.MainframeToXml.convertNode(Unknown Source)
    at net.sf.cb2xml.convert.MainframeToXml.convertNode(Unknown Source)
    at net.sf.cb2xml.convert.MainframeToXml.convertNode(Unknown Source)
    Then it shows stackOverflow error.

    Best Regards
    S.Mouttou

     
  • Bruce Martin

    Bruce Martin - 2015-09-07

    Copybook and data would also be useful ???

     
  • Mouttou Coumara Vadivel

    Hi,

    since we are working in a closed environment, it is difficult to share our copybook and the data. We could simulate this error even with 2k data. If possible, can you please provide a iterative code instead of recursive.

    Best Regards
    S.Mouttou

     
  • Bruce Martin

    Bruce Martin - 2015-09-09

    I am willing to do it as part of the JRecord project (using a Stax parser). JRecord is orientated toward processing Cobol-Data-files while cb2xml is more about analysing the Copybook.
    Doing it in JRecord hass both advantages and disadvantages:

    • JRecord can handle more data types than cb2xml (these include binary fields)
    • JRecord can handle more file types (including binaruy format (Fixed-Length Records, VB files).
    • JRecord can handle several Cobol dialects.

    • JRecord Flattens the structure, initial vewrsion would not have "Group" details.

    • Arrays would be a problem

    Also can you provide more details on the copybooks; are they complicated / simple; do they have arrays; is there multiple records ???

    Also you might want to look at Legstar project

     

    Last edit: Bruce Martin 2015-09-09
  • Mouttou Coumara Vadivel

    Hi,

    Whether JRecord has the ability to convert cobol to xml and xml to copy book. If it does, can you please let us know which Java file from your source code does cobol to xml and vice versa as in cb2xml project we have Dat2Xml and Xml2Dat.

    Best Regards
    S.Mouttou

     
  • Bruce Martin

    Bruce Martin - 2015-09-09

    It does not currently have the ability, It would need to be written from scratch

     
  • Mouttou Coumara Vadivel

    Hi,

    INput data:
    2014-12-22 000000 000000AAAAAAAAAAA 000000000000 CCCC 000000 DDDD 000.00YYY 40.0000 777777777 FFFFFFFFFFFF HHHH ZZZZZZZZZZZZ FFFF FFFF 2015-07-28T06:28:21
    <copybook filename="HHHHH.txt">
    <item display-length="4798" level="05" name="AAAA" position="1" storage-length="4798">
    <item display-length="2038" level="10" name="BBBBBBB" position="1" storage-length="2038">
    <item display-length="10" level="15" name="CCCCCCC" picture="X(10)" position="1" storage-length="10">
    <item display-length="10" level="15" name="DDDDDD" picture="X(10)" position="11" storage-length="10">
    <item display-length="840" level="15" name="EEEEEEEE" picture="X(840)" position="21" storage-length="840">
    <item display-length="35" level="15" name="FFFFFF" position="861" storage-length="35">
    <item display-length="5" level="20" name="GGGGGGGGG" picture="X(05)" position="861" storage-length="5" value="spaces">
    <item display-length="30" level="20" name="EEEEEEE" picture="X(30)" position="866" storage-length="30">
    </item>
    <item display-length="1143" level="15" name="FFFFFFF" position="896" storage-length="1143">
    <item display-length="64" level="20" name="GGGGGG" picture="X(64)" position="896" storage-length="64">
    <item display-length="6" level="20" name="HHHHHHHHHH" numeric="true" picture="9(6)" position="960" storage-length="6">
    <item display-length="30" level="20" name="IIIIIIII" picture="X(30)" position="966" storage-length="30">
    <item display-length="1032" level="20" name="JJJJJJJJJ" position="996" storage-length="1032">
    <item display-length="30" level="25" name="KKKKKKK" picture="X(30)" position="996" storage-length="30">
    <item display-length="840" level="25" name="LLLLLLL" picture="X(840)" position="1026" storage-length="840">
    <item display-length="34" level="25" name="MMMMMMMM" picture="X(34)" position="1866" storage-length="34">
    <item display-length="128" level="25" name="NNNNNNNN" picture="X(128)" position="1900" storage-length="128">
    </item>
    <item display-length="5" level="20" name="OOOOOO" picture="X(05)" position="2028" storage-length="5">
    <item display-length="6" level="20" name="PPPPPPPPPPP" numeric="true" picture="9(6)" position="2033" storage-length="6">
    </item>
    </item>
    <item display-length="702" level="10" name="QQQQQQQQ" position="2039" storage-length="702">
    <item display-length="160" level="15" name="RRRRR" position="2039" storage-length="160">
    <item display-length="64" level="20" name="SSSSS" picture="X(64)" position="2039" storage-length="64">
    <item display-length="32" level="20" name="TTTTTTT" position="2103" storage-length="32">
    <item display-length="6" level="25" name="UUUUUU" picture="X(6)" position="2103" storage-length="6">
    <item display-length="6" level="25" name="VVVVVV" picture="X(6)" position="2109" storage-length="6" value="spaces">
    <item display-length="8" level="25" name="WWWWWWWWW" picture="X(8)" position="2115" storage-length="8">
    <item display-length="12" level="25" name="XXXXXXX" picture="X(12)" position="2123" storage-length="12" value="spaces">
    </item>
    <item display-length="32" level="20" name="YYYYYYYYYYYYYYYYY" picture="X(32)" position="2135" storage-length="32">
    <item display-length="32" level="20" name="ZZZZZZZZZZZZZZ" picture="X(32)" position="2167" storage-length="32">
    </item>
    <item display-length="530" level="15" name="AAAAAAAAAAAA1111" position="2199" storage-length="530">
    <item display-length="518" level="20" name="BBBBBBBBBB1111" position="2199" storage-length="518">
    <item display-length="6" level="25" name="CCCCCCCCC111" numeric="true" picture="9(6)" position="2199" storage-length="6">
    <item display-length="12" level="25" name="DDDDDDDDD1111" picture="X(12)" position="2205" storage-length="12">
    <item display-length="500" level="25" name="EEEEEEEEE111" picture="X(500)" position="2217" storage-length="500">
    </item>
    <item display-length="12" level="20" name="FFFFFFF111" picture="X(12)" position="2717" storage-length="12">
    </item>
    <item display-length="12" level="15" name="GGGGGGGG11111" picture="X(12)" position="2729" storage-length="12">
    </item>
    <item display-length="96" level="10" name="HHHH1111" position="2741" storage-length="96">
    <item display-length="30" level="15" name="IIIIII111" picture="X(30)" position="2741" storage-length="30">
    <item display-length="47" level="15" name="JJJJJJJ1111" position="2771" storage-length="47">
    <item display-length="17" insert-decimal-point="true" level="20" name="KKKKKKKK111111" numeric="true" picture="99999999999999.99" position="2771" scale="2" storage-length="17">
    <item display-length="30" level="20" name="LLLLLLLLL1111" picture="X(30)" position="2788" storage-length="30">
    </item>
    <item display-length="19" insert-decimal-point="true" level="15" name="MMMMMMMM1111" numeric="true" picture="99999999999999.9999" position="2818" scale="4" storage-length="19">
    </item>
    <item display-length="698" level="10" name="NNNNNN111" position="2837" storage-length="698">
    <item display-length="658" level="15" name="OOOOOO1111" position="2837" storage-length="658">
    <item display-length="12" level="20" name="PPPPPPP1111" picture="X(12)" position="2837" storage-length="12">
    <item display-length="12" level="20" name="QQQQQQQQ111" picture="X(12)" position="2849" storage-length="12">
    <item display-length="30" level="20" name="RRRRR111" picture="X(30)" position="2861" storage-length="30">
    <item display-length="12" level="20" name="SSSSS111" picture="X(12)" position="2891" storage-length="12">
    <item display-length="32" level="20" name="TTTTT111" picture="X(32)" position="2903" storage-length="32">
    <item display-length="12" level="20" name="UUUU111" picture="X(12)" position="2935" storage-length="12">
    <item display-length="12" level="20" name="VVVVV111" picture="X(12)" position="2947" storage-length="12">
    <item display-length="12" level="20" name="WWWWW111" picture="X(12)" position="2959" storage-length="12">
    <item display-length="12" level="20" name="XXXX1111" picture="X(12)" position="2971" storage-length="12">
    <item display-length="12" level="20" name="YYYY111" picture="X(12)" position="2983" storage-length="12">
    <item display-length="500" level="20" name="ZZZZ111" position="2995" storage-length="500">
    <item display-length="500" level="25" name="AAAAA2222" picture="X(500)" position="2995" storage-length="500">
    </item>
    </item>
    <item display-length="10" level="15" name="B2222" picture="X(10)" position="3495" storage-length="10">
    <item display-length="30" level="15" name="CCCC2222" picture="X(30)" position="3505" storage-length="30" value="spaces">
    </item>
    <item display-length="1259" level="10" name="DDDD2222" position="3535" storage-length="1259">
    <item display-length="64" level="15" name="EEEEE2222" picture="X(64)" position="3535" storage-length="64">
    <item display-length="64" level="15" name="FFFFF222" picture="X(64)" position="3599" storage-length="64">
    <item display-length="179" level="15" name="GGGGG2222" position="3663" storage-length="179">
    <item display-length="160" level="20" name="HHHHH222" position="3663" storage-length="160">
    <item display-length="64" level="25" name="IIII222" picture="X(64)" position="3663" storage-length="64">
    <item display-length="32" level="25" name="JJJJ222" picture="X(32)" position="3727" storage-length="32">
    <item display-length="32" level="25" name="KKKKK2222" picture="X(32)" position="3759" storage-length="32">
    <item display-length="32" level="25" name="LLLLLL2222" picture="X(32)" position="3791" storage-length="32">
    </item>
    <item display-length="19" level="20" name="MMMMMMM2222" picture="X(19)" position="3823" storage-length="19">
    </item>
    <item display-length="100" level="15" name="NNNN222" picture="X(100)" position="3842" storage-length="100">
    <item display-length="840" level="15" name="OOOOOO222" picture="X(840)" position="3942" storage-length="840">
    <item display-length="12" level="15" name="PPPP222" picture="X(12)" position="4782" storage-length="12">
    </item>
    <item display-length="5" level="10" name="QQQ222" picture="X(05)" position="4794" storage-length="5" value="spaces">
    </item>
    </item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></item></copybook>

     
  • Bruce Martin

    Bruce Martin - 2015-09-10

    I will start looking at this shortly

     
  • Mouttou Coumara Vadivel

    Any updates on this?

     
  • Bruce Martin

    Bruce Martin - 2015-09-18

    I should have a JRecord based version out next week (monday Tuesday ???)

     
  • Bruce Martin

    Bruce Martin - 2015-09-22

    I have released JRecord 0.80.8f (sourceforge.net)
    which has 2 replacement program Data2Xml and Xml2Data. Both programs need 3 jars:

    • cb2xml.jar
    • JRecord.jar
    • Cobol2Xml.jar

    There are 2 example bat files data2xml.bat and xml2data.bat.


    Now there are differences between the dat2xml and xml2dat programs in cb2xml:

    • For String variables, trailing spaces are dropped
    • For numeric variable leading / trailing spaces are dropped + Leading zero's are dropped
    • By default The main Xml tag is coboldata (instead of copybook). There is an option to
      change this.
    • <tag></tag> is changed to <tag/>. You will also find <tag>        </tag> is changed to <tag/>

    But the new programs should run with ** very large** data files - records are read and processed one at a time (as apposed to loading and processing the file.
    I have tested Data2Xml with 90mb input file.

    The new programs can be used like:

    Data2Xml.bat -cobol CobolData2Xml/DTAR020.cbl -font cp037 -fileOrganisation FixedWidth -input CobolData2Xml/in/DTAR020.bin -output CobolData2Xml/out/DTAR020_A.xml
    

    You can also use it in Java programs like:

    Cobol2Xml.newCobol2Xml(opts.cobolCopybook)
                    .setFont(opts.font)
                    .setFileOrganization(opts.fileOrganisation)
                    .setDialect(opts.dialect)
                    .setDropCopybookNameFromFields(opts.dropCopybookName)
                    .setXmlMainElement(opts.mainXmlTag)
               .cobol2xml(opts.inputFile, opts.outputFile);
    
     

    Last edit: Bruce Martin 2015-09-22
  • Mouttou Coumara Vadivel

    Hi,

    Thanks for this new tool. We have tried with comp data type. while we run we are getting the following exception. We have .setFileOrganization(Constants.IO_STANDARD_TEXT_FILE)

    Caused by: java.lang.RuntimeException: This is a binary Layout (Schema) but you have selected a Character file Organisation, you must use Byte based Organisation?.

    Can you please let us know how to set Byte based Organisation?.

    Best Regards
    S.Mouttou

     
  • Bruce Martin

    Bruce Martin - 2015-09-24

    I put a test in to stop people reading a binary files as Text file, because it is not reliable.

    Reasons:

    • using Constants.IO_STANDARD_TEXT_FILE, JRecord will read the file as Character's,
      the binary types require bytes. The byte ==> character ==> byte can corrupt the data.

    • If data originated on the mainframe, the Ebcidic ==> Ascii will corrupt binary fields
      e.g. 0x40 is a perfectly valid Binary/Packed-Decimal value but it also happens to be the Ebcdic
      space character and will be translated to 0x20 in ascii.

    • Finally the Carriage return and line feed characters 10 and 13 (decimal) again are perfectly valid
      in binary fields - how do you decide wether to treat a decimal 10/13 as a end of line or not.


    Where does the data come from ???? is it the mainframe ???

    For the mainframe, I would suggest:

    1. Make sure it is a FB file on the mainframe
    2. Do a raw/binary file transfer (keeping the ebcidic)
    3. Specify File-Structure=Fixed and the apropriate Font (IBM037 for US EBCIDIC,
      other languages have there own EBCIDIC code (e.g. IBM273 for German).

    For other Binary Cobol files again I suggest using FB (or fixed)


    Also are you going to want to process Multi-Record files ???; At the moment that can not be done

     

    Last edit: Bruce Martin 2015-09-24
  • Mouttou Coumara Vadivel

    Hi,

    <copybook filename="cbl2xml_Test102.cbl">
    <item display-length="173" level="01" name="Ams-Vendor" position="1" storage-length="173">
    <item display-length="3" level="03" name="Brand" picture="x(3)" position="1" storage-length="3">
    <item display-length="4" level="03" name="Location-Number" numeric="true" picture="9(4)" position="4" storage-length="4">
    <item display-length="2" level="03" name="Location-Type" picture="XX" position="8" storage-length="2">
    <item display-length="35" level="03" name="Location-Name" picture="X(35)" position="10" storage-length="35">
    <item display-length="40" level="03" name="Address-1" picture="X(40)" position="45" storage-length="40">
    <item display-length="40" level="03" name="Address-2" picture="X(40)" position="85" storage-length="40">
    <item display-length="35" level="03" name="Address-3" picture="X(35)" position="125" storage-length="35">
    <item display-length="4" level="03" name="Postcode" numeric="true" picture="9(4)" position="160" storage-length="4">
    <item display-length="6" level="03" name="Empty" picture="X(6)" position="164" storage-length="6">
    <item display-length="3" level="03" name="State" picture="XXX" position="170" storage-length="3">
    <item display-length="1" level="03" name="Location-Active" picture="X" position="173" storage-length="1">
    </item>
    </item></item></item></item></item></item></item></item></item></item></item></copybook>

    <copybook>
    <ams-vendor>
    <brand>TAR</brand>
    <location-number>-839</location-number>
    <location-type>DC</location-type>
    <location-name>DC - Taras Ave</location-name>
    <address-1></address-1>
    <address-2>30-68 Taras Ave</address-2>
    <address-3>Altona North</address-3>
    <postcode>3025</postcode>
    <empty></empty>
    <state>VIC</state>
    <location-active>A</location-active>
    </ams-vendor>
    </copybook>

    Based on the above, configuration, When I send negative value for xml2data.
    net.sf.JRecord.Common.RecordException: Only positive numbers are allowed
    at net.sf.JRecord.Types.TypeNum.checkValue(TypeNum.java:450)
    at net.sf.JRecord.Types.TypeNum.setField(TypeNum.java:273)
    at net.sf.JRecord.Details.Line.setField(Line.java:398)
    at net.sf.JRecord.Details.BasicLine.setField(BasicLine.java:193)
    at net.sf.JRecord.Details.FieldValue.set(FieldValue.java:105)
    at net.sf.JRecord.cbl2xml.impl.Cobol2GroupXml.xml2Cobol(Cobol2GroupXml.java:346)
    at net.sf.JRecord.cbl2xml.XmlToData.xml2data(XmlToData.java:50)
    at net.sf.JRecord.cbl2xml.XmlToData.main(XmlToData.java:21)

    We need negative values need to be sent.
    
    Best Regards
    S.Mouttou
    
     
    • Bruce Martin

      Bruce Martin - 2015-09-30

      Try changing the Cobol-Copybook;

      05 numeric-field              pic 9(4).
      

      is a positive field. You need to talk to the Cobol Programmers to find
      out what they are expecting. You might want to change to one of the following

      05 numeric-field              pic s9(4).
      
      05 numeric-field              pic -9(4).
      
      05 numeric-field              pic -9(3).
      

      Each has its advantages and disadvantages:

      • -9(4) / - 9(3) will require the Cobol programmers to check for the sign.
        is actual a character field and would need to be converted to a numeric field
      • s9(4) is a proper Cobol numeric field and if you are using a PC-Cobol or US - EBCDIC (IBM037 / CP037)
        there will be no problems. But for other EBCDIC (e.g. German IBM273 there are issues you need to be aware of).

      • -0 and +0 are represented by different characters !!! in different versions of EBCIDIC. For US EBCDIC they are {} while in German Ebcdic it is äü.
        There is 2 solutions:

      • There is a Method Conversion.setDefaultEbcidicCharacterset for Setting the default EBCDIC character set

      • Do the conversion two EBCDIC in JRecord

      You will find that for the mainframe:

                           +---------  Representation ------------+
                 Value     9(4)      s9(4) (cp037)    s9(4) (CP273)
                   123     0123      012c              012c
                   100     0100      010{              010ä
                  -100               010}              010ü
      
       
  • Mouttou Coumara Vadivel

    Hi,

    We are facing the following issue in the occurs clause for xml to copy book conversion.

    Exception in thread "main" java.lang.RuntimeException: Field: "Address-1" does not exist !!!
    at net.sf.JRecord.Details.BaseLine.getFieldValue(BaseLine.java:49)
    at net.sf.JRecord.Details.BaseLine.getFieldValue(BaseLine.java:5)
    at net.sf.JRecord.cbl2xml.impl.Cobol2GroupXml.xml2Cobol(Cobol2GroupXml.java:338)
    at net.sf.JRecord.cbl2xml.XmlToData.xml2data(XmlToData.java:50)
    at net.sf.JRecord.cbl2xml.XmlToData.main(XmlToData.java:21)

    Copy Book:

    <copybook filename="cbl2xml_Test102.cbl">
    <item display-length="289" level="01" name="Ams-Vendor" position="1" storage-length="289">
    <item display-length="9" level="03" name="Brand" numeric="true" picture="s9(9)" position="1" signed="true" storage-length="4" usage="computational">
    <item display-length="4" level="03" name="Location-Number" numeric="true" picture="9(4)" position="5" storage-length="4">
    <item display-length="2" level="03" name="Location-Type" picture="XX" position="9" storage-length="2">
    <item display-length="35" level="03" name="Location-Name" picture="X(35)" position="11" storage-length="35">
    <item display-length="115" level="03" name="Address" occurs="2" position="46" storage-length="115">
    <item display-length="40" level="04" name="Address-1" picture="X(40)" position="46" storage-length="40">
    <item display-length="40" level="04" name="Address-2" picture="X(40)" position="86" storage-length="40">
    <item display-length="35" level="04" name="Address-3" picture="X(35)" position="126" storage-length="35">
    </item>
    <item display-length="4" level="03" name="Postcode" numeric="true" picture="9(4)" position="276" storage-length="4">
    <item display-length="6" level="03" name="Empty" picture="X(6)" position="280" storage-length="6">
    <item display-length="3" level="03" name="State" picture="XXX" position="286" storage-length="3">
    <item display-length="1" level="03" name="Location-Active" picture="X" position="289" storage-length="1">
    </item>
    </item></item></item></item></item></item></item></item></item></item></item></copybook>

    Sample Data

    <copybook>
    <ams-vendor>
    <brand>1413567029</brand>
    <location-number>839</location-number>
    <location-type>CD</location-type>
    <location-name>C - Taras Ave N</location-name>
    </ams-vendor></copybook>


    <address-1>ew Tars 3</address-1>
    <address-2>0-68 Taras Ave A</address-2>
    <address-3>ltona North N</address-3>


    <address-1>ew Tars 3</address-1>
    <address-2>0-68 Taras Ave A</address-2>
    <address-3>ltona North 3</address-3>

    <postcode>25</postcode>
    <empty></empty>
    <state>VIC</state>
    <location-active>A</location-active>

    Best Regards
    S.Mouttou

     
    • Bruce Martin

      Bruce Martin - 2015-09-30

      Array processing has not been implemented in Xml2Datayet (it also not in cb2xml's Xml2Dat). I will look at implementing it.

       
  • Mouttou Coumara Vadivel

    Hi,

    Thanks for your response. We are heavily depending on array processing. It would be great and appreciated, if it gets implemented in few days. Thanks again for your effort.

    Best Regards
    S.Mouttou

     
  • Mouttou Coumara Vadivel

    Hi,

    We have one more requirement.
    Only for numeric fields, data should be right aligned. This is required only for xmltocopybook conversion not for copybookToXml.

    Best Regards
    S.Mouttou

     
    • Bruce Martin

      Bruce Martin - 2015-10-01

      The new JRecord: Xml2Data should already do this (for the most part anyway), it has a better understanding of Cobol Data Types than Cb2Xml. That is why you got the error when you assigned a negative number to a "positive" field.
      The old xml2dat treated everything as Text.

      For the new Xml2Data if you define a field as 9(4), it should right align and zero fill. There is a grey area - Editted-Numeric (e.g. ---9) traditionally this is actually text field and mostly used for reports (or transfering data to other systems). In this case it may not be right alligned, traditionally it would need to be decoded in Cobol although sop compilers will treat it as numeric

       
  • Mouttou Coumara Vadivel

    Hi,

    It seems like our backend team handling this. Hence, alignment need not to be handled in JRecord.

    Best Regards
    S.Mouttou

     
  • Mouttou Coumara Vadivel

    Hi,

    Thanks for let us know about alignment handling on new Xml2Data in JRecord. As we request earlier, once you finished array handling, please let us know.

    Best Regards
    S.Mouttou

     
1 2 > >> (Page 1 of 2)

Log in to post a comment.

MongoDB Logo MongoDB