Menu ▾ ▴

#9 Extrem bad Performance with large Cobol Copy Book

v1.0_(example)
open
Bruce Martin
None
1
2021-02-15
2017-01-20
Anonymous
No

The import of a Cobol copybook takes 20 seconds on a high-end PC, Java is using one processor core in 100%. Using this defintion with a file, the same happens even with one record. After loading the file every interaction is very slow. You can find attached the the Cobol structure and some data.

Gyoergy Balint
Computer Center of State Austria
gyoergy.balint at
extern.brz.gv.at

1 Attachments
D:\TEMP\BrzRecordEditorBugs.zip

Discussion

  • Bruce Martin

    Bruce Martin - 2017-01-21

    I plan to do full rewrite of how Layouts are store (later this year ???). This rewrite will solve this problem.

    I will also look at some minor changes in the mean time.

    I am assuming you are using the Windows version: In the RecordEditor/Utilities directory there is a Cobol editor. You will find the Cobol editor works better for this copybook (and all copybooks where there are a large arrays). See attached picture. I would suggest using the Cobol editor for Single Record files with large arrays.

    Other versions of the RecordEditor have bat/shell script/java program to run the Cobol editor

    Note for me: This is basically the same issue as reported in https://sourceforge.net/p/record-editor/discussion/467594

     

    Last edit: Bruce Martin 2017-01-21
    Thumbnail
    CobolEditor-or8.png

  • Bruce Martin

    Bruce Martin - 2017-02-08
    • Description has changed:

    Diff:

    --- old
+++ new
@@ -2,4 +2,5 @@

 Gyoergy Balint
 Computer Center of State Austria
-gyoergy.balint@extern.brz.gv.at
+gyoergy.balint at 
+extern.brz.gv.at
     
  • Comment has been marked as spam. 
    Undo

    View and moderate all "bugs Discussion" comments posted by this user

    Mark all as spam, and block user from posting to "Bugs"

    Anonymous

    Anonymous - 2017-02-12

    For you info I made some profiling with mission control and I can tell you which of your statements are burning the CPU.
    The first test was done on Windows, but for the "production" use a pure AIX installation is planned as the files are created by Cobol and PLI for AIX. For security and performance reasons files must not be moved off from the AIX box.

     

  • Bruce Martin

    Bruce Martin - 2017-02-13

    I would be interested to know which statements is burning cpu.

    I am expecting it to be related to DB insert / expanding arrays but it would be good to get it confirmed.

    Some background info when I started the Recordeditor I had no need for arrays
    so did not include them. When I added Cobol I simply converted arrays to individual
    fields. While this works it does not work well for large arrays.

    What I plan to do is implement a more Cobol like Structure (including arrays).
    This should reduce the Copybook-load for your problem copybook by 95% + but to
    do this, I will need to:

    • Write test Code
    • Completely rewrite the import code
    • Change the DB structue.
    • Write DB Conversion code
    • Rewrite the DB Read code
    • Completely rewrite the 'layout edit' screens
    • Rewrite the 'Xml Export'
    • Update the JRecord project

    This will tae a lot of time!!!

    In the mean time some tweaks / change of backend might help.
    I will do anothr release of the RecordEditor-Generic DB (where you vhoose the
    backend DB.

    If you can use the CobolEditor, it should work a lot faster

     

    Last edit: Bruce Martin 2017-02-13

  • Bruce Martin

    Bruce Martin - 2017-02-14

    I made changes that significantly reduce the Copybook load time by using Batch Updates. On my computer the load time fropped from 180+ seconds to couple of seconds.

    What I plan to do

    • by default the existing slow codewill be used
    • if a parameter CommitCount=... (e.g. CommitCount=2000) is added to the Params.Properties file in the RecordEditor lib directory; the new code will be used.
     

  • Bruce Martin

    Bruce Martin - 2017-02-16

    Version 0.98.2 has been released:

    By default the old slow code will be executed. But if you place

    CommitCount=1000
FetchSize=1000

    in the Param.Properties file, the new code will be executed. This file
    will be in the RecordEditor/lib directory. On Windows (using HSQL installer) it will be in:

    • C:\Program Files (x86)\RecordEdit\HSQL\lib

    For linux when using the cross-plaform installer it will be in

    • <home>/RecordEdit/HSQLDB/lib</home>

    I have attached an update Param.Properties

     
    Params.Properties

  • Bruce Martin

    Bruce Martin - 2019-01-19

    With version 0.90 supports editting with a Cobol Copybook. This should be much faster in this case.

     

Anonymous
Anonymous

Add attachments
Cancel