visit: Performance slow down with repeated rendering

A user sent a email to visitusers detailing an issue where the rendering would slow down as he progressed through a time sequence. He created a script that replicated the behavior, just replotting the same plot over and over from the same file. I have attached his e-mail:

Hi, I am running VisIt 2.7.3 on a 64-bit Linux server with the kernel release 2.6.18, compiled using

build_visit2_7_3 console thirdpartypath 3rdparty/ parallel python ccmio cfitsio cgns gdal hdf5 netcdf cmake icet nomesa qt silo szip vtk makeflags 'j 6'

Maybe I should add that it is not a clientserver install, only a regular compilation on the server to which I ssh -X and where I run the script with visit nowin cli -s script.py. I attach a stripped down version of the script (using a single file instead of extracting a new silo file at each loop iteration and not setting any plot options) which still went down from 20 pngs/min to 3/min output speed in 2 hours before I stopped it. I hope this helps, let me know if you need more info. Is this a similar type of case than J.L. Simon’s original post? Best regards,Yann

On 07/15/2014 07:34 PM, Brad Whitlock wrote: Unless the geometry produced by the plots gets wildly more complex over time, it sounds like your combination of plots and operators is resulting in a memory leak. The only user workaround for that is to periodically close the compute engine and start it up again via scripting. However, that should not be necessary as the leak should not exist. To fix problems like this, it would be helpful to know the version/platform of VisIt being used and for you to share a version of your script that reproduces the problem.

On Tue, Jul 15, 2014 at 4:52 AM, Jrgen Lorenz Simon juergen_simon@mac.com wrote: Hello again, once more I am availing myself of the lists excellent know-how to solve the following problem: I am running pretty complex pythons scripts in Visit to visualize large data sets. The sets contain about 300 data files (NetCDF) each and process about 5 runs in parallel. This means, that I have bash scripts that start 5 instances of Visit from the command line, each with a script. I observe, that the processing time per data set is a function resembling a 1exp(x) function. At first the processing goes reasonably fast (510 seconds), but very soon the performance starts to sag. In end end, it takes about 30 mins to 1 hour to process one file. I am taking care to close all databases I open and I’m reasonably sure the python code does not produce any memory leaks. Perhaps I am going on about the problem in the wrong way. Any ideas on how to improve my predicament? Kind Regards, J.Simon

His script:

from subprocess import Popen
cpt = 1
total = 10000
for index in range(0,total):
    print cpt, "/" , total
    cpt = cpt+1
    fileLocation="/data/kempf/mosaic/velgrid.rotated.123456.0000002.silo"
    OpenDatabase(fileLocation, 0)
    AddPlot("Contour", "avgs", 1, 1)
    ContourAtts = ContourAttributes()
    #setting options...
    SetPlotOptions(ContourAtts)
    DrawPlots()
    View3DAtts = View3DAttributes()
    #setting attributes...
    SetView3D(View3DAtts)
    ViewAxisArrayAtts = ViewAxisArrayAttributes()
    #setting attributes...
    SetViewAxisArray(ViewAxisArrayAtts)
    SaveWindowAtts = SaveWindowAttributes()
    SaveWindowAtts.outputToCurrentDirectory = 0
    SaveWindowAtts.outputDirectory = "/data/kempf/mosaic/backbone_test"
    SaveWindowAtts.fileName = "name_"+str(index).rjust(7, '0')
    SaveWindowAtts.family = 0
    SaveWindowAtts.format = SaveWindowAtts.PNG # BMP, CURVE, JPEG, OBJ, PNG, POSTSCRIPT, POVRAY, PPM, RGB, STL, TIFF, ULTRA, VTK, PLY
    SaveWindowAtts.width = 1024
    SaveWindowAtts.height = 1024
    SaveWindowAtts.screenCapture = 0
    SaveWindowAtts.saveTiled = 0
    SaveWindowAtts.quality = 80
    SaveWindowAtts.progressive = 0
    SaveWindowAtts.binary = 0
    SaveWindowAtts.stereo = 0
    SaveWindowAtts.compression = SaveWindowAtts.PackBits # None, PackBits, Jpeg, Deflate
    SaveWindowAtts.forceMerge = 0
    SaveWindowAtts.resConstraint = SaveWindowAtts.ScreenProportions # NoConstraint, EqualWidthHeight, ScreenProportions
    SaveWindowAtts.advancedMultiWindowSave = 0
    SetSaveWindowAttributes(SaveWindowAtts)
    SaveWindow()
    DeleteAllPlots()
    ResetView()
    CloseDatabase(fileLocation)
exit()

-----------------------REDMINE MIGRATION----------------------- This ticket was migrated from Redmine. As such, not all information was able to be captured in the transition. Below is a complete record of the original redmine ticket.

Ticket number: 1917 Status: Pending Project: VisIt Tracker: Bug Priority: High Subject: Performance slow down with repeated rendering Assigned to: - Category: - Target version: - Author: Eric Brugger Start: 07/22/2014 Due date: % Done: 0% Estimated time: Created: 07/22/2014 05:25 pm Updated: 08/26/2014 04:28 pm Likelihood: 3 - Occasional Severity: 3 - Major Irritation Found in version: 2.7.3 Impact: Expected Use: OS: All Support Group: Any Description: A user sent a email to visitusers detailing an issue where the rendering would slow down as he progressed through a time sequence. He created a script that replicated the behavior, just replotting the same plot over and over from the same file. I have attached his e-mail. Hi, I am running VisIt 2.7.3 on a 64-bit Linux server with the kernel release 2.6.18, compiled using build_visit2_7_3 console thirdpartypath 3rdparty/ parallel python ccmio cfitsio cgns gdal hdf5 netcdf cmake icet nomesa qt silo szip vtk makeflags 'j 6’Maybe I should add that it is not a clientserver install, only a regular compilation on the server to which I ssx -X and where I run the script with visit nowin cli -s script.pyI attach a stripped down version of the script (using a single file instead of extracting a new silo file at each loop iteration and not setting any plot options) which still went down from 20 pngs/min to 3/min output speed in 2 hours before I stopped it. I hope this helps, let me know if you need more info. Is this a similar type of case than J.L. Simon’s original post? Best regards,Yann On 07/15/2014 07:34 PM, Brad Whitlock wrote:Unless the geometry produced by the plots gets wildly more complex over time, it sounds like your combination of plots and operators is resulting in a memory leak. The only user workaround for that is to periodically close the compute engine and start it up again via scripting. However, that should not be necessary as the leak should not exist. To fix problems like this, it would be helpful to know the version/platform of VisIt being used and for you to share a version of your script that reproduces the problem.Brad On Tue, Jul 15, 2014 at 4:52 AM, Jrgen Lorenz Simon juergen_simon@mac.com wrote:Hello again, once more I am availing myself of the listss excellent know-how to solve the following problem: I am running pretty complex pythons scripts in Visit to visualize large data sets. The sets contain about 300 data files (NetCDF) each and process about 5 runs in parallel. This means, that I have bash scripts that start 5 instances of Visit from the command line, each with a script. I observe, that the processing time per data set is a function resembling a 1exp(x) function. At first the processing goes reasonably fast (510 seconds), but very soon the performance starts to sag. In end end, it takes about 30 mins to 1 hour to process one file. I am taking care to close all databases I open and Im reasonably sure the python code does not produce any memory leaks. Perhaps I am going on about the problem in the wrong way. Any ideas on how to improve my predicament? Kind Regards,J.SimonVisIt Users Wiki: http://visitusers.org/Frequently Asked Questions for VisIt: http://visit.llnl.gov/FAQ.htmlTo Unsubscribe: send a blank email to visitusersunsubscribe@elist.ornl.govMore Options: https://elist.ornl.gov/mailman/listinfo/visit-users His script: from subprocess import Popen cpt = 1total = 10000for index in range(0,total): print cpt, “/” , total cpt = cpt+1 fileLocation=“/data/kempf/mosaic/velgrid.rotated.123456.0000002.silo” OpenDatabase(fileLocation, 0) AddPlot(“Contour”, “avgs”, 1, 1) ContourAtts = ContourAttributes() setting options SetPlotOptions(ContourAtts) DrawPlots() View3DAtts = View3DAttributes() setting attributes SetView3D(View3DAtts) ViewAxisArrayAtts = ViewAxisArrayAttributes() #setting attributes SetViewAxisArray(ViewAxisArrayAtts) SaveWindowAtts = SaveWindowAttributes() SaveWindowAtts.outputToCurrentDirectory = 0 SaveWindowAtts.outputDirectory = “/data/kempf/mosaic/backbone_test” SaveWindowAtts.fileName = “name_”+str(index).rjust(7, ‘0’) SaveWindowAtts.family = 0 SaveWindowAtts.format = SaveWindowAtts.PNG # BMP, CURVE, JPEG, OBJ, PNG, POSTSCRIPT, POVRAY, PPM, RGB, STL, TIFF, ULTRA, VTK, PLY SaveWindowAtts.width = 1024 SaveWindowAtts.height = 1024 SaveWindowAtts.screenCapture = 0 SaveWindowAtts.saveTiled = 0 SaveWindowAtts.quality = 80 SaveWindowAtts.progressive = 0 SaveWindowAtts.binary = 0 SaveWindowAtts.stereo = 0 SaveWindowAtts.compression = SaveWindowAtts.PackBits # None, PackBits, Jpeg, Deflate SaveWindowAtts.forceMerge = 0 SaveWindowAtts.resConstraint = SaveWindowAtts.ScreenProportions # NoConstraint, EqualWidthHeight, ScreenProportions SaveWindowAtts.advancedMultiWindowSave = 0 SetSaveWindowAttributes(SaveWindowAtts) SaveWindow() DeleteAllPlots() ResetView() CloseDatabase(fileLocation) exit()

Comments: