OpenRefine: Show out of memory errors on UI for get-project-metadata, instead of silent spinner

When reporting a bug please provide the following information to help reproduce the bug:

Version of OpenRefine used (Google Refine 2.6, OpenRefine2.8, an other distribution?):

OpenRefine 2.8

Operating Systems and version:

Windows 7 64bit

Browser + version used - Please note that OpenRefine doesn’t support Internet Explorer but works OK in most cases:

Firefox 59.0.1 (64bit)

Steps followed to create the issue:

Open a large project in OpenRefine that has thousands of rows and about 10 columns.

If you are allowed and are OK with making your data public, it would be awesome if you can include the data causing the issue or a URL pointing to where the data is (if your concerned about keeping your data private, ping us on our mailing list):

Current Results:

Forever Spinner on UI

10:04:09.293 [            refine_server] Starting Server bound to '127.0.0.1:333
3' (0ms)
10:04:09.371 [            refine_server] Initializing context: '/' from 'C:\User
s\eguitha\Downloads\openrefine-2.8\webapp' (78ms)
10:04:13.458 [            refine_server] Failed to use jdatapath to detect user
data path: resorting to environment variables (4087ms)
10:04:13.458 [            refine_server] Failed to use jdatapath to detect user
data path: resorting to environment variables (0ms)
10:04:13.489 [                   refine] Starting OpenRefine 2.8 [TRUNK]... (31m
s)
10:04:20.860 [                   refine] POST /command/core/load-language (7371m
s)
10:04:20.906 [                   refine] GET /command/core/get-preference (46ms)

10:04:20.918 [                   refine] POST /command/core/load-language (12ms)

10:04:21.057 [                   refine] POST /command/core/get-importing-config
uration (139ms)
10:04:21.166 [                   refine] GET /command/core/get-all-project-metad
ata (109ms)
10:04:21.358 [                   refine] GET /command/core/get-languages (192ms)

10:04:21.400 [                   refine] GET /command/core/get-version (42ms)
10:04:34.966 [          org.mortbay.log] /favicon.ico (13566ms)
java.lang.IllegalStateException: Committed
        at org.mortbay.jetty.Response.resetBuffer(Response.java:1024)
        at javax.servlet.ServletResponseWrapper.resetBuffer(ServletResponseWrapp
er.java:202)
        at org.mortbay.servlet.GzipFilter$GZIPResponseWrapper.resetBuffer(GzipFi
lter.java:310)
        at org.mortbay.servlet.GzipFilter$GZIPResponseWrapper.sendError(GzipFilt
er.java:319)
        at edu.mit.simile.butterfly.Butterfly.error(Butterfly.java:1020)
        at edu.mit.simile.butterfly.Butterfly.service(Butterfly.java:528)
        at com.google.refine.RefineServlet.service(RefineServlet.java:200)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511
)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1166)
        at org.mortbay.servlet.UserAgentFilter.doFilter(UserAgentFilter.java:81)

        at org.mortbay.servlet.GzipFilter.doFilter(GzipFilter.java:132)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:3
88)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.jav
a:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:1
82)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:7
65)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)

        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:1
52)
        at org.mortbay.jetty.Server.handle(Server.java:326)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:54
2)
        at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpCo
nnection.java:923)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:547)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
        at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.
java:228)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)
10:04:36.422 [                   refine] POST /command/core/load-language (1456m
s)
10:04:36.456 [                   refine] GET /command/core/get-preference (34ms)

10:04:36.473 [                   refine] POST /command/core/load-language (17ms)

10:04:36.551 [                   refine] GET /command/core/get-project-metadata
(78ms)
10:08:37.262 [          org.mortbay.log] Error for /command/core/get-project-met
adata (240711ms)
java.lang.OutOfMemoryError: GC overhead limit exceeded
        at com.fasterxml.jackson.core.JsonFactory._createParser(JsonFactory.java
:1351)
        at com.fasterxml.jackson.core.JsonFactory.createParser(JsonFactory.java:
906)
        at com.fasterxml.jackson.core.JsonFactory.createJsonParser(JsonFactory.j
ava:1108)
        at com.google.refine.model.Row.loadStreaming(Row.java:223)
        at com.google.refine.model.Row.load(Row.java:218)
        at com.google.refine.model.Project.loadFromReader(Project.java:224)
        at com.google.refine.model.Project.loadFromInputStream(Project.java:187)

        at com.google.refine.io.ProjectUtilities.loadFromFile(ProjectUtilities.j
ava:157)
        at com.google.refine.io.ProjectUtilities.load(ProjectUtilities.java:118)

        at com.google.refine.io.FileProjectManager.loadProject(FileProjectManage
r.java:231)
        at com.google.refine.ProjectManager.getProject(ProjectManager.java:482)
        at com.google.refine.commands.Command.getProject(Command.java:181)
        at com.google.refine.commands.project.GetProjectMetadataCommand.doGet(Ge
tProjectMetadataCommand.java:56)
        at com.google.refine.RefineServlet.service(RefineServlet.java:170)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511
)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1166)
        at org.mortbay.servlet.UserAgentFilter.doFilter(UserAgentFilter.java:81)

        at org.mortbay.servlet.GzipFilter.doFilter(GzipFilter.java:132)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:3
88)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.jav
a:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:1
82)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:7
65)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)

        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:1
52)
        at org.mortbay.jetty.Server.handle(Server.java:326)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:54
2)
        at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpCo
nnection.java:923)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:547)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
10:12:49.460 [          org.mortbay.log] /favicon.ico (252198ms)
java.lang.IllegalStateException: Committed
        at org.mortbay.jetty.Response.resetBuffer(Response.java:1024)
        at javax.servlet.ServletResponseWrapper.resetBuffer(ServletResponseWrapp
er.java:202)
        at org.mortbay.servlet.GzipFilter$GZIPResponseWrapper.resetBuffer(GzipFi
lter.java:310)
        at org.mortbay.servlet.GzipFilter$GZIPResponseWrapper.sendError(GzipFilt
er.java:319)
        at edu.mit.simile.butterfly.Butterfly.error(Butterfly.java:1020)
        at edu.mit.simile.butterfly.Butterfly.service(Butterfly.java:528)
        at com.google.refine.RefineServlet.service(RefineServlet.java:200)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511
)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1166)
        at org.mortbay.servlet.UserAgentFilter.doFilter(UserAgentFilter.java:81)

        at org.mortbay.servlet.GzipFilter.doFilter(GzipFilter.java:132)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:3
88)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.jav
a:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:1
82)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:7
65)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)

        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:1
52)
        at org.mortbay.jetty.Server.handle(Server.java:326)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:54
2)
        at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpCo
nnection.java:923)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:547)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
        at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.
java:228)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

Expected Results:

An Error Message popup or banner on the UI saying “Project could not be loaded (possible out of memory)” at this stage

10:08:37.262 [          org.mortbay.log] Error for /command/core/get-project-met
adata (240711ms)
java.lang.OutOfMemoryError: GC overhead limit exceeded
        at com.fasterxml.jackson.core.JsonFactory._createParser(JsonFactory.java
:1351)
        at com.fasterxml.jackson.core.JsonFactory.createParser(JsonFactory.java:
906)
        at com.fasterxml.jackson.core.JsonFactory.createJsonParser(JsonFactory.j
ava:1108)
        at com.google.refine.model.Row.loadStreaming(Row.java:223)

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 18 (18 by maintainers)

Most upvoted comments

close it for now since cannot reproduce.

+1
jackyq2015 on Nov 30, 2018
Read more comments on GitHub
← OpenRefine: Processing of reconciliation is slow?
OpenRefine: spark-prototype: temp files not being deleted on Windows →