feather: Error from read_feather (in the feather R library)
I’ve been getting the following error when using read_feather():
*** caught segfault ***
address 0x7f9b75bab010, cause 'memory not mapped'
Traceback:
1: .Call("feather_coldataFeather", PACKAGE = "feather", feather, indexes)
2: coldataFeather(x, i)
3: `[.feather`(x, )
4: x[]
5: as_data_frame.feather(data)
6: as_data_frame(data)
7: read_feather(file.match)
My sessionInfo():
R version 3.2.1 (2015-06-18)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 14.04.3 LTS
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=en_US.UTF-8
[9] LC_ADDRESS=en_US.UTF-8 LC_TELEPHONE=en_US.UTF-8
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] feather_0.0.1 metricsgraphics_0.8.5 dygraphs_0.5
[4] RColorBrewer_1.1-2 htmlwidgets_0.5 htmltools_0.2.6
[7] DT_0.1 shinydashboard_0.5.1 shiny_0.12.2
[10] lubridate_1.3.3 helpRs_0.1 magrittr_1.5
[13] ggplot2_2.1.0 RPostgres_0.1-1 data.table_1.9.6
[16] dplyr_0.4.3
loaded via a namespace (and not attached):
[1] Rcpp_0.12.5 futile.logger_1.4.1 plyr_1.8.1
[4] R.methodsS3_1.7.1 futile.options_1.0.0 R.utils_2.3.0
[7] tools_3.2.1 digest_0.6.8 packrat_0.4.1-1
[10] tibble_1.0 lattice_0.20-31 memoise_0.2.1
[13] gtable_0.1.2 DBI_0.4-1 parallel_3.2.1
[16] rJava_0.9-8 mailR_0.4.1 stringr_1.0.0
[19] httr_1.1.0 grid_3.2.1 R6_2.1.2
[22] lambda.r_1.1.7 scales_0.4.0 urltools_1.4.0
[25] tldextract_1.0 assertthat_0.1 mime_0.4
[28] colorspace_1.2-4 xtable_1.7-4 httpuv_1.3.3
[31] stringi_1.0-1 munsell_0.4.2 chron_2.3-47
[34] zoo_1.7-11 R.oo_1.20.0
And the str(the.table_I_tried.to.save) of the object I’m writing:
Classes ‘data.table’ and 'data.frame': 1888384 obs. of 5 variables:
$ uuid : chr "00105052-ff64-43ed-9405-2d4bf847ac40" "0001a6a0-2cf2-49ca-adc3-ff0cd87cf1aa" "00027a78-6774-48cc-a53f-6c985321d24f" "0000867e-666e-4c99-9bc0-1a153bd5e53e" ...
$ status : chr "no data" "success" "no data" "success" ...
$ update_date : chr "2016-02-15" "2016-02-15" "2016-02-15" "2016-02-15" ...
$ table_name : chr "location" "location" "location" "location" ...
$ last_update_date: chr "2016-02-15" "2016-02-15" "2016-02-15" "2016-02-15" ...
- attr(*, ".internal.selfref")=<externalptr>
I wish I could produce a reproducible example but I haven’t been able to come up with one, and unfortunately I can’t post the data I’m using. Fwiw, I tried writing a plain data.frame instead of a data.table but the same happened. Afaik there shouldn’t be anything funky in the values of the variables.
A couple more details I’ve noticed:
- This tends to happen with large data.frames
- If I first read the data.frame from an .rds file and then read the same object from a .feather file this error doesn’t happen (observed this while debugging).
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Comments: 15 (3 by maintainers)
Commits related to this issue
- Add missing protect statements #150 — committed to wesm/feather by hadley 8 years ago
- [R] Add missing protect statements This should alleviate some of the gc issues - I'm not 100% I got them all though #150 Author: hadley <h.wickham@gmail.com> Closes #204 from hadley/missing-protec... — committed to wesm/feather by hadley 8 years ago
TL;DR: problem seems fixed!
I just installed the latest version of feather, did a quick test, and then a more thorough one, and can report that the problems is no longer. Thanks for the fix!
Detail:
BEFORE UPGRADE: sessionInfo() tells me version of feather is: [1] feather_0.0.1
Reinstall feather/R via devtools (did this on Thu Aug 11 2016, at 11h00 GMT)
AFTER UPGRADE: R sessionInfo now says: [1] feather_0.0.1.9000
A quick, short test, proved successfull, then I reran the longer test I described here:
http://data.munging.ninja/pubcoms/20160526_bug_feather_mem_not_mapped/
And it works fine! Great!