awless: Running out of memory with 12GB available
Running awless -e sync with 12GB of memory available ends up in the process being killed because of an OOM situation. Would have expected that 12GB is plenty of memory for the task.
~/go/src/github.com/wallix/awless$ uname -a
Linux ubuntu 3.13.0-98-generic #145-Ubuntu SMP Sat Oct 8 20:13:07 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
~/go/src/github.com/wallix/awless$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.5 LTS
Release: 14.04
Codename: trusty
~/go/src/github.com/wallix/awless$ free
total used free shared buffers cached
Mem: 16427688 2612424 13815264 652 260 27940
-/+ buffers/cache: 2584224 13843464
Swap: 1046524 1046524 0
~/go/src/github.com/wallix/awless$ ./awless --version
awless version=v0.1.1
~/go/src/github.com/wallix/awless$ rm -rf ~/.awless
~/go/src/github.com/wallix/awless$ ./awless -e sync
First install. Welcome!
Found existing AWS region 'us-east-1'. Setting it as your default region.
Region updated to 'us-east-1'.
You might want to update your default AMI with `awless config set instance.image $(awless search images amazonlinux --id-only --silent)`
Syncing new region...
[extra] sync: fetched lambda service took 406.591864ms
[extra] sync: fetched cloudformation service took 408.829508ms
[extra] sync: fetched cdn service took 489.701489ms
[extra] sync: fetched dns service took 895.959112ms
[extra] sync: fetched messaging service took 1.280953266s
[extra] sync: fetched infra service took 3.58098968s
Killed
~/go/src/github.com/wallix/awless$ tail /var/log/syslog
Jun 30 13:01:00 localhost kernel: [77434.411864] [21886] 0 21886 376 0 5 16 0 sh
Jun 30 13:01:00 localhost kernel: [77434.411865] [21949] 0 21949 1489483 64503 295 17131 0 java
Jun 30 13:01:00 localhost kernel: [77434.411866] [25321] 1000 25321 7042 1902 20 0 0 bash
Jun 30 13:01:00 localhost kernel: [77434.411867] [27581] 1000 27581 2386 43 11 0 0 less
Jun 30 13:01:00 localhost kernel: [77434.411868] [27596] 1000 27596 9187110 3412059 14185 0 0 awless
Jun 30 13:01:00 localhost kernel: [77434.411869] Out of memory: Kill process 27596 (awless) score 784 or sacrifice child
Jun 30 13:01:00 localhost kernel: [77434.411889] Killed process 27596 (awless) total-vm:36748440kB, anon-rss:13648236kB, file-rss:0kB
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Comments: 25 (18 by maintainers)
Commits related to this issue
- Avoid over usage of memory: apply resolving on snapshot instead of graph to avoid numerous in loop call to triplestore.Source.Snapshot() (call that pre allocates memory) Tackle part of issue #116 Ov... — committed to wallix/awless by simcap 7 years ago
- Sync addRegionParent: no more store snapshot. Results: (issue #116) Running with /usr/bin/time --verbose we get a decrease of 2.5 in Maximum Resident Size memory. Profiling give us a decrease in al... — committed to wallix/awless by simcap 7 years ago
- Sync performance: better CPU / mem. See below: Avoid snapshotting datastore for querying when not necessary when building relations (issue #116) * Go profiling: - making Snapshotting not on top 5 C... — committed to wallix/awless by simcap 7 years ago
Ugh, don’t know what is up with my time. Maybe I overrode it with a brew coreutil or something?
Sorry my run isn’t the demonstrative figure you were looking for, but I am certainly happen that it went from not working at all and failing after almost a minute to being nice and snappy with just a few seconds. 😃