allsky: end of night script doesn't support large file numbers

I have an ASI290MM and shoot with these settings

Dec 13 12:55:02 allsky allsky.sh[26948]: ZWO ASI290MM Information:
Dec 13 12:55:02 allsky allsky.sh[26948]:   - Native Resolution: 1936x1096
Dec 13 12:55:02 allsky allsky.sh[26948]:   - Pixel Size: 2.9microns
Dec 13 12:55:02 allsky allsky.sh[26948]:   - Supported Bins: 1 2
Dec 13 12:55:02 allsky allsky.sh[26948]:   - Mono camera
Dec 13 12:55:02 allsky allsky.sh[26948]: Camera does not support serialNumber
Dec 13 12:55:02 allsky allsky.sh[26948]: - Sensor temperature: 41.10
Dec 13 12:55:02 allsky allsky.sh[26948]: Capture Settings:
Dec 13 12:55:02 allsky allsky.sh[26948]:  Image Type: ASI_IMG_RAW8
Dec 13 12:55:02 allsky allsky.sh[26948]:  Resolution (before any binning): 1936x1096
Dec 13 12:55:02 allsky allsky.sh[26948]:  Quality: 95
Dec 13 12:55:02 allsky allsky.sh[26948]:  Daytime capture: Yes
Dec 13 12:55:02 allsky allsky.sh[26948]:  Exposure (day): 0.500ms, Auto: Yes
Dec 13 12:55:02 allsky allsky.sh[26948]:  Exposure (night): 1,000ms, Auto: Yes
Dec 13 12:55:02 allsky allsky.sh[26948]:  Max Auto-Exposure (day): 60,000ms (60.0s)
Dec 13 12:55:02 allsky allsky.sh[26948]:  Max Auto-Exposure (night): 4,000ms (4.0s)
Dec 13 12:55:02 allsky allsky.sh[26948]:  Delay (day): 5,000ms
Dec 13 12:55:02 allsky allsky.sh[26948]:  Delay (night): 100ms
Dec 13 12:55:02 allsky allsky.sh[26948]:  Gain (night only): 250, Auto: No, max: 200
Dec 13 12:55:02 allsky allsky.sh[26948]:  Gain Transition Time: 15.0 minutes
Dec 13 12:55:02 allsky allsky.sh[26948]:  Brightness (day): 50
Dec 13 12:55:02 allsky allsky.sh[26948]:  Brightness (night): 50

Nearing the Northern Hemisphere’s winter solstice, with long nights I get this many files:


allsky:~/allsky/images/20211211 $ ls -1 | wc -l
12335

In most cases the keogram and startrails are working well, but the timelapse does not produce a video.

In trying to clean them up with… /home/pi/allsky/scripts/removeBadImages.sh .

it throws this error:

/home/pi/allsky/scripts/removeBadImages.sh: line 93: /usr/bin/which: Argument list too long
/home/pi/allsky/scripts/removeBadImages.sh: line 131: /usr/bin/egrep: Argument list too long
/home/pi/allsky/scripts/removeBadImages.sh: line 139: /usr/bin/bc: Argument list too long
/home/pi/allsky/scripts/removeBadImages.sh: line 143: /usr/bin/bc: Argument list too long
/home/pi/allsky/scripts/removeBadImages.sh: line 144: [: -eq: unary operator expected
/home/pi/allsky/scripts/removeBadImages.sh: line 153: /usr/bin/bc: Argument list too long

So I have 2 issues in one.

The timelapse NEVER generates a result over the past 6 nights
My inability to clean up any bad images via the included script.

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 20

Most upvoted comments

@maphilli14 I’m 99% sure timelapse is failing due to memory or swap being too small. See this Wiki Page. The ffmpeg command that creates the video does a lousy job with memory. Let me know if the suggestions on the Wiki page helped.

EricClaeys on Dec 24, 2021

@maphilli14, @jimjackii, please grab the latest scripts/removeBadImages.sh and let me know how it goes. It fixed the problem on my test machine.

@maphilli14, What do your log files say regarding the timelapse not working?

EricClaeys on Dec 21, 2021

let me know how this comes across…

maphilli14 on Dec 15, 2021

Hi @maphilli14 We can certainly fix the high image count issues in removeBadImages.sh and the timelapse generation (actually, timelapse may not be fixable on the Pi due to memory requirements of ffmpeg…) However, I suspect there’s a mix-up in your config that is causing the high file counts…

Dec 13 12:55:02 allsky allsky.sh[26948]:  Max Auto-Exposure (day): 60,000ms (60.0s)
Dec 13 12:55:02 allsky allsky.sh[26948]:  Max Auto-Exposure (night): 4,000ms (4.0s)

Max auto-exposure: DAY - 60s NIGHT - 4s

With these settings, I’m surprised if you’re not getting only the very brightest stars in your night exposures. Is this intentional?

linuxkidd on Dec 15, 2021