What is best approach for long-term Edge CLI logging to file?

Developer Programs Writing Edge Drivers
1

I’ve been trying to set up logging in Linux for one of my Edge drivers so that the log output is continuously saved to a file, but I’m running up against a flushing issue. No logging output (besides the initial ‘connecting’ and listening for logs’ messages) is written to the file until I Ctrl-c out of the CLI logging command:

st edge:drivers:logcat <driverId> --hub-address 192.168.1.101 >> logs/as_edge.log 2>&1

I’ve tried various remedies recommended in stackoverflow, such as using stdbuf, but nothing seems to work. I want to be able to run tail on it to monitor the file when needed, and also have a periodic log rotation process.

Is there anything I can do to address the flush problem or would that require a fix to the CLI itself?

2

Hi, Todd!
I shared this situation with the engineering team and they’re analyzing it. As soon as I get more information, I’ll share it with you.

1 Like
3

Hi @TAustin, I did some quick tests and using both redirection and tee to concurrently write to stdout and file. I only saw the buffering change when redirecting stdout as well as stderr (2>&1) which I suspect is some result of the fancy output of the live logger. I believe all the logs get emitted to stdout so redirecting stderr probably isn’t required.

Here’s what I ran to tee the output with append.

$ smartthings edge:drivers:logcat --hub-address=192.168.1.4 -a \
     | tee -a driver-logs.txt

For log rotation, somewhat surprisingly there’s nothing in gnu coreutils that provides this out of the box. Apache has rotatelogs which you could setup: rotatelogs - Piped logging program to rotate Apache logs - Apache HTTP Server Version 2.4. Since we use a lot of rust I quickly did a cargo install of https://crates.io/crates/pipe-logger which worked well for me doing the following:

$ smartthings edge:drivers:logcat --hub-address=192.168.1.4 -a \
     | pipe-logger --err -r 1M -c 10 driver-logs.log

Which will rotate logs at 1MB and keep 10 log files. We’ve written a similar tool we use in a few places on the hub.

3 Likes
4

You’re awesome! Using tee solves it: no output buffering, i.e. logs are written realtime to the file.

Regarding log rotation, indeed I use rotatelogs, so I should be good now.

THANK YOU!

5

No problem, FWIW I think the problem was actually realted to stderr redirection to stdout which was changing the stdout buffering. I believe removing 2>&1 from your original command should work as well (but tee is nice for also seeing the live output). At least that is what I was seeing in my testing.

2 Likes