Redirect Job Dispatcher logs to stdout #120
Conversation
vbirsan-sony
left a comment
vbirsan-sony
left a comment
There was a problem hiding this comment.
@helshazly-sony I might be mistaken, but aren’t the Octopus server logs also streamed to a file? Will that be handled in a different PR?
|
|
||
| logger = logging.getLogger(__name__) | ||
| logging.basicConfig(filename=Constants.LOG_FILE_PATH, | ||
| logging.basicConfig(stream=sys.stdout, |
There was a problem hiding this comment.
I was wondering if you’ve considered adding a prefix to the logs. Right now, both the job dispatcher and Spark run in the same container, Spark is sending its logs to stdout, and we’ll now be sending the job dispatcher’s logs there as well.
Will we be able to tell which log lines belong to which service? It’s not an issue when the job dispatcher runs in its own container, but in this setup, all logs will be mixed together. Perhaps adding a prefix or an identifier to each log record would make it easier to filter out the job dispatcher logs
| poetry run uvicorn src.main:app --host 0.0.0.0 --reload & | ||
|
|
||
| # Print the log file location | ||
| echo "$LOG_FILE" |
There was a problem hiding this comment.
Do we still need this line and line 7?
Instead of writing logs to a file, this PR redirects all Job Dispatcher's logs, prints, messages to stdout. This setup is useful in Docker, Kubernetes, or Spark driver/executor logs.