Only try to create log directory if file logging is enabled #10949
Conversation
|
Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA. In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, please reach out through a comment on this PR. CLA has not been signed by users: @ericfreese |
|
Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide. |
There's no need to create the log directory if we won't be logging to a file in it.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #10949 /- ##
==========================================
- Coverage 89.19% 89.12% -0.08%
==========================================
Files 183 183
Lines 23496 23496
==========================================
- Hits 20958 20941 -17
- Misses 2538 2555 17
Flags with carried forward coverage won't be shown. Click here to find out more.
|
There was a problem hiding this comment.
Thanks for opening this PR @ericfreese !
We haven't finished evaluating #10948 as a feature request yet (i.e. whether we want to commit to this behavior or not).
If we do choose to adopt it, then we'd need to add applicable testing to this PR.
| @@ -0,0 1,6 @@ | |||
| kind: Fixes | |||
There was a problem hiding this comment.
We're considering this a feature rather than a fix, so could you make this update?
| kind: Fixes | |
| kind: Features |
Resolves #10948
Problem
I ran into this while deploying DBT core in an environment where dbt's current working directory is not writeable. I had set
DBT_LOG_PATHto a separate writeable directory, but then realized recently that we actually want to entirely disable DBT's file logging because we store the stdout/stderr output separately.After some digging through the source (because I couldn't find documentation of how to disable the file logging), I found this line:
dbt-core/core/dbt/events/logging.py
Line 95 in dd77210
A bit more digging and I found that I could set a
DBT_LOG_LEVEL_FILE=noneenvironment variable to effectively disable the file logging. I figured that after disabling file logging, I would no longer need to setDBT_LOG_PATHso I removed that environment variable.Upon testing, I was surprised to see that
dbt debugnow provided no output and only exited with error code 2. I'd run into this issue before when it couldn't open the log file for writing, so I addedDBT_LOG_PATH=/tmpback to the environment, and it ran successfully.It seems to me that if file logging is disabled, dbt should not try to make the logging directory.
Solution
As mentioned above, I can work around the problem for now by setting
DBT_LOG_PATH=/tmpso that it will just create an empty directory in/tmpbut that feels kind of silly.I think it would be best to only try to make the logging directory if file logging is enabled, as the changes in my PR implement.
Checklist