Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1355764 - Excessive DNF logging (all messages - including debug - from dnf, libdnf, librepo and rpm interface logged to file by default, not configurable) [NEEDINFO]
Summary: Excessive DNF logging (all messages - including debug - from dnf, libdnf, lib...
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: rawhide
Hardware: All
OS: All
Target Milestone: ---
Assignee: Lukáš Hrázký
QA Contact: Fedora Extras Quality Assurance
: 1364029 (view as bug list)
Depends On:
Blocks: 1476926
TreeView+ depends on / blocked
Reported: 2016-07-12 13:30 UTC by Artem S. Tashkinov
Modified: 2019-04-16 08:27 UTC (History)
18 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2017-06-19 13:58:48 UTC
sudo: needinfo? (jmracek)

Attachments (Terms of Use)

Description Artem S. Tashkinov 2016-07-12 13:30:20 UTC
Sorry to be nitpicky but DNF by default writes too many logs:


I humbly request to stop logging to all these files unless DNF is ran with debugging flags/command line options. These logs are simply unnecessary for 99% of use cases.

Comment 1 Honza Silhan 2016-07-18 13:55:47 UTC
Hi, we actually request "dnf.log" and "dnf.librepo.log" from users reporting bugs from time to time. It's quite handy when the problem occurs just once and is not reproducible again. This is the purpose of logs. You can control the level of output to stdout. You could change the logrotote (/etc/logrotate/dnf) values on your own if it takes too much space on your disk. Closing...

Comment 2 Artem S. Tashkinov 2016-07-18 14:08:39 UTC
Tell me honestly, why almost all other applications in Fedora run with debugging output turned off, and for some reasons DNF developers don't favour this convention.

Comment 3 Honza Silhan 2016-08-01 11:21:39 UTC
Can you share the reason reducing logging, please? Do you want to reduce IO operations because it slowing down your machine or is it the matter of space consumption?

Comment 4 Artem S. Tashkinov 2016-08-01 14:03:09 UTC
(In reply to Jan Silhan from comment #3)

Reason 1: Files in /var/log are already heavily fragmented and since dnf auto updates its db on every startup it increases fragmentation for the filesystem even further.

Reason 2: One in 1000 Fedora users might need those files, so 999 others need not. Again, why do all other packages in fedora get on just fine without extra logging and dnf has a very special status? Why don't we run a debug version of the kernel by default? Why don't we install GCC and dbg packages by default? Someone might need them right?

Reason 3: in pre dnf systems there was a hugely useful file /var/log/yum.log which had only two logging options: package.rpm installed or removed. Also this file was never removed. For some reasons DNS developers abandoned this file altogether and it's rotated every week and old records quickly disappear.

Reason 4: exactly how difficult is it to run dnf --debug=on (or something like that) so that all the necessary information gets collected and dumped?

Comment 5 Igor Gnatenko 2016-08-08 11:21:14 UTC
*** Bug 1364029 has been marked as a duplicate of this bug. ***

Comment 6 Michael S. 2017-02-22 11:08:05 UTC
So to add my voice, this did fill one of my server (I gave a 4G disk, since it just serve to run ansible). Using default upgrade, this did create a 430m file, which is 10% of the disk I used. 

I can see why that's useful for sure (especially since you usually need them after you see there was a problem), but we do not always have 10% of the disk to spare (think for embedded systems, for example).

Would it be possible to maybe be a bit more agressive into cleaning them (or compressing) ? After all, if the goal is to debug and discuss with developpers, we do not need them uncompressed.

Comment 7 Michael S. 2017-02-22 11:14:35 UTC
Ok so I found out that for some reason, logrotate wasn't installed on this system so I do not know how old are the logs, or how relevant is the disk usage.

Comment 8 Artem S. Tashkinov 2017-02-22 12:00:46 UTC
I've found a workaround for this unabated debugging madness:

# cd /var/log
# rm -f dnf.librepo.log dnf.log hawkey.log
# cp -a /dev/null dnf.librepo.log
# cp -a /dev/null dnf.log
# cp -a /dev/null hawkey.log

Now everything's much better.

Comment 9 Jaroslav Mracek 2017-06-19 13:58:48 UTC
Thank you Artem for work-around. Unfortunately we cannot do more than you provided. DNF is now in fast development stage and logging is very useful for us for debugging. I know that only 1% users uses logs but most of them don't know in advance, when they will need them. And as I can say about 50% of reported issues in dnf requires log information to solve them, therefore disabling logging will have direct negative effect on DNF improvement ant this price we cannot pay. Thanks a lot for your report.

Comment 10 Artem S. Tashkinov 2017-06-19 14:16:55 UTC
Logging can be enabled on a per case basis and I still don't see a single justification for having it enabled by default. A person has problems with DNF? Go enable debugging. It's a single comment from you or the DNF dev team. You still expect people to upload logs, so you may as well ask them to enable logging. How difficult is that?

"Please enabled DNF logging by ... and upload your /var/log/... files".

I'm tired of repeating that for the Nth time: why on earth Fedora doesn't have debugging packages installed by default? Why the debug version of the kernel is installed only for Rawhide users?

Why is DNF exempt from the common debug policy in Fedora? And why are you saying that debugging helps you immensely? It doesn't make sense - production ready packages shouldn't have a lot of bugs filed against them in the first place. That speaks volumes about the quality of DNF implementation.

Also I've been asking for years for a replacement of the old /var/log/yum.log file which is _actually_ useful, because it never gets rotated and you can trace the entire installation/deinstallation history. DNF simply doesn't have anything similar.

So instead of having one useful log file, you have three totally useless files containing the information which 99% of your users will never need.

If you're content with closing this bug report every 4 months, I'm content with reopening it every single day because, unlike you, I care about my computer systems and I don't turn them into steaming piles of debugging garbage just because some developers believe it's useful to litter end users' PCs.

Comment 11 Trevor Cordes 2017-09-11 06:09:16 UTC
I like the logs on by default.  Logs of this sort are fairly normal in the *NIX world.  I run a ton of things that produce way more logs than dnf.  (You can always set your logrotate settings on dnf files to be radical, like delete the logs every night.)

Perhaps when the devs get spare time they could add an cmdline option to turn off logging for your use case.  I don't think anyone would object to a new option like that.  If you want to speed up the effort to get this new feature, you could poke around the python code and add it in yourself and submit it as a patch here (providing a good ready-to-use git patch works wonders for getting your desired changes upstream).  My hunch is logging is all centralized and would be like 20 lines of python max.

As another workaround, make a wrapper script (or even shell alias!) you use instead of dnf that runs dnf with your specified options, then runs rm -f /var/log/dnf*.log afterwards.  I do something similar to force certain file perms after dnf upgrades when rpm/dnf undoes what I require.

Comment 12 Artem S. Tashkinov 2017-09-11 08:47:42 UTC
(In reply to Trevor Cordes from comment #11)
> I like the logs on by default. 

Tastes differ.

> Logs of this sort are fairly normal in the *NIX world.

No, they are not. I don't know another Unix daemon which logs so much (mostly useless) data.

> I run a ton of things that produce way more logs than dnf.

Care to provide examples? Also, I have a suspicion the amount of logs (data) you have corresponds to the activity of the said things. E.g. when you run postfix/exim on high load systems which process thousands of emails per minute you can expect a lot of data. However neither exim, nor postfix log TCP connection attempts or stuff like opening and closing file descriptors. That doesn't make any sense.

> (You can always set your logrotate settings on dnf files to be radical, like delete the logs every night.)

This is just a waste of everyone's time. Yum never logged that much and people were happy. Also, /var/log/yum.log file could be used to trace your history back to the day of installation, while DNF has nothing like that. It quickly rotates all of its logs and install/update/remove info gets rapidly removed.

Make sure you reread comment 10. No Unix daemon has the DEBUG level of logging enabled by default, only DNF does.

Also, I already solved/hacked this issue as explained in comment 8.

Comment 13 plaffiazho 2017-09-28 19:28:02 UTC
related (request for equivalent of yum.log (log of installs, removes, and updates)):

Comment 14 Fedora End Of Life 2017-11-16 19:49:39 UTC
This message is a reminder that Fedora 25 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 25. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '25'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 25 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 15 Jonathan Wakely 2018-04-26 18:49:12 UTC
(In reply to Trevor Cordes from comment #11)
> I like the logs on by default.  Logs of this sort are fairly normal in the
> *NIX world.

Not with the silly level of detail in dnf.librepo.log

17:59:19 lr_handle_prepare_metalink: Local metalink.xml found at /var/cache/dnf/rpmfusion-free-debuginfo-5859699b8393ad49/metalink.xml
17:59:19 lr_handle_prepare_metalink: Parsing metalink.xml
17:59:19 lr_handle_prepare_metalink: Mirrors from metalink:
[dozens of lines of URLs]
17:59:19 lr_handle_prepare_metalink: Metalink parsed
17:59:19 lr_handle_prepare_internal_mirrorlist: Finalizing internal mirrorlist
17:59:19 lr_handle_prepare_internal_mirrorlist: Finalizing mirrors reported via LRI_MIRRORS
17:59:19 lr_handle_perform: Downloading/Locating yum repo
17:59:19 lr_yum_use_local: Locating repo..
17:59:19 lr_yum_use_local_load_base: Found local metalink: /var/cache/dnf/rpmfusion-free-debuginfo-5859699b8393ad49/metalink.xml
17:59:19 lr_yum_use_local_load_base: Parsing repomd.xml
17:59:19 lr_yum_use_local_load_base: Repomd revision: 1499687612
17:59:19 lr_yum_use_local: Repository was successfully located
17:59:19 lr_yum_check_checksum_of_md_record: Checking checksum of /var/cache/dnf/rpmfusion-free-debuginfo-5859699b8393ad49/repodata/9a6cf985f7f55f9a322e31e45b5c1cfcff7875b79778ae9509fc6970457f3919-filelists.xml.gz (expected: 9a6cf985f7f55f9a322e31e45b5c1cfcff7875b79778ae9509fc6970457f3919 [sha256])
17:59:19 lr_checksum_fd_compare: Using checksum cached in xattr: [user.Zif.MdChecksum[1502737917]] 9a6cf985f7f55f9a322e31e45b5c1cfcff7875b79778ae9509fc6970457f3919
17:59:19 lr_yum_check_checksum_of_md_record: Checksum check - Passed
17:59:19 lr_yum_check_checksum_of_md_record: Checking checksum of /var/cache/dnf/rpmfusion-free-debuginfo-5859699b8393ad49/repodata/b53f93a2a7bf5c228ef5a34f943dfd62b4e190b1a74866444e53c8e0e327b7d7-primary.xml.gz (expected: b53f93a2a7bf5c228ef5a34f943dfd62b4e190b1a74866444e53c8e0e327b7d7 [sha256])
17:59:19 lr_yum_check_checksum_of_md_record: Checksum check - Passed
17:59:19 lr_handle_perform: Restoring an old SIGINT handler

This is not "I did a thing" like logging when a mail server forwards an email, this is "I'm about to do a thing ... OK doing the thing ... OK done the thing ... the thing is finished now"

Surely the level of detail can be reviewed and some non-essential logging (in code that isn't usually involve in crashes) can be reduced.

And the default /etc/logrotate.d/dnf file could keep fewer than four weeks of logs.

Comment 16 Jonathan Wakely 2018-04-26 18:57:01 UTC
And in dnf.log:

2018-04-25T12:43:24Z DDEBUG timer: config: 8 ms
2018-04-25T12:43:24Z DEBUG Loaded plugins: builddep, config-manager, copr, debug, debuginfo-install, download, generate_completion_cache, needs-restarting, playground, repoclosure, repograph, repomanage, reposync, system-upgrade
2018-04-25T12:43:24Z DEBUG DNF version: 2.7.5
2018-04-25T12:43:24Z DDEBUG Command: dnf makecache timer
2018-04-25T12:43:24Z DDEBUG Installroot: /
2018-04-25T12:43:24Z DDEBUG Releasever: 26
2018-04-25T12:43:24Z DEBUG cachedir: /var/cache/dnf
2018-04-25T12:43:24Z DDEBUG Base command: makecache
2018-04-25T12:43:24Z DDEBUG Extra commands: ['makecache', 'timer']

If this info is really important it should not be at DEBUG and DDEBUG level, it should be INFO. I'm not debugging dnf, so I don't need DEBUG logging.

And lots of the DEBUG info is identical every time dnf runs.

This isn't useful either:
2018-04-25T12:43:24Z INFO --- logging initialized ---

It's pretty clear logging has been initialized, because of all the logs!

Comment 17 Charles Butterfield 2018-07-08 17:55:30 UTC
I'd like to agree with the sub-issue about the removal of yum.log with no suitable replacement.

Today I attempted to get my new Fedora-28 system working and I have failed to install something critical that I clearly had installed on Fedora-27.  Sadly, although I saved my entire Fedora-27 disk, the log of exactly which packages were installed around the time I solved my problem on Fedora-27 are simply gone, gone, gone, presumably overwritten by the silly rotation.

Given that the DNF devs clearly like (love?) logging, it seems incredible that they have done away with the permanent activity summary log that used to be stored in yum.log.  PLEASE address this ASAP.  Let me know if this should be a distinct ticket.

Also, if the owner could bump the version from F27 to F28 that would be helpful so that this ticket doesn't simply expire due to EOL.

Comment 18 Artem S. Tashkinov 2018-07-08 18:11:44 UTC
Actually let's change the version to RAWHIDE to make this bug stick out.

Comment 19 Charles Butterfield 2018-07-08 18:23:03 UTC
Just noticed (and commented on) the related ticket

Comment 20 Jan Kurik 2018-08-14 10:22:29 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 29 development cycle.
Changing version to '29'.

Comment 21 Adam Williamson 2018-11-09 18:56:49 UTC
hawkey.log is frigging *huge* on my system lately - I just noticed it's eating 5GB(!) of space all on its own:

[root@adam log]# ls -lh hawkey.log-2018*
-rw-------. 1 root root  51K Jun 18 08:57 hawkey.log-20180618
-rw-------. 1 root root  50K Jun 25 09:06 hawkey.log-20180625
-rw-------. 1 root root 762K Sep 15 10:38 hawkey.log-20180915
-rw-------. 1 root root 4.3G Nov  7 16:47 hawkey.log-20181107

it's possible I did something to suddenly put it in a different debugging mode, but I don't *think* so.

Comment 22 Adam Williamson 2018-11-09 18:57:55 UTC
oh, sorry, there's also hawkey.log (the live version) which is ~1GB:

[root@adam log]# ls -lh hawkey.log
-rw-------. 1 root root 937M Nov  9 10:02 hawkey.log

that's only since Nov 7 18:23:12 - so it's growing at like over 500MB a day.

Comment 23 Adam Williamson 2018-11-09 18:58:38 UTC
oh gah, that is me, I forgot I was running a scratch libdnf with additional logging enabled to try and figure out a bug. My bad.

Comment 24 Magnus Glantz 2019-04-03 07:19:35 UTC
Hi all, actually because of doing DEBUG level logging to dnf.librepo.log, reposync is effectively broken on F29.
Half way through doing a reposync of RHEL8 to my F29 instance, I got +42 GB of logs, which fills up my filesystem. In order to sync RHEL8, I estimate I would need +100 GB of storage for /var/log/dnf.librepo.log.

I would argue this is bad.

Comment 25 Magnus Glantz 2019-04-03 07:25:04 UTC
Also, I'm guessing this must have quite some performance impact on reposync, grinding systems to a halt if they cannot handle the fact that for each GB of rpms sync'd, you write 10 times as much logs.

Comment 26 Artem S. Tashkinov 2019-04-03 07:44:39 UTC
I wonder how many more RedHat employees we need to CC to this bug report in order to fix it. ;-)

Again I semi-solved it by /dev/null'ing the respective files but I'm still missing a bog standard yum.log replacement which is not logrotated to oblivion.

Comment 27 Magnus Glantz 2019-04-03 11:33:37 UTC
Jaroslav Mracek, do you have any suggestion regarding reposync effectively broken with current loglevel?
Can you reconsider the WONTFIX?

Comment 29 Adam Williamson 2019-04-03 15:54:43 UTC
Just FWIW, I looked at the code for's a bit weird to figure out (like a lot of dnf code...) but basically dnf has config values for logging verbosity, only they *only apply to what gets sent to stdout and stderr*. You can't configure what gets logged to the log files at all.

So dnf actually invents some named log levels that don't exist in standard Python 'logging'. The lowest named level in the logging module is DEBUG, which is 10; dnf invents 'DDEBUG' (8), 'SUBDEBUG' (6) and 'TRACE' (4). So for any messages coming in from anything outside of dnf itself, logging at any of those levels is going to include everything, most likely, unless they also have their own invented super-low levels.

The dnf logger is just hard coded to the dnf-invented level called 'TRACE', so basically just means "log absolutely everything ever". The dnf.rpm logger is hard coded to the also-dnf-invented level called 'SUBDEBUG', so means 'log absolutely everything except TRACE'.

The dnf.librepo logger is handled in libdnf. It just adds a handler which logs every message it's sent; it takes the log level from the message and converts it to a text representation, but it uses the G_LOG_LEVEL_MASK flag, which as defined at means 'a mask including all log levels'. So that logger literally just dumps absolutely every log message from librepo to the file.

As I mentioned, there are config options (which can be set via config file, CLI args, override files and environment vars...) - `verbose` and `quiet` each set a specific 'debuglevel' and 'errorlevel', and there are explicit `debuglevel` and `errorlevel` options you can use to set any desired level - but these levels are applied *only to the stdout and stderr handlers for the loggers* (debuglevel is what gets sent to stdout, errorlevel is what gets sent to stderr). So you can affect what goes to stdout and stderr, but you have no control over what goes to the log files (unless you just go in and edit and change the values there).

This is all in dnf/ Logging._setup(), if anyone else wants a look:

I might send a PR that changes it, if only to kickstart discussion a bit harder.

Comment 33 Adam Williamson 2019-04-03 17:20:42 UTC
As we have to cover the 'no equivalent to yum.log' aspect, let's make this bug cover *only* the excessive logging aspect.

Comment 34 Lukáš Hrázký 2019-04-04 09:24:21 UTC

I've recently attempted to fix the dnf.librepo.log case for this bug:

PRs here:

It basically turns off debug logging for librepo by default and turns it on if --debuglevel is set to higher than 2 (the default).

So that should help a bit. Otherwise, I agree that logging in general is in quite a sorry state both in terms of the actual messages and the levels they're logged on as well as the convoluted level setup Adam described in comment #29.

I'm wondering how many bugs we actually have against logging and I think this should be reasonably high on the priority list to fix...

Comment 37 Magnus Glantz 2019-04-04 12:09:34 UTC
I can confirm that this hits RHEL8.

Note You need to log in before you can comment on or make changes to this bug.