How To Configure YAML To Create Fresh Log Files Instead Of Appending Them?

In a python logger implementation given below, each time I run my program, the logs are appended each time to the existing log files. How do I ensure that each time I run my applic

Solution 1:

I know this already has an accepted answer but I'm not sure it answers the question very cleanly/entirely and I am adding this answer in case others stumble across a similar problem. Instead, one option to include this in your configuration YAML file:

handlers:
    info_file_handler:
       class: logging.FileHandler
       formatter: complex
       filename: log_info.log
       mode: 'w'

The 'w' in mode is the same as the filemode option used in basicConfig() and essentially overwrites the log file instead of appending (which would be option 'a'). I also think the mode key can be used for RotatingFileHandler.

Then I add to my main code so that each filename is changed with each model run (this currently changes the log filename every day [I have a daily program running]). See below:

def setup_logging(default_path="logging_config.yml",
                  default_level=logging.INFO,
                  env_key="LOG_CFG"):

    """
    Setup logging configuration

    This reads a .yml file and extracts the relevant logging information
    required by the logging module.

    """

    path = default_path
    value = os.getenv(env_key, None)

    if value:
         path = value

    # Open the logging configuration file
    if os.path.exists(path):
        with open(path, "rb") as f:
            config = yaml.safe_load(f.read())

            # Append the date stamp to the filename for each handler
            # This is so each log file has a unique filename if run
            # on a separate date.
            # This can be changed to suit needs, hourly/minutely etc.
            for i in (config["handlers"].keys()):
                # Check if filename in handler.
                # Console handlers might be present in config file
                if 'filename' in config['handlers'][i]:
                    log_filename = config["handlers"][i]["filename"]
                    base, extension = os.path.splitext(log_filename)
                    today = datetime.datetime.today()
                    log_filename = "{}{}{}".format(base,
                                               today.strftime("_%Y_%m_%d"),
                                               extension)
                config["handlers"][i]["filename"] = log_filename

        logging.config.dictConfig(config)

    else:
        logging.basicConfig(level=default_level)

I hope this helps and apologies if the accepted answer does actually work for you, I just felt this was a cleaner solution to the problem the OP posed.

Solution 2:

Set the filemode to w, the default is a (append).

Or alternatively just add the following line to overwrite your old log file (after reading the yaml file):

with open(config['handlers']['info_handler']['filename'], 'w') as f:
    pass