proper logging implementation for Python modules -
does know of examples of modules nice logging implementations?
i have been doing logging couple of different ways, i'm not sure pythonic.
for scripts, i've been doing:
import logging logger = logging.getlogger(__program__) stream = logging.streamhandler() formatter = logging.formatter('(%(name)s) %(levelname)s: %(message)s') stream.setformatter(formatter) logger.addhandler(stream) def main(): logger.warning('this warning message.') that executed in global namespace , can call logger anywhere.
the aforementioned solution not such idea modules because code executed upon import. modules, have been calling _logging() function on set logging.
def _logging(): import logging global logger logger = logging.getlogger(__program__) stream = logging.streamhandler() formatter = logging.formatter('(%(name)s) %(levelname)s: %(message)s') stream.setformatter(formatter) logger.addhandler(stream) def main(): _logging() logger.warning('this warning message.') since logger global, can call anywhere needed. pylint barks out global-variable-undefined warning. defined global variable logger undefined @ module level used when variable defined through “global” statement variable not defined in module scope i'm not sure why issue here.
or should call _logger() function on (minus global) , create logger everywhere needed?
def _logging(): import logging logger = logging.getlogger(__program__) stream = logging.streamhandler() formatter = logging.formatter('(%(name)s) %(levelname)s: %(message)s') stream.setformatter(formatter) logger.addhandler(stream) def main(): _logging() logger = logging.getlogger(__program__) logger.warning('this warning message.') the last technique seems cleanest, albeit tedious, since logging within dozens of small classes, functions, methods, et cetera. there examples people/modules have blazed path through territory?
if understood correctly, configuring logging in each module separately. unnecessary , against design of logging module.
i think key logging understanding logging module stateful object in python process. @ least me after insight there 1 obvious way logging in situations.
you should configure logging @ beginning of program. define handlers, formatters, etc., , configuration remain throughout program, long isn't explicitly overridden.
all modules logging can define global logger right after logging imported. there no need put function. recommended documentation, practice name each logger according module name (including package path):
import logging logger = logging.getlogger(__name__) it important understand loggers in program form hierarchy. default, loggers propagate records parents. means @ bottom there 1 logger (root) gets records unless configure loggers prevent this. may enough configure root logger.
to little more concrete, let's make program 2 modules, one.py , two.py. one.py contains function main entry point program. we'll configure logging using dictconfig, lets separate logging configuration nicely rest of code. we'll put configuration dict in separate yaml file, this:
# logging_config.yaml version: 1 formatters: brief: format: '%(message)s' default: format: '%(asctime)s %(levelname)-8s %(name)-15s %(message)s' datefmt: '%y-%m-%d %h:%m:%s' handlers: console: class: logging.streamhandler formatter: brief stream: ext://sys.stdout file: class: logging.handlers.rotatingfilehandler formatter: default filename: example.log maxbytes: 1024 backupcount: 3 loggers: two: level: info handlers: [file] propagate: false root: level: info handlers: [console] these snippets adapted the documentation. in configuration define above level info logger two logged file. records two not propagated further. comes root logger fed console if above level info.
the definition of module one this:
# one.py import logging import logging.config import yaml def configure_logging(filename): open(filename) f: config = yaml.load(f) logging.config.dictconfig(config) def main(): configure_logging('logging_config.yaml') 2 import func logger = logging.getlogger(__name__) logger.info('starting program') func() logger.info('finished') one tricky detail here import module two , define logger after configuration set. done because default dictconfig disables existing loggers.
and finally, here definition module two:
# two.py import logging logger = logging.getlogger(__name__) def func(): logger.info('doing stuff') now if run program, following output:
>>> import 1 >>> one.main() starting program finished and log file example.log contains following line:
2015-06-07 15:04:15 info 2 doing stuff excellent examples of logging can found in python documentation:
Comments
Post a Comment