Richard Jones' Log: Improved DEBUG logging only when it's needed
Thanks to Kent Johnson for reminding me about contexts (I don't get to play with them much so I sometimes forget :) usage is a lot simpler:
for i in range(5): with DebugCapture(log): task(i)
This is produced by modifying the code from yesterday a little:
'''Capture a logger's DEBUG output for potential later dumping. Simple use (requires "with" statement in Python 2.5 or later): with DebugCapture(logger): do_stuff() If an exception occurs the DEBUG output will be emitted to the log followed by an exception ERROR message. Alternatively for Python < 2.5 you may use it more manually: handler = DebugCapture(logger) And at the end of a transaction: handler.clear() If an error occurs: handler.emit_debug() And to uninstall (stop trapping DEBUG messages): handler.uninstall() Copyright 2009 Richard Jones (richard@mechanicalcat.net) and you're free to use / modify / ignore it as you like. ''' import logging import weakref class DebugCapture(logging.Handler): def __init__(self, logger): logging.Handler.__init__(self, logging.DEBUG) self.records = [] # insert myself as the handler for the logger self.logger = weakref.proxy(logger) self.slaves = logger.handlers logger.handlers = [self] def emit_debug(self): # emit stored records to the original logger handler(s) for record in self.records: for slave in self.slaves: slave.handle(record) def clear(self): self.records = [] def uninstall(self): self.logger.handlers = self.slaves self.slaves = self.logger = None def emit(self, record): self.records.append(record) def handle(self, record): # if its a DEBUG level record then intercept otherwise # pass through to the original logger handler(s) if record.levelno == logging.DEBUG: return logging.Handler.handle(self, record) return sum(slave.handle(record) for slave in self.slaves) def __enter__(self): pass def __exit__(self, exc_type, exc_value, traceback): if exc_type is not None: self.emit_debug() self.logger.error('unhandled exception', exc_info=(exc_type, exc_value, traceback)) self.uninstall() return True # # SAMPLE PROGRAM # logging.basicConfig() log = logging.getLogger() log.setLevel(logging.DEBUG) def task(argument): log.info('run task (argument=%s)'%argument) log.debug('debug info should only appear next to error') if argument%2: BROKEN log.debug('all done!') if __name__ == '__main__': for i in range(5): with DebugCapture(log): task(i)
I disagree just because a) if your i's look like 1's, you're using the wrong font. I may agree with 'o' because it's not clear of it's purpose but 'i' has been used for a very specific use for so many years it would be better for these new programmers to get used to it asap. I hazard to claim that 'i', 'j', and 'k' may be the most used variable names in computer programs as a whole (note: this is a guess, I have no proof not intend to look for any since it really doesn't matter.) Sure if something more reasonable like 'line_no' fit but i would not change 'i' to 'idx' or 'index' or something. that's just silly, stupid, and makes code harder to read imo.
Throw a 'b)' after the 'asap.'
Or to phrase it in more historical words of a wiser man:
LOCAL variable names should be short, and to the point. If you have some random integer loop counter, it should probably be called "i". Calling it "loop_counter" is non-productive, if there is no chance of it being mis-understood. Similarly, "tmp" can be just about any type of variable that is used to hold a temporary value.
Hi Richard,
Thanks for sharing this.
I was trying to use DebugCapture when I noticed a huge increase of memory usage, so after digging a bit I found the cause!
Extending from logging.Handler makes the instance to be added to logging._handlers and logging._handlerList, so they are never garbage collected and keep pilling up.
So as a workaround, in order to use it as a context manager, only one instance of DebugCapture should be created and use it each call:
dc = DebugCapture(logger)
with dc:
# do funky stuff
Also some tweaks are needed in the __enter__/__exit__ steps.
here is the diff to support using a single instance as context manager: http://pastebin.com/m8d8dfae
Regards,
Oh, I just noticed that another option is to call self.close() in uninstall :)
Cheers,
for i in range(5):
Using 'i', 'l', or 'o' as single digit variable names is bad form. They look too much like numbers, and introduce bad habits to anyone just learning.