Richard Jones' Log: Simple usage of Python's logging module
Several people have posted about Python's new logging module. Most are confused by its complexity and inaccessible documentation (great reference, where's the introduction?) Well, I've used it once now, and here's the simplest I could make it:
import logging
logger = logging.getLogger('myapp')
hdlr = logging.FileHandler('var/myapp.log')
formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
hdlr.setFormatter(formatter)
logger.addHandler(hdlr)
logger.setLevel(logging.INFO)
And then to use it:
logger.info('a log message')
... of course, for most people, the ideal usage would be:
import logging
logger = logging.open('var/myapp.log')
Gee, that'd be nice. Maybe I'll submit that as a patch, in one of my spare moments.
But I don't want to involve that horribly complicated config file stuff. I just want an object that'll log stuff to a file with different levels etc.
Doesn't the root logger log to stdout? And my first chunk of code is the only way to override that behaviour AFAICT.
Is the preferred way to close the log
logger.removeHandler(hdlr)
hdlr.close()
? closing the handle seems to be problematic.
Thanks.
Twisted's logging package seems to be a little friendlier for setup:
from twisted.python import log
log.startLogging(file('app.log', 'w'))
log.msg("Tra la la, la la, la la")
twisted.python.log is capable of all the same things the logging package is. I would really have liked it if the logging package hadn't been so over-engineered. Then I could probably have convinced the right people that twisted.python.log wasn't necessary anymore. Alas...
I fully sympathise with Richard's view that the common use case of logging to a file should be made easier. I will try to come up with something easier. However, a logger is not analogous to a file - rather, it is an event sink which can be routed to many different listeners, one of which might be a file. Hence comparisons with Twisted's logging do not seem accurate to me (based on a quick reading of Twisted's logging API in epydoc format). From what I can see, in Twisted's view, a logger appears analogous to a file or stream. If I'm wrong, please point me in the right direction.
Response to Stuart Bishop's comment about the config file format, which Richard says is "horribly complicated":
You could just use the Tkinter-based GUI application to set up a configuration for your application, and call fileConfig() to make the configuration effective. Why should you care what "cruft" is in the .ini file? You need never look at it. Some of the apparent cruft is there only to support the GUI configurator. Of course if someone can suggest a better config file format (while still sticking with ConfigParser), all suggestions will be gratefully accepted. I'm not suggesting that what's there now is optimal. If you have Copious Free Time to look at alternative configuration implementations, I'd be happy to hear suggestions.
Response to john's comment:
Please file a bug report on SourceForge. Closing handlers used to be problematic, but I am not aware of specific problems with current Python CVS or standalone version 0.4.9.2 (http://www.red-dove.com/python_logging.html#download).
Response to Jp Calderone's comment:
Twisted's logging package may well be easier for certain specific setups. But can you substantiate that "twisted.python.log is capable of all the same things the logging package is"? For example, emailing logs, logging to HTTP, TCP, UDP, SOAP servers, logging to Unix syslog and NT's event log. Perhaps the logging package is over-engineered for your needs, but that's not to say it's a commonly held view - not everyone's needs are the same.
On GNU/Linux or UNIX you want to use the syslog:
import logging, logging.handlers
logger = logging.getLogger("a_name")
hdlr = logging.handlers.SysLogHandler(
facility=logging.handlers.SysLogHandler.LOG_DAEMON)
formatter = logging.Formatter(
'%(filename)s: %(levelname)s: %(message)s')
hdlr.setFormatter(formatter)
logger.addHandler(hdlr)
Together with an advanced syslog daemon, such as syslog-ng,
this works very well.
I have stand-alone program execution that logs to a file via the logger FileHandler, but if I start the program on a thread, the launch command arg includes a socket number and the logger replaces FileHandler with SocketHandler. Then all my threaded programs can log via socket to the launching program.
I think this boils down to "logging works great, once you are up to speed. It does not scale *down* very well." easylog is an attempt to do this. Since it treats the underlying logging module as a black box, it should benefit from future improvements. (But since it deals with a black box, the internals are uglier than I would like.) I am definately looking for feedback before I remove the alpha tag.
The new basicConfig() (in Python 2.4 and also from the standalone distribution v 0.4.9.5) addresses these concerns. This should be made clear in an update to the posting. You can now use basicConfig to set up a file handler, root logger level and formatter in one call. See the Python 2.4 docs online for more details:
http://docs.python.org/lib/minimal-example.html
I'm very happy with this simple way to use logging. I'm currently trying to get my head around (and maintain) a zope installation with a bunch of pretty complex products. I just needed some simple way to output debugging info to a separate file. I've found the existing documentation and examples to be a bit complex for someone who is just learning zope and python. This makes it very easy. Thanks.
Hello,
it would be nice to found a sample that explain how to have both a log in a file and on the console (stdout or stderr).
For now I'm using
log = logging.getLogger("MyApp")
hdlr = logging.FileHandler('log.csv')
FORMAT='%(asctime)s\t%(levelname)s\t%(message)s'
formatter = logging.Formatter(FORMAT)
logging.basicConfig(format=FORMAT) # log sur console
hdlr.setFormatter(formatter)
log.addHandler(hdlr)
log.setLevel(logging.DEBUG) #set verbosity to show all messages of severity >= DEBUG
So as you can see I'm using basicConfig for doing this but that's not the best way...
I guess that I should use someting like
hdlrConsole = logging.StreamHandler('/dev/stdout')
hdlrConsole.setFormatter(formatter)
Best regards
I think the preferred usage should be:
>>> logging.config.fileConfig('log.ini')
>>> logger = logging.getLogger('myapp')
>>> logger.info('oops I did it again')
Or for most applications, rather than libraries, just use the root logger:
>>> logging.info('played with your heart, right till the end')
This works file right now, except that unfortunatly the current format of log.ini is full of dead chickens and error prone. I think the goal is good, but the implementation falls down at this point (if you are going to have python code that is eval'd in your config files, you might as well save everyone the bother and just use real Python modules). I think this is where interested people should spend their Copious Free Time�.