bugzilla.py
author Scott Ehlert <ds@alliedmods.net>
Fri Dec 11 17:25:09 2009 -0600 (2009-12-11)
changeset 4 83eace28a7c2
parent 2 afc063529895
child 6 bb7c01182512
permissions -rw-r--r--
Merged in changes from bugzilla extension in mercurial 1.4.1.
     1 # bugzilla.py - bugzilla integration for mercurial
     2 #
     3 # Copyright 2006 Vadim Gelfer <[email protected]>
     4 #
     5 # This software may be used and distributed according to the terms of the
     6 # GNU General Public License version 2, incorporated herein by reference.
     7 
     8 '''hooks for integrating with the Bugzilla bug tracker
     9 
    10 This hook extension adds comments on bugs in Bugzilla when changesets
    11 that refer to bugs by Bugzilla ID are seen. The hook does not change
    12 bug status.
    13 
    14 The hook updates the Bugzilla database directly. Only Bugzilla
    15 installations using MySQL are supported.
    16 
    17 The hook relies on a Bugzilla script to send bug change notification
    18 emails. That script changes between Bugzilla versions; the
    19 'processmail' script used prior to 2.18 is replaced in 2.18 and
    20 subsequent versions by 'config/sendbugmail.pl'. Note that these will
    21 be run by Mercurial as the user pushing the change; you will need to
    22 ensure the Bugzilla install file permissions are set appropriately.
    23 
    24 The extension is configured through three different configuration
    25 sections. These keys are recognized in the [bugzilla] section:
    26 
    27 host
    28   Hostname of the MySQL server holding the Bugzilla database.
    29 
    30 db
    31   Name of the Bugzilla database in MySQL. Default 'bugs'.
    32 
    33 user
    34   Username to use to access MySQL server. Default 'bugs'.
    35 
    36 password
    37   Password to use to access MySQL server.
    38 
    39 timeout
    40   Database connection timeout (seconds). Default 5.
    41 
    42 version
    43   Bugzilla version. Specify '3.0' for Bugzilla versions 3.0 and later,
    44   '2.18' for Bugzilla versions from 2.18 and '2.16' for versions prior
    45   to 2.18.
    46 
    47 bzuser
    48   Fallback Bugzilla user name to record comments with, if changeset
    49   committer cannot be found as a Bugzilla user.
    50 
    51 bzdir
    52    Bugzilla install directory. Used by default notify. Default
    53    '/var/www/html/bugzilla'.
    54 
    55 notify
    56   The command to run to get Bugzilla to send bug change notification
    57   emails. Substitutes from a map with 3 keys, 'bzdir', 'id' (bug id)
    58   and 'user' (committer bugzilla email). Default depends on version;
    59   from 2.18 it is "cd %(bzdir)s && perl -T contrib/sendbugmail.pl
    60   %(id)s %(user)s".
    61 
    62 regexp
    63   Regular expression to match bug IDs in changeset commit message.
    64   Must contain one "()" group. The default expression matches 'Bug
    65   1234', 'Bug no. 1234', 'Bug number 1234', 'Bugs 1234,5678', 'Bug
    66   1234 and 5678' and variations thereof. Matching is case insensitive.
    67 
    68 style
    69   The style file to use when formatting comments.
    70 
    71 template
    72   Template to use when formatting comments. Overrides style if
    73   specified. In addition to the usual Mercurial keywords, the
    74   extension specifies::
    75 
    76     {bug}       The Bugzilla bug ID.
    77     {root}      The full pathname of the Mercurial repository.
    78     {webroot}   Stripped pathname of the Mercurial repository.
    79     {hgweb}     Base URL for browsing Mercurial repositories.
    80 
    81   Default 'changeset {node|short} in repo {root} refers '
    82           'to bug {bug}.\\ndetails:\\n\\t{desc|tabindent}'
    83 
    84 strip
    85   The number of slashes to strip from the front of {root} to produce
    86   {webroot}. Default 0.
    87 
    88 usermap
    89   Path of file containing Mercurial committer ID to Bugzilla user ID
    90   mappings. If specified, the file should contain one mapping per
    91   line, "committer"="Bugzilla user". See also the [usermap] section.
    92 
    93 The [usermap] section is used to specify mappings of Mercurial
    94 committer ID to Bugzilla user ID. See also [bugzilla].usermap.
    95 "committer"="Bugzilla user"
    96 
    97 Finally, the [web] section supports one entry:
    98 
    99 baseurl
   100   Base URL for browsing Mercurial repositories. Reference from
   101   templates as {hgweb}.
   102 
   103 Activating the extension::
   104 
   105     [extensions]
   106     hgext.bugzilla =
   107 
   108     [hooks]
   109     # run bugzilla hook on every change pulled or pushed in here
   110     incoming.bugzilla = python:hgext.bugzilla.hook
   111 
   112 Example configuration:
   113 
   114 This example configuration is for a collection of Mercurial
   115 repositories in /var/local/hg/repos/ used with a local Bugzilla 3.2
   116 installation in /opt/bugzilla-3.2. ::
   117 
   118     [bugzilla]
   119     host=localhost
   120     password=XYZZY
   121     version=3.0
   122     [email protected]
   123     bzdir=/opt/bugzilla-3.2
   124     template=Changeset {node|short} in {root|basename}.
   125              {hgweb}/{webroot}/rev/{node|short}\\n
   126              {desc}\\n
   127     strip=5
   128 
   129     [web]
   130     baseurl=http://dev.domain.com/hg
   131 
   132     [usermap]
   133     [email protected][email protected]
   134 
   135 Commits add a comment to the Bugzilla bug record of the form::
   136 
   137     Changeset 3b16791d6642 in repository-name.
   138     http://dev.domain.com/hg/repository-name/rev/3b16791d6642
   139 
   140     Changeset commit comment. Bug 1234.
   141 '''
   142 
   143 from mercurial.i18n import _
   144 from mercurial.node import short
   145 from mercurial import cmdutil, templater, util
   146 import os, re, time
   147 
   148 MySQLdb = None
   149 
   150 def buglist(ids):
   151     return '(' + ','.join(map(str, ids)) + ')'
   152 
   153 class bugzilla_comment(object):
   154     def __init__(self, user, text):
   155         self.user = user
   156         self.text = text
   157 
   158 class bugzilla_2_16(object):
   159     '''support for bugzilla version 2.16.'''
   160 
   161     def __init__(self, ui):
   162         self.ui = ui
   163         host = self.ui.config('bugzilla', 'host', 'localhost')
   164         user = self.ui.config('bugzilla', 'user', 'bugs')
   165         passwd = self.ui.config('bugzilla', 'password')
   166         db = self.ui.config('bugzilla', 'db', 'bugs')
   167         timeout = int(self.ui.config('bugzilla', 'timeout', 5))
   168         usermap = self.ui.config('bugzilla', 'usermap')
   169         if usermap:
   170             self.ui.readconfig(usermap, sections=['usermap'])
   171         self.ui.note(_('connecting to %s:%s as %s, password %s\n') %
   172                      (host, db, user, '*' * len(passwd)))
   173         self.conn = MySQLdb.connect(host=host, user=user, passwd=passwd,
   174                                     db=db, connect_timeout=timeout)
   175         self.cursor = self.conn.cursor()
   176         self.longdesc_id = self.get_longdesc_id()
   177         self.user_ids = {}
   178         self.bugs = {}
   179         self.default_notify = "cd %(bzdir)s && ./processmail %(id)s %(user)s"
   180 
   181     def run(self, *args, **kwargs):
   182         '''run a query.'''
   183         self.ui.note(_('query: %s %s\n') % (args, kwargs))
   184         try:
   185             self.cursor.execute(*args, **kwargs)
   186         except MySQLdb.MySQLError:
   187             self.ui.note(_('failed query: %s %s\n') % (args, kwargs))
   188             raise
   189 
   190     def get_longdesc_id(self):
   191         '''get identity of longdesc field'''
   192         self.run('select fieldid from fielddefs where name = "longdesc"')
   193         ids = self.cursor.fetchall()
   194         if len(ids) != 1:
   195             raise util.Abort(_('unknown database schema'))
   196         return ids[0][0]
   197 
   198     def filter_real_bug_ids(self, ids):
   199         '''filter not-existing bug ids from list.'''
   200         self.run('select bug_id from bugs where bug_id in %s' % buglist(ids))
   201         return sorted([c[0] for c in self.cursor.fetchall()])
   202 
   203     def filter_unknown_bug_ids(self, repo, node, ids):
   204         '''filter bug ids from list that already refer to this changeset.'''
   205 
   206         reponame = repo.root
   207         idx = reponame.rfind(os.path.sep)
   208         if idx != -1:
   209             reponame = reponame[idx+1:]
   210         self.run('''select bug_id from longdescs where
   211                     bug_id in %s and thetext like "%%%s%%" and thetext like "%%%s%%"''' %
   212                  (buglist(ids), short(node), reponame))
   213         unknown = set(ids)
   214         for (id,) in self.cursor.fetchall():
   215             self.ui.status(_('bug %d already knows about changeset %s\n') %
   216                            (id, short(node)))
   217             unknown.discard(id)
   218         return sorted(unknown)
   219 
   220     def notify(self):
   221         '''tell bugzilla to send mail.'''
   222 
   223         self.ui.status(_('telling bugzilla to send mail:\n'))
   224         for id, comments in self.bugs.iteritems():
   225             if len(comments) == 1:
   226                 (user, userid) = self.get_bugzilla_user(comments[0].user)
   227             else:
   228                 (user, userid) = self.get_bugzilla_user('\0')
   229             self.ui.status(_('  bug %s\n') % id)
   230             cmdfmt = self.ui.config('bugzilla', 'notify', self.default_notify)
   231             bzdir = self.ui.config('bugzilla', 'bzdir', '/var/www/html/bugzilla')
   232             try:
   233                 # Backwards-compatible with old notify string, which
   234                 # took one string. This will throw with a new format
   235                 # string.
   236                 cmd = cmdfmt % id
   237             except TypeError:
   238                 cmd = cmdfmt % {'bzdir': bzdir, 'id': id, 'user': user}
   239             self.ui.note(_('running notify command %s\n') % cmd)
   240             fp = util.popen('(%s) 2>&1' % cmd)
   241             out = fp.read()
   242             ret = fp.close()
   243             if ret:
   244                 self.ui.warn(out)
   245                 raise util.Abort(_('bugzilla notify command %s') %
   246                                  util.explain_exit(ret)[0])
   247         self.ui.status(_('done\n'))
   248 
   249     def get_user_id(self, user):
   250         '''look up numeric bugzilla user id.'''
   251         try:
   252             return self.user_ids[user]
   253         except KeyError:
   254             try:
   255                 userid = int(user)
   256             except ValueError:
   257                 self.ui.note(_('looking up user %s\n') % user)
   258                 self.run('''select userid from profiles
   259                             where login_name like %s''', user)
   260                 all = self.cursor.fetchall()
   261                 if len(all) != 1:
   262                     raise KeyError(user)
   263                 userid = int(all[0][0])
   264             self.user_ids[user] = userid
   265             return userid
   266 
   267     def map_committer(self, user):
   268         '''map name of committer to bugzilla user name.'''
   269         for committer, bzuser in self.ui.configitems('usermap'):
   270             if committer.lower() == user.lower():
   271                 return bzuser
   272         return user
   273 
   274     def get_bugzilla_user(self, committer):
   275         '''see if committer is a registered bugzilla user. Return
   276         bugzilla username and userid if so. If not, return default
   277         bugzilla username and userid.'''
   278         user = self.map_committer(committer)
   279         try:
   280             userid = self.get_user_id(user)
   281         except KeyError:
   282             try:
   283                 defaultuser = self.ui.config('bugzilla', 'bzuser')
   284                 if not defaultuser:
   285                     raise util.Abort(_('cannot find bugzilla user id for %s') %
   286                                      user)
   287                 userid = self.get_user_id(defaultuser)
   288                 user = defaultuser
   289             except KeyError:
   290                 raise util.Abort(_('cannot find bugzilla user id for %s or %s') %
   291                                  (user, defaultuser))
   292         return (user, userid)
   293 
   294     def add_comment(self, bugid, text, committer):
   295         '''add comment to bug. try adding comment as committer of
   296         changeset, otherwise as default bugzilla user.'''
   297         (user, userid) = self.get_bugzilla_user(committer)
   298         now = time.strftime('%Y-%m-%d %H:%M:%S')
   299         self.run('''insert into longdescs
   300                     (bug_id, who, bug_when, thetext)
   301                     values (%s, %s, %s, %s)''',
   302                  (bugid, userid, now, text))
   303         self.run('''insert into bugs_activity (bug_id, who, bug_when, fieldid)
   304                     values (%s, %s, %s, %s)''',
   305                  (bugid, userid, now, self.longdesc_id))
   306         self.conn.commit()
   307 
   308 class bugzilla_2_18(bugzilla_2_16):
   309     '''support for bugzilla 2.18 series.'''
   310 
   311     def __init__(self, ui):
   312         bugzilla_2_16.__init__(self, ui)
   313         self.default_notify = "cd %(bzdir)s && perl -T contrib/sendbugmail.pl %(id)s %(user)s"
   314 
   315 class bugzilla_3_0(bugzilla_2_18):
   316     '''support for bugzilla 3.0 series.'''
   317 
   318     def __init__(self, ui):
   319         bugzilla_2_18.__init__(self, ui)
   320 
   321     def get_longdesc_id(self):
   322         '''get identity of longdesc field'''
   323         self.run('select id from fielddefs where name = "longdesc"')
   324         ids = self.cursor.fetchall()
   325         if len(ids) != 1:
   326             raise util.Abort(_('unknown database schema'))
   327         return ids[0][0]
   328 
   329 class bugzilla(object):
   330     # supported versions of bugzilla. different versions have
   331     # different schemas.
   332     _versions = {
   333         '2.16': bugzilla_2_16,
   334         '2.18': bugzilla_2_18,
   335         '3.0':  bugzilla_3_0
   336         }
   337 
   338     _default_bug_re = (r'bugs?\s*,?\s*(?:#|nos?\.?|num(?:ber)?s?)?\s*'
   339                        r'((?:\d+\s*(?:,?\s*(?:and)?)?\s*)+)')
   340 
   341     _bz = None
   342 
   343     def __init__(self, ui, repo):
   344         self.ui = ui
   345         self.repo = repo
   346 
   347     def bz(self):
   348         '''return object that knows how to talk to bugzilla version in
   349         use.'''
   350 
   351         if bugzilla._bz is None:
   352             bzversion = self.ui.config('bugzilla', 'version')
   353             try:
   354                 bzclass = bugzilla._versions[bzversion]
   355             except KeyError:
   356                 raise util.Abort(_('bugzilla version %s not supported') %
   357                                  bzversion)
   358             bugzilla._bz = bzclass(self.ui)
   359         return bugzilla._bz
   360 
   361     def __getattr__(self, key):
   362         return getattr(self.bz(), key)
   363 
   364     _bug_re = None
   365     _split_re = None
   366 
   367     def find_bug_ids(self, ctx):
   368         '''find valid bug ids that are referred to in changeset
   369         comments and that do not already have references to this
   370         changeset.'''
   371 
   372         if bugzilla._bug_re is None:
   373             bugzilla._bug_re = re.compile(
   374                 self.ui.config('bugzilla', 'regexp', bugzilla._default_bug_re),
   375                 re.IGNORECASE)
   376             bugzilla._split_re = re.compile(r'\D+')
   377         start = 0
   378         ids = set()
   379         while True:
   380             m = bugzilla._bug_re.search(ctx.description(), start)
   381             if not m:
   382                 break
   383             start = m.end()
   384             for id in bugzilla._split_re.split(m.group(1)):
   385                 if not id: continue
   386                 ids.add(int(id))
   387         if ids:
   388             ids = self.filter_real_bug_ids(ids)
   389         if ids:
   390             ids = self.filter_unknown_bug_ids(self.repo, ctx.node(), ids)
   391         return ids
   392 
   393     def build_comment(self, bugid, ctx):
   394         '''format a bugzilla comment with reference to changeset.'''
   395 
   396         def webroot(root):
   397             '''strip leading prefix of repo root and turn into
   398             url-safe path.'''
   399             count = int(self.ui.config('bugzilla', 'strip', 0))
   400             root = util.pconvert(root)
   401             while count > 0:
   402                 c = root.find('/')
   403                 if c == -1:
   404                     break
   405                 root = root[c+1:]
   406                 count -= 1
   407             return root
   408 
   409         mapfile = self.ui.config('bugzilla', 'style')
   410         tmpl = self.ui.config('bugzilla', 'template')
   411         t = cmdutil.changeset_templater(self.ui, self.repo,
   412                                         False, None, mapfile, False)
   413         if not mapfile and not tmpl:
   414             tmpl = _('changeset {node|short} in repo {root} refers '
   415                      'to bug {bug}.\ndetails:\n\t{desc|tabindent}')
   416         if tmpl:
   417             tmpl = templater.parsestring(tmpl, quoted=False)
   418             t.use_template(tmpl)
   419         self.ui.pushbuffer()
   420         t.show(ctx, changes=ctx.changeset(),
   421                bug=str(bugid),
   422                hgweb=self.ui.config('web', 'baseurl'),
   423                root=self.repo.root,
   424                webroot=webroot(self.repo.root))
   425         data = self.ui.popbuffer()
   426         return data
   427 
   428     def add_update(self, bugid, ctx):
   429         text = self.build_comment(bugid, ctx)
   430         user = util.email(ctx.user())
   431         if not self.bugs.has_key(bugid):
   432             self.bugs[bugid] = [bugzilla_comment(user, text)]
   433         else:
   434             comments = self.bugs[bugid]
   435             for comment in comments:
   436                 if comment.user == user:
   437                     idx = comments.index(comment)
   438                     self.bugs[bugid][idx].text += '\n\n' + text
   439                     return
   440             self.bugs[bugid].append(bugzilla_comment(user, text))
   441 
   442     def update(self):
   443         for bugid, comments in self.bugs.iteritems():
   444             for comment in comments:
   445                 self.add_comment(bugid, comment.text, comment.user)
   446 
   447 def hook(ui, repo, hooktype, node=None, **kwargs):
   448     '''add comment to bugzilla for each changeset that refers to a
   449     bugzilla bug id. only add a comment once per bug, so same change
   450     seen multiple times does not fill bug with duplicate data.'''
   451     try:
   452         import MySQLdb as mysql
   453         global MySQLdb
   454         MySQLdb = mysql
   455     except ImportError, err:
   456         raise util.Abort(_('python mysql support not available: %s') % err)
   457 
   458     if node is None:
   459         raise util.Abort(_('hook type %s does not pass a changeset id') %
   460                          hooktype)
   461     try:
   462         bz = bugzilla(ui, repo)
   463         if hooktype == 'incoming':
   464             ctx = repo[node]
   465             ids = bz.find_bug_ids(ctx)
   466             if ids:
   467                 for id in ids:
   468                     bz.add_update(id, ctx)
   469                 bz.update()
   470                 bz.notify()
   471         elif hooktype == 'changegroup':
   472             start = repo[node].rev()
   473             end = len(repo)
   474             for rev in xrange(start, end):
   475                 node = repo.changelog.node(rev)
   476                 ctx = repo[node]
   477                 ids = bz.find_bug_ids(ctx)
   478                 if ids:
   479                     for id in ids:
   480                         bz.add_update(id, ctx)
   481             bz.update()
   482             bz.notify()
   483         else:
   484             raise util.Abort(_('hook type %s not supported') % hooktype)
   485     except MySQLdb.MySQLError, err:
   486         raise util.Abort(_('database error: %s') % err[1])
   487