1.. SPDX-License-Identifier: GPL-2.0+ 2.. Copyright (c) 2011 The Chromium OS Authors 3.. Simon Glass <sjg@chromium.org> 4.. Maxim Cournoyer <maxim.cournoyer@savoirfairelinux.com> 5.. v1, v2, 19-Oct-11 6.. revised v3 24-Nov-11 7.. revised v4 Independence Day 2020, with Patchwork integration 8 9Patman patch manager 10==================== 11 12This tool is a Python script which: 13 14- Creates patch directly from your branch 15- Cleans them up by removing unwanted tags 16- Inserts a cover letter with change lists 17- Runs the patches through checkpatch.pl and its own checks 18- Optionally emails them out to selected people 19 20It also has some Patchwork features: 21 22- shows review tags from Patchwork so you can update your local patches 23- pulls these down into a new branch on request 24- lists comments received on a series 25 26It is intended to automate patch creation and make it a less 27error-prone process. It is useful for U-Boot and Linux work so far, 28since they use the checkpatch.pl script. 29 30It is configured almost entirely by tags it finds in your commits. 31This means that you can work on a number of different branches at 32once, and keep the settings with each branch rather than having to 33git format-patch, git send-email, etc. with the correct parameters 34each time. So for example if you put:: 35 36 Series-to: fred.blogs@napier.co.nz 37 38in one of your commits, the series will be sent there. 39 40In Linux and U-Boot this will also call get_maintainer.pl on each of your 41patches automatically (unless you use -m to disable this). 42 43 44Installation 45------------ 46 47You can install patman using:: 48 49 pip install patch-manager 50 51The name is chosen since patman conflicts with an existing package. 52 53If you are using patman within the U-Boot tree, it may be easiest to add a 54symlink from your local `~/.bin` directory to `/path/to/tools/patman/patman`. 55 56How to use this tool 57-------------------- 58 59This tool requires a certain way of working: 60 61- Maintain a number of branches, one for each patch series you are 62 working on 63- Add tags into the commits within each branch to indicate where the 64 series should be sent, cover letter, version, etc. Most of these are 65 normally in the top commit so it is easy to change them with 'git 66 commit --amend' 67- Each branch tracks the upstream branch, so that this script can 68 automatically determine the number of commits in it (optional) 69- Check out a branch, and run this script to create and send out your 70 patches. Weeks later, change the patches and repeat, knowing that you 71 will get a consistent result each time. 72 73 74How to configure it 75------------------- 76 77For most cases of using patman for U-Boot development, patman can use the 78file 'doc/git-mailrc' in your U-Boot directory to supply the email aliases 79you need. To make this work, tell git where to find the file by typing 80this once:: 81 82 git config sendemail.aliasesfile doc/git-mailrc 83 84For both Linux and U-Boot the 'scripts/get_maintainer.pl' handles 85figuring out where to send patches pretty well. For other projects, 86you may want to specify a different script to be run, for example via 87a project-specific `.patman` file:: 88 89 # .patman configuration file at the root of some project 90 91 [settings] 92 get_maintainer_script: etc/teams.scm get-maintainer 93 94The `get_maintainer_script` option corresponds to the 95`--get-maintainer-script` argument of the `send` command. It is 96looked relatively to the root of the current git repository, as well 97as on PATH. It can also be provided arguments, as shown above. The 98contract is that the script should accept a patch file name and return 99a list of email addresses, one per line, like `get_maintainer.pl` 100does. 101 102During the first run patman creates a config file for you by taking the default 103user name and email address from the global .gitconfig file. 104 105To add your own, create a file `~/.patman` like this:: 106 107 # patman alias file 108 109 [alias] 110 me: Simon Glass <sjg@chromium.org> 111 112 u-boot: U-Boot Mailing List <u-boot@lists.denx.de> 113 wolfgang: Wolfgang Denk <wd@denx.de> 114 others: Mike Frysinger <vapier@gentoo.org>, Fred Bloggs <f.bloggs@napier.net> 115 116As hinted above, Patman will also look for a `.patman` configuration 117file at the root of the current project git repository, which makes it 118possible to override the `project` settings variable or anything else 119in a project-specific way. The values of this "local" configuration 120file take precedence over those of the "global" one. 121 122Aliases are recursive. 123 124The checkpatch.pl in the U-Boot tools/ subdirectory will be located and 125used. Failing that you can put it into your path or ~/bin/checkpatch.pl 126 127If you want to avoid sending patches to email addresses that are picked up 128by patman but are known to bounce you can add a [bounces] section to your 129.patman file. Unlike the [alias] section these are simple key: value pairs 130that are not recursive:: 131 132 [bounces] 133 gonefishing: Fred Bloggs <f.bloggs@napier.net> 134 135 136If you want to change the defaults for patman's command-line arguments, 137you can add a [settings] section to your .patman file. This can be used 138for any command line option by referring to the "dest" for the option in 139patman.py. For reference, the useful ones (at the moment) shown below 140(all with the non-default setting):: 141 142 [settings] 143 ignore_errors: True 144 process_tags: False 145 verbose: True 146 smtp_server: /path/to/sendmail 147 patchwork_url: https://patchwork.ozlabs.org 148 149If you want to adjust settings (or aliases) that affect just a single 150project you can add a section that looks like [project_settings] or 151[project_alias]. If you want to use tags for your linux work, you could do:: 152 153 [linux_settings] 154 process_tags: True 155 156 157How to run it 158------------- 159 160First do a dry run: 161 162.. code-block:: bash 163 164 ./tools/patman/patman send -n 165 166If it can't detect the upstream branch, try telling it how many patches 167there are in your series 168 169.. code-block:: bash 170 171 ./tools/patman/patman -c5 send -n 172 173This will create patch files in your current directory and tell you who 174it is thinking of sending them to. Take a look at the patch files: 175 176.. code-block:: bash 177 178 ./tools/patman/patman -c5 -s1 send -n 179 180Similar to the above, but skip the first commit and take the next 5. This 181is useful if your top commit is for setting up testing. 182 183 184How to install it 185----------------- 186 187The most up to date version of patman can be found in the U-Boot sources. 188However to use it on other projects it may be more convenient to install it as 189a standalone application. A distutils installer is included, this can be used 190to install patman: 191 192.. code-block:: bash 193 194 cd tools/patman && python setup.py install 195 196 197How to add tags 198--------------- 199 200To make this script useful you must add tags like the following into any 201commit. Most can only appear once in the whole series. 202 203Series-to: email / alias 204 Email address / alias to send patch series to (you can add this 205 multiple times) 206 207Series-cc: email / alias, ... 208 Email address / alias to Cc patch series to (you can add this 209 multiple times) 210 211Series-version: n 212 Sets the version number of this patch series 213 214Series-prefix: prefix 215 Sets the subject prefix. Normally empty but it can be RFC for 216 RFC patches, or RESEND if you are being ignored. The patch subject 217 is like [RFC PATCH] or [RESEND PATCH]. 218 In the meantime, git format.subjectprefix option will be added as 219 well. If your format.subjectprefix is set to InternalProject, then 220 the patch shows like: [InternalProject][RFC/RESEND PATCH] 221 222Series-postfix: postfix 223 Sets the subject "postfix". Normally empty, but can be the name of a 224 tree such as net or net-next if that needs to be specified. The patch 225 subject is like [PATCH net] or [PATCH net-next]. 226 227Series-name: name 228 Sets the name of the series. You don't need to have a name, and 229 patman does not yet use it, but it is convenient to put the branch 230 name here to help you keep track of multiple upstreaming efforts. 231 232Series-links: [id | version:id]... 233 Set the ID of the series in patchwork. You can set this after you send 234 out the series and look in patchwork for the resulting series. The 235 URL you want is the one for the series itself, not any particular patch. 236 E.g. for http://patchwork.ozlabs.org/project/uboot/list/?series=187331 237 the series ID is 187331. This property can have a list of series IDs, 238 one for each version of the series, e.g. 239 240 :: 241 242 Series-links: 1:187331 2:188434 189372 243 244 Patman always uses the one without a version, since it assumes this is 245 the latest one. When this tag is provided, patman can compare your local 246 branch against patchwork to see what new reviews your series has 247 collected ('patman status'). 248 249Series-patchwork-url: url 250 This allows specifying the Patchwork URL for a branch. This overrides 251 both the setting files ("patchwork_url") and the command-line argument. 252 The URL should include the protocol and web site, with no trailing slash, 253 for example 'https://patchwork.ozlabs.org/project' 254 255Cover-letter: 256 Sets the cover letter contents for the series. The first line 257 will become the subject of the cover letter:: 258 259 Cover-letter: 260 This is the patch set title 261 blah blah 262 more blah blah 263 END 264 265Cover-letter-cc: email / alias 266 Additional email addresses / aliases to send cover letter to (you 267 can add this multiple times) 268 269Series-notes: 270 Sets some notes for the patch series, which you don't want in 271 the commit messages, but do want to send, The notes are joined 272 together and put after the cover letter. Can appear multiple 273 times:: 274 275 Series-notes: 276 blah blah 277 blah blah 278 more blah blah 279 END 280 281Commit-notes: 282 Similar, but for a single commit (patch). These notes will appear 283 immediately below the ``---`` cut in the patch file:: 284 285 Commit-notes: 286 blah blah 287 blah blah 288 more blah blah 289 290Signed-off-by: Their Name <email> 291 A sign-off is added automatically to your patches (this is 292 probably a bug). If you put this tag in your patches, it will 293 override the default signoff that patman automatically adds. 294 Multiple duplicate signoffs will be removed. 295 296Tested-by / Reviewed-by / Acked-by 297 These indicate that someone has tested/reviewed/acked your patch. 298 When you get this reply on the mailing list, you can add this 299 tag to the relevant commit and the script will include it when 300 you send out the next version. If 'Tested-by:' is set to 301 yourself, it will be removed. No one will believe you. 302 303 Example:: 304 305 Tested-by: Their Name <fred@bloggs.com> 306 Reviewed-by: Their Name <email> 307 Acked-by: Their Name <email> 308 309Series-changes: n 310 This can appear in any commit. It lists the changes for a 311 particular version n of that commit. The change list is 312 created based on this information. Each commit gets its own 313 change list and also the whole thing is repeated in the cover 314 letter (where duplicate change lines are merged). 315 316 By adding your change lists into your commits it is easier to 317 keep track of what happened. When you amend a commit, remember 318 to update the log there and then, knowing that the script will 319 do the rest. 320 321 Example:: 322 323 Series-changes: n 324 - Guinea pig moved into its cage 325 - Other changes ending with a blank line 326 <blank line> 327 328Commit-changes: n 329 This tag is like Series-changes, except changes in this changelog will 330 only appear in the changelog of the commit this tag is in. This is 331 useful when you want to add notes which may not make sense in the cover 332 letter. For example, you can have short changes such as "New" or 333 "Lint". 334 335 Example:: 336 337 Commit-changes: n 338 - This line will not appear in the cover-letter changelog 339 <blank line> 340 341Cover-changes: n 342 This tag is like Series-changes, except changes in this changelog will 343 only appear in the cover-letter changelog. This is useful to summarize 344 changes made with Commit-changes, or to add additional context to 345 changes. 346 347 Example:: 348 349 Cover-changes: n 350 - This line will only appear in the cover letter 351 <blank line> 352 353Patch-cc: Their Name <email> 354 This copies a single patch to another email address. Note that the 355 Cc: used by git send-email is ignored by patman, but will be 356 interpreted by git send-email if you use it. 357 358Series-process-log: sort, uniq 359 This tells patman to sort and/or uniq the change logs. Changes may be 360 multiple lines long, as long as each subsequent line of a change begins 361 with a whitespace character. For example, 362 363 Example:: 364 365 - This change 366 continues onto the next line 367 - But this change is separate 368 369 Use 'sort' to sort the entries, and 'uniq' to include only 370 unique entries. If omitted, no change log processing is done. 371 Separate each tag with a comma. 372 373Change-Id: 374 This tag is used to generate the Message-Id of the emails that 375 will be sent. When you keep the Change-Id the same you are 376 asserting that this is a slightly different version (but logically 377 the same patch) as other patches that have been sent out with the 378 same Change-Id. The Change-Id tag line is removed from outgoing 379 patches, unless the `keep_change_id` settings is set to `True`. 380 381Various other tags are silently removed, like these Chrome OS and 382Gerrit tags:: 383 384 BUG=... 385 TEST=... 386 Review URL: 387 Reviewed-on: 388 Commit-xxxx: (except Commit-notes) 389 390Exercise for the reader: Try adding some tags to one of your current 391patch series and see how the patches turn out. 392 393 394Where Patches Are Sent 395---------------------- 396 397Once the patches are created, patman sends them using git send-email. The 398whole series is sent to the recipients in Series-to: and Series-cc. 399You can Cc individual patches to other people with the Patch-cc: tag. Tags 400in the subject are also picked up to Cc patches. For example, a commit like 401this:: 402 403 commit 10212537b85ff9b6e09c82045127522c0f0db981 404 Author: Mike Frysinger <vapier@gentoo.org> 405 Date: Mon Nov 7 23:18:44 2011 -0500 406 407 x86: arm: add a git mailrc file for maintainers 408 409 This should make sending out e-mails to the right people easier. 410 411 Patch-cc: sandbox, mikef, ag 412 Patch-cc: afleming 413 414will create a patch which is copied to x86, arm, sandbox, mikef, ag and 415afleming. 416 417If you have a cover letter it will get sent to the union of the Patch-cc 418lists of all of the other patches. If you want to sent it to additional 419people you can add a tag:: 420 421 Cover-letter-cc: <list of addresses> 422 423These people will get the cover letter even if they are not on the To/Cc 424list for any of the patches. 425 426 427Patchwork Integration 428--------------------- 429 430Patman has a very basic integration with Patchwork. If you point patman to 431your series on patchwork it can show you what new reviews have appeared since 432you sent your series. 433 434To set this up, add a Series-link tag to one of the commits in your series 435(see above). 436 437Then you can type: 438 439.. code-block:: bash 440 441 patman status 442 443and patman will show you each patch and what review tags have been collected, 444for example:: 445 446 ... 447 21 x86: mtrr: Update the command to use the new mtrr 448 Reviewed-by: Wolfgang Wallner <wolfgang.wallner@br-automation.com> 449 + Reviewed-by: Bin Meng <bmeng.cn@gmail.com> 450 22 x86: mtrr: Restructure so command execution is in 451 Reviewed-by: Wolfgang Wallner <wolfgang.wallner@br-automation.com> 452 + Reviewed-by: Bin Meng <bmeng.cn@gmail.com> 453 ... 454 455This shows that patch 21 and 22 were sent out with one review but have since 456attracted another review each. If the series needs changes, you can update 457these commits with the new review tag before sending the next version of the 458series. 459 460To automatically pull into these tags into a new branch, use the -d option: 461 462.. code-block:: bash 463 464 patman status -d mtrr4 465 466This will create a new 'mtrr4' branch which is the same as your current branch 467but has the new review tags in it. The tags are added in alphabetic order and 468are placed immediately after any existing ack/review/test/fixes tags, or at the 469end. You can check that this worked with: 470 471.. code-block:: bash 472 473 patman -b mtrr4 status 474 475which should show that there are no new responses compared to this new branch. 476 477There is also a -C option to list the comments received for each patch. 478 479 480Example Work Flow 481----------------- 482 483The basic workflow is to create your commits, add some tags to the top 484commit, and type 'patman' to check and send them. 485 486Here is an example workflow for a series of 4 patches. Let's say you have 487these rather contrived patches in the following order in branch us-cmd in 488your tree where 'us' means your upstreaming activity (newest to oldest as 489output by git log --oneline):: 490 491 7c7909c wip 492 89234f5 Don't include standard parser if hush is used 493 8d640a7 mmc: sparc: Stop using builtin_run_command() 494 0c859a9 Rename run_command2() to run_command() 495 a74443f sandbox: Rename run_command() to builtin_run_command() 496 497The first patch is some test things that enable your code to be compiled, 498but that you don't want to submit because there is an existing patch for it 499on the list. So you can tell patman to create and check some patches 500(skipping the first patch) with: 501 502.. code-block:: bash 503 504 patman -s1 send -n 505 506If you want to do all of them including the work-in-progress one, then 507(if you are tracking an upstream branch): 508 509.. code-block:: bash 510 511 patman send -n 512 513Let's say that patman reports an error in the second patch. Then: 514 515.. code-block:: bash 516 517 git rebase -i HEAD~6 518 # change 'pick' to 'edit' in 89234f5 519 # use editor to make code changes 520 git add -u 521 git rebase --continue 522 523Now you have an updated patch series. To check it: 524 525.. code-block:: bash 526 527 patman -s1 send -n 528 529Let's say it is now clean and you want to send it. Now you need to set up 530the destination. So amend the top commit with: 531 532.. code-block:: bash 533 534 git commit --amend 535 536Use your editor to add some tags, so that the whole commit message is:: 537 538 The current run_command() is really only one of the options, with 539 hush providing the other. It really shouldn't be called directly 540 in case the hush parser is bring used, so rename this function to 541 better explain its purpose:: 542 543 Series-to: u-boot 544 Series-cc: bfin, marex 545 Series-prefix: RFC 546 Cover-letter: 547 Unified command execution in one place 548 549 At present two parsers have similar code to execute commands. Also 550 cmd_usage() is called all over the place. This series adds a single 551 function which processes commands called cmd_process(). 552 END 553 554 Change-Id: Ica71a14c1f0ecb5650f771a32fecb8d2eb9d8a17 555 556 557You want this to be an RFC and Cc the whole series to the bfin alias and 558to Marek. Two of the patches have tags (those are the bits at the front of 559the subject that say mmc: sparc: and sandbox:), so 8d640a7 will be Cc'd to 560mmc and sparc, and the last one to sandbox. 561 562Now to send the patches, take off the -n flag: 563 564.. code-block:: bash 565 566 patman -s1 send 567 568The patches will be created, shown in your editor, and then sent along with 569the cover letter. Note that patman's tags are automatically removed so that 570people on the list don't see your secret info. 571 572Of course patches often attract comments and you need to make some updates. 573Let's say one person sent comments and you get an Acked-by: on one patch. 574Also, the patch on the list that you were waiting for has been merged, 575so you can drop your wip commit. 576 577Take a look on patchwork and find out the URL of the series. This will be 578something like `http://patchwork.ozlabs.org/project/uboot/list/?series=187331` 579Add this to a tag in your top commit:: 580 581 Series-links: 187331 582 583You can use then patman to collect the Acked-by tag to the correct commit, 584creating a new 'version 2' branch for us-cmd: 585 586.. code-block:: bash 587 588 patman status -d us-cmd2 589 git checkout us-cmd2 590 591You can look at the comments in Patchwork or with: 592 593.. code-block:: bash 594 595 patman status -C 596 597Then you can resync with upstream: 598 599.. code-block:: bash 600 601 git fetch origin # or whatever upstream is called 602 git rebase origin/master 603 604and use git rebase -i to edit the commits, dropping the wip one. 605 606Then update the `Series-cc:` in the top commit to add the person who reviewed 607the v1 series:: 608 609 Series-cc: bfin, marex, Heiko Schocher <hs@denx.de> 610 611and remove the Series-prefix: tag since it it isn't an RFC any more. The 612series is now version two, so the series info in the top commit looks like 613this:: 614 615 Series-to: u-boot 616 Series-cc: bfin, marex, Heiko Schocher <hs@denx.de> 617 Series-version: 2 618 Cover-letter: 619 ... 620 621Finally, you need to add a change log to the two commits you changed. You 622add change logs to each individual commit where the changes happened, like 623this:: 624 625 Series-changes: 2 626 - Updated the command decoder to reduce code size 627 - Wound the torque propounder up a little more 628 629(note the blank line at the end of the list) 630 631When you run patman it will collect all the change logs from the different 632commits and combine them into the cover letter, if you have one. So finally 633you have a new series of commits:: 634 635 faeb973 Don't include standard parser if hush is used 636 1b2f2fe mmc: sparc: Stop using builtin_run_command() 637 cfbe330 Rename run_command2() to run_command() 638 0682677 sandbox: Rename run_command() to builtin_run_command() 639 640so to send them: 641 642.. code-block:: bash 643 644 patman 645 646and it will create and send the version 2 series. 647 648 649General points 650-------------- 651 652#. When you change back to the us-cmd branch days or weeks later all your 653 information is still there, safely stored in the commits. You don't need 654 to remember what version you are up to, who you sent the last lot of patches 655 to, or anything about the change logs. 656#. If you put tags in the subject, patman will Cc the maintainers 657 automatically in many cases. 658#. If you want to keep the commits from each series you sent so that you can 659 compare change and see what you did, you can either create a new branch for 660 each version, or just tag the branch before you start changing it: 661 662 .. code-block:: bash 663 664 git tag sent/us-cmd-rfc 665 # ...later... 666 git tag sent/us-cmd-v2 667 668#. If you want to modify the patches a little before sending, you can do 669 this in your editor, but be careful! 670#. If you want to run git send-email yourself, use the -n flag which will 671 print out the command line patman would have used. 672#. It is a good idea to add the change log info as you change the commit, 673 not later when you can't remember which patch you changed. You can always 674 go back and change or remove logs from commits. 675#. Some mailing lists have size limits and when we add binary contents to 676 our patches it's easy to exceed the size limits. Use "--no-binary" to 677 generate patches without any binary contents. You are supposed to include 678 a link to a git repository in your "Commit-notes", "Series-notes" or 679 "Cover-letter" for maintainers to fetch the original commit. 680#. Patches will have no changelog entries for revisions where they did not 681 change. For clarity, if there are no changes for this patch in the most 682 recent revision of the series, a note will be added. For example, a patch 683 with the following tags in the commit:: 684 685 Series-version: 5 686 Series-changes: 2 687 - Some change 688 689 Series-changes: 4 690 - Another change 691 692 would have a changelog of::: 693 694 (no changes since v4) 695 696 Changes in v4: 697 - Another change 698 699 Changes in v2: 700 - Some change 701 702 703Other thoughts 704-------------- 705 706This script has been split into sensible files but still needs work. 707Most of these are indicated by a TODO in the code. 708 709It would be nice if this could handle the In-reply-to side of things. 710 711The tests are incomplete, as is customary. Use the 'test' subcommand to run 712them: 713 714.. code-block:: bash 715 716 $ tools/patman/patman test 717 718Note that since the test suite depends on data files only available in 719the git checkout, the `test` command is hidden unless `patman` is 720invoked from the U-Boot git repository. 721 722Alternatively, you can run the test suite via Pytest: 723 724.. code-block:: bash 725 726 $ cd tools/patman && pytest 727 728Error handling doesn't always produce friendly error messages - e.g. 729putting an incorrect tag in a commit may provide a confusing message. 730 731There might be a few other features not mentioned in this README. They 732might be bugs. In particular, tags are case sensitive which is probably 733a bad thing. 734