diff --git a/README.md b/README.md index 83a2056..bf89947 100644 --- a/README.md +++ b/README.md @@ -1,19 +1,142 @@ # p4-diff -Tool to generate a Perforce unified-diff from open or shelved changelists supporting add, edited and deleted files. -See help via p4-diff --help. +`p4-diff` is a small Python CLI that prints a **unified diff** for every file in a Helix Core / Perforce **changelist**. It supports **pending (opened) changelists** and **shelved** changelists, and handles **add**, **edit**, **integrate**, **delete**, and related move/branch operations. -### Diff from default changelist using opened files -p4-diff -c default > default.diff +The script shells out to `p4` and to your system `diff` (or whatever `P4DIFF` points at for Perforce-invoked paths). It does **not** use the Perforce C++ API. -### Diff from specific changelist using the shelved files -p4-diff -c 123456 -s > shelved_123456.diff +## Requirements -### Diff from specific changelist using opened files -p4-diff -c 123456 > opened_123456.diff +- `p4` on your `PATH` (the script runs `which p4` before doing anything). +- Python **3** via **`uv`** (first line is `#!/usr/bin/env -S uv run python3`). Run with `./p4-diff` after `chmod +x`, or `uv run python3 p4-diff …`. -

-cd ~/other_source -
-patch --dry-run -Np6 -i opened_123456.diff -

+## Usage + +```text +p4-diff [--changelist CL] [-s] [-l] [-t TICKET] [-v] +``` + +| Option | Meaning | +|--------|--------| +| `-c` / `--changelist` | Changelist number; **default is `default`** (your default pending changelist) when omitted. Use an explicit number for shelved CLs. | +| `-s` / `--shelved` | Treat the CL as **shelved** (pending files on the server) instead of **opened** in your client. | +| `-l` / `--local` | **Only with `-s`:** for each file, write shelved content with **`p4 print`** to a temp file, then **`diff -u `**. Workspace path comes from **`p4 where`**; if the file is **missing** on disk, the left side is **`/dev/null`**. | +| `-t` / `--ticket` | Passes **`-P TICKET`** to every `p4` invocation (ticket-based auth). | +| `-v` / `--verbose` | Prints each command before it runs. **Warning:** this output is mixed with real diff text and will break consumers that expect a clean patch stream. | + +```bash +./p4-diff --help +``` + +## What it does (two modes) + +### 1. Opened changelist (no `-s`) + +File list comes from: + +```text +p4 opened -s -c +``` + +For each depot path it maps to a workspace file with `p4 where` when it needs a local path. + +| Operation | Behavior | +|-----------|----------| +| **add**, **branch**, **move/add** | Runs **`diff -u /dev/null `** so the patch is “new file” vs empty. | +| **edit**, **integrate** | Runs **`p4 diff -du `**. That is Perforce’s normal **client workspace vs opened revision** diff (your checked-out content is involved). | +| **delete**, **move/delete** | Runs **`diff -u /dev/null`** (file disappearing). | + +“Empty” detection for edits: if `p4 diff` returns only **three** lines (header-only), the script treats it as no diff and prints nothing for that file (unless `-v`). + +### 2. Shelved changelist (`-s`) + +File list comes from: + +```text +p4 describe -S -s +``` + +It parses only lines starting with `...` and expects the word layout this script was written for (see **Limitations**). + +| Operation | Behavior | +|-----------|----------| +| **add**, **branch**, **move/add** | **`p4 fstat`** (head type) and **`p4 print -q depot@=CL`** into a temp file, then **`diff -u /dev/null temp`**. The second diff header line is rewritten to show the **depot** path instead of the temp path. Symlinks: strips an extra trailing newline Perforce adds. | +| **edit**, **integrate** | **`p4 diff2 -u depot# depot@=`** — compares the **depot revision before the shelf** with the **shelved depot revision**. This is **not** “shelved server file vs my local disk”; both sides are **depot** revisions. | +| **delete**, **move/delete** | **`p4 print`** of the **pre-delete depot revision** (`#rev` from describe), temp file, then **`diff -u temp /dev/null`**, with the first header line rewritten to the depot path. Same symlink newline tweak as adds. | + +“Empty” detection for shelved edits: if `diff2` output splits to a **single** line, treated as empty (unless `-v`). + +### 3. Shelved changelist + **`-l` / `--local`** (`-s -l`) + +Requires **`-s`**. For every shelved file, the script: + +1. Materializes the **shelved** revision with **`p4 print -q`** into a **temp file** (same symlink newline handling as the default shelved-add path). Revision spec is **`depot@=CL`** for all operations except **`delete` / `move/delete`**, which use **`depot#rev`** from `p4 describe` (the revision being removed). +2. Resolves the workspace path with **`p4 where`**. +3. Runs **`diff -u LEFT TEMP`** where **`LEFT`** is the local file if **`os.path.isfile`** is true, else **`/dev/null`**. +4. Rewrites diff headers so the left side is labeled **`… (workspace)`** when present and the right side **`… (shelved)`**. + +This is the mode to use when you care about **your tree vs the shelf**, not **`p4 diff2`** between two depot revisions. + +| Operation | `p4 print` spec | Left side of `diff -u` | +|-----------|-----------------|-------------------------| +| **add**, **branch**, **move/add**, **edit**, **integrate** | `depot@=CL` | Workspace file or `/dev/null` | +| **delete**, **move/delete** | `depot#rev` | Workspace file or `/dev/null` | + +**Note:** This is **similar in spirit** to **`p4 diff file@=CL`** for edits, but implemented as **`diff`** on **`p4 print`** output vs the file on disk, so whitespace or encoding handling may not match Perforce’s internal diff exactly. + +## Environment: `P4DIFF` + +Before each `p4` subprocess, the script copies your environment and sets **`P4DIFF`** to `diff` unless **`P4DIFF`** is already set. That controls which program `p4` uses when it needs an external diff. Your plain `diff -u` invocations for add/delete still use the string `diff` from `get_diff_tool()` (same default / same override). + +## Shelved vs local workspace + +- **Default shelved mode (`-s` only):** **edit**/**integrate** use **`p4 diff2`** (depot `#base` vs depot `@=shelf`), not your workspace file. +- **`-s -l`:** workspace file (or **`/dev/null`**) vs **`p4 print`** of the shelved revision — see **§3** above. + +For a single file, native **`p4 diff path@=`** may still differ in details from **`-l`** (Perforce vs external **`diff`**). + +## Examples + +```bash +# Pending change: diff everything in your default changelist (same with or without -c default) +./p4-diff > default.diff +./p4-diff -c default > default.diff + +# Shelved change 123456 (depot-vs-depot for edits) +./p4-diff -c 123456 -s > shelved_123456.diff + +# Shelved change: workspace (or /dev/null) vs shelved print for every file +./p4-diff -c 123456 -s -l > shelved_123456_vs_local.diff + +# Opened (non-shelved) numbered change +./p4-diff -c 123456 > opened_123456.diff +``` + +Dry-run applying a saved patch (strip level depends on how the diff was generated; adjust `-p` as needed): + +```bash +cd ~/other_tree +patch --dry-run -Np1 -i opened_123456.diff +``` + +## Output and exit behavior + +- Diffs are written to **stdout**; errors from `p4` on stderr cause the script to print them and **`sys.exit(1)`**. +- Subprocess text is decoded as **UTF-8** with **`errors='replace'`**; binary-ish depot files may not round-trip perfectly through the temp-file path. + +## Limitations / caveats + +1. **`p4 describe -S -s` parsing** is line-based and assumes fields split on spaces match the script’s expectations; unusual depot paths or future output changes can break parsing. +2. **`p4 where` output** is split on spaces and the **third** token is taken as the local path — same fragility for paths with spaces. +3. **Temp files** for shelved add/delete use `NamedTemporaryFile(delete=False)`; they are not deleted by the script (same as the original design intent: short-lived paths for `diff`). +4. **`integrate`** is grouped with **edit** for diff generation. +5. **`--verbose`** corrupts a pure-patch stdout stream by interleaving command traces. +6. **Omitting `-c`** uses the literal changelist name **`default`**, which is right for **`p4 opened -c default`** but is often **wrong for shelved work** (`p4 describe -S`); pass **`-c `** for shelves. + +## Repository layout + +| File | Role | +|------|------| +| `p4-diff` | Executable script (Python 3, `uv` shebang). | +| `README.md` | This document. | + +Related public tool with the same general idea: [JonParr/p4-diff](https://github.com/JonParr/p4-diff). This tree may include local changes (Python 3, `uv` shebang, README). diff --git a/p4-diff b/p4-diff index 472e310..68fa3d3 100755 --- a/p4-diff +++ b/p4-diff @@ -1,4 +1,4 @@ -#!/usr/bin/env python +#!/usr/bin/env -S uv run python3 import sys import os @@ -22,14 +22,33 @@ def userHasPerforce(): return True return False +def get_diff_tool(): + my_env = os.environ.copy() + my_diff_tool = "diff" + if 'P4DIFF' in my_env.keys(): + my_diff_tool = my_env["P4DIFF"] + return my_diff_tool + +def override_env(): + my_env = os.environ.copy() + my_env["P4DIFF"] = get_diff_tool() + return my_env + def runCommand(cmd): if Verbose: - print 'Executing: {}'.format(cmd) - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + print('Executing: {}'.format(cmd)) + p = subprocess.Popen( + cmd, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + env=override_env(), + encoding='utf-8', + errors='replace', + ) # subprocess.Popen( (out,err) = p.communicate() if len(err) > 0: - print 'error: {}'.format(err) - exit(1) + print('error: {}'.format(err)) + sys.exit(1) return (out,err) def convertDepotPathToLocal(depot_path): @@ -40,77 +59,103 @@ def convertDepotPathToLocal(depot_path): # on a single line, so just grab the third item. return out[2].strip() +def p4_print_revision_to_temp(revision_spec): + """Run p4 fstat + p4 print -q for the same depot revision spec (…@=CL or …#rev). + + Returns the path to a closed temp file (NamedTemporaryFile(delete=False).name). + """ + command = getP4Command() + ['fstat', '-T', 'headType', revision_spec] + (content, _) = runCommand(command) + fileType = content.split('\n')[0].split(' ')[2] + command = getP4Command() + ['print', '-q', revision_spec] + (content, _) = runCommand(command) + tmp = tempfile.NamedTemporaryFile(delete=False) + with open(tmp.name, 'w', encoding='utf-8', errors='replace') as tmpfile: + if fileType == 'symlink': + tmpfile.write(content[:-1]) + else: + tmpfile.write(content) + return tmp.name + def generateFileDiff(new,newfile): + my_diff_tool = get_diff_tool() if new: - command = ['diff','-u','/dev/null',newfile] + command = [my_diff_tool,'-u','/dev/null',newfile] else: - command = ['diff','-u',newfile,'/dev/null'] + command = [my_diff_tool,'-u',newfile,'/dev/null'] (diff,_) = runCommand(command) - print diff + print(diff) return diff -def generateAddedFileDiffFromShelved(changelist,depot_path): - # p4 print adds an extra newline only for symlinks, so we have to check the - # file type and remove it in this case. - command = getP4Command() + ['fstat','-T','headType','{}@={}'.format(depot_path,changelist)] - (content,_) = runCommand(command) - fileType = content.split('\n')[0].split(' ')[2] - command = getP4Command() + ['print','-q','{}@={}'.format(depot_path,changelist)] - (content,_) = runCommand(command) - tmp = tempfile.NamedTemporaryFile() - tmpfile = open(tmp.name,'w') - if fileType == 'symlink': - tmpfile.write(content[:-1]) - else: - tmpfile.write(content) - tmpfile.close() - command = ['diff','-u','/dev/null',tmp.name] - (diff,_) = runCommand(command) +def generateAddedFileDiffFromShelved(changelist, depot_path): + spec = '{}@={}'.format(depot_path, changelist) + tmp_name = p4_print_revision_to_temp(spec) + my_diff_tool = get_diff_tool() + command = [my_diff_tool, '-u', '/dev/null', tmp_name] + (diff, _) = runCommand(command) lines = diff.split('\n') # replace tmp file name with the depot one for clarity - lines[1] = lines[1].replace(tmp.name,depot_path) - print '\n'.join(lines) - -def generateDeletedFileDiffFromShelved(changelist,depot_path,file_revision): - # p4 print adds an extra newline only for symlinks, so we have to check the - # file type and remove it in this case. - command = getP4Command() + ['fstat','-T','headType','{}#{}'.format(depot_path,file_revision)] - (content,_) = runCommand(command) - fileType = content.split('\n')[0].split(' ')[2] - command = getP4Command() + ['print','-q','{}#{}'.format(depot_path,file_revision)] - (content,_) = runCommand(command) - tmp = tempfile.NamedTemporaryFile() - tmpfile = open(tmp.name,'w') - if fileType == 'symlink': - tmpfile.write(content[:-1]) - else: - tmpfile.write(content) - tmpfile.close() - command = ['diff','-u',tmp.name,'/dev/null'] - (diff,_) = runCommand(command) + lines[1] = lines[1].replace(tmp_name, depot_path) + print('\n'.join(lines)) + +def generateDeletedFileDiffFromShelved(changelist, depot_path, file_revision): + spec = '{}#{}'.format(depot_path, file_revision) + tmp_name = p4_print_revision_to_temp(spec) + my_diff_tool = get_diff_tool() + command = [my_diff_tool, '-u', tmp_name, '/dev/null'] + (diff, _) = runCommand(command) lines = diff.split('\n') # replace tmp file name with the depot one for clarity - lines[0] = lines[0].replace(tmp.name,depot_path) - print '\n'.join(lines) + lines[0] = lines[0].replace(tmp_name, depot_path) + print('\n'.join(lines)) + +def generateShelvedVsLocalDiff(changelist, depot_path, operation, file_revision): + """Unified diff: workspace file (---) vs shelved depot content (+++, from p4 print temp). + + Missing workspace file uses /dev/null as the left side. For shelved delete/move-delete, + the temp holds ``p4 print`` of ``depot#rev`` (revision being removed), still compared to local. + """ + if operation in ('delete', 'move/delete'): + if not file_revision: + print('error: {} has no file revision for {}'.format(operation, depot_path)) + sys.exit(1) + spec = '{}#{}'.format(depot_path, file_revision) + else: + spec = '{}@={}'.format(depot_path, changelist) + + tmp_name = p4_print_revision_to_temp(spec) + local_path = convertDepotPathToLocal(depot_path) + left_path = local_path if os.path.isfile(local_path) else '/dev/null' + my_diff_tool = get_diff_tool() + command = [my_diff_tool, '-u', left_path, tmp_name] + (diff, _) = runCommand(command) + lines = diff.split('\n') + if len(lines) >= 2: + if left_path == '/dev/null': + lines[1] = lines[1].replace(tmp_name, depot_path + ' (shelved)') + else: + lines[0] = lines[0].replace(left_path, local_path + ' (workspace)') + lines[1] = lines[1].replace(tmp_name, depot_path + ' (shelved)') + print('\n'.join(lines)) def generateEditFileDiff(depot_path): command = getP4Command() + ['diff','-du',depot_path] (diff,_) = runCommand(command) if len(diff.split('\n')) != 3: - print diff + print(diff) return diff if Verbose: - print 'Empty diff detected for {}'.format(depot_path) + print('Empty diff detected for {}'.format(depot_path)) return '' def generateEditFileDiffFromShelved(base_revision, changelist,depot_path): command = getP4Command() + ['diff2','-u','{}#{}'.format(depot_path,base_revision),'{}@={}'.format(depot_path,changelist)] (diff,_) = runCommand(command) if len(diff.split('\n')) != 1: - print diff + print(diff) return diff if Verbose: - print 'Empty diff detected for {}'.format(depot_path) + print('Empty diff detected for {}'.format(depot_path)) return '' def getOpenedFiles(changelist): @@ -128,7 +173,7 @@ def getOpenedFiles(changelist): file_revision = None assert(fields[1] == '-') operation = fields[2] - assert(operation in ['add','edit']) + assert(operation in ['add','branch','move/delete','move/add','edit','delete','integrate']) opened_files.append((operation,file_name,file_revision)) return opened_files @@ -147,58 +192,83 @@ def getShelvedFiles(changelist): assert(operation in ['add','branch','move/delete','move/add','edit','delete','integrate']) shelved_files.append((operation,file_name,file_revision)) return shelved_files - + def main(): p = argparse.ArgumentParser(description='Generate a diff from Perforce given the changelist number. If you wish to generate the diff from a shelved changelist, please specify the --shelved/-s option.', prog='p4-diff') - p.add_argument('--changelist','-c',metavar='CHANGELIST',help='Perforce changelist number, e.g. 123456',required=True) + p.add_argument( + '--changelist', + '-c', + metavar='CHANGELIST', + default='default', + help='Perforce changelist number (default when omitted: default). Example: 123456', + ) # p.add_argument( p.add_argument('--shelved','-s',action='store_true',help='Changelist is shelved') + p.add_argument( + '-l', + '--local', + action='store_true', + help='With --shelved: diff workspace file (---) vs shelved p4 print content (+++); ' + 'missing workspace file uses /dev/null on the left.', + ) # p.add_argument( p.add_argument('--ticket','-t',metavar='P4TICKET',help='If required, use a P4TICKET to authenticate.') p.add_argument('--verbose','-v',action='store_true',help='Enable verbose output useful such as executing Perforce commands. Note, this will corrupt the diff output') options = p.parse_args() + if options.local and not options.shelved: + p.error('--local (-l) requires --shelved (-s)') + if not userHasPerforce(): - print 'error: could not find perforce executable.' - exit(1) - + print('error: could not find perforce executable.') + sys.exit(1) + changelist = options.changelist global Verbose Verbose = options.verbose global Ticket Ticket = options.ticket - + if options.shelved: file_list = getShelvedFiles(changelist) else: file_list = getOpenedFiles(changelist) if not len(file_list): - print 'No files to generate diff from.' + print('No files to generate diff from.') else: for (op,f,file_revision) in file_list: local_f = convertDepotPathToLocal(f) if Verbose: - print '{} was open for {} [{}]'.format(local_f,op,f) + print('{} was open for {} [{}]'.format(local_f,op,f)) if op == 'add' or op == 'branch' or op == 'move/add': if options.shelved: - generateAddedFileDiffFromShelved(changelist,f) + if options.local: + generateShelvedVsLocalDiff(changelist, f, op, file_revision) + else: + generateAddedFileDiffFromShelved(changelist, f) else: generateFileDiff(True,local_f) elif op == 'edit' or op == 'integrate': if options.shelved: - generateEditFileDiffFromShelved(file_revision,changelist,f) + if options.local: + generateShelvedVsLocalDiff(changelist, f, op, file_revision) + else: + generateEditFileDiffFromShelved(file_revision,changelist,f) else: generateEditFileDiff(f) elif op == 'delete' or op == 'move/delete': if options.shelved: - generateDeletedFileDiffFromShelved(changelist,f,file_revision) + if options.local: + generateShelvedVsLocalDiff(changelist, f, op, file_revision) + else: + generateDeletedFileDiffFromShelved(changelist, f, file_revision) else: generateFileDiff(False,local_f) else: - print ('Unhandled operation {}'.format(op)) + print('Unhandled operation {}'.format(op)) assert(False) if __name__ == '__main__':