Improve status output

[404Wolf]

[stevekrouse]

This is what I'm looking for:

$ vt status
On branch main @5 (local) | remote @9
Your branch is behind remote by 4 versions, and there are no local changes to push.

Nothing to pull.
Nothing to push.

$ vt status
On branch main @5 (local) | remote @10
Your branch is behind remote by 5 versions, and there are no local changes to push.

Changes you can pull:
  M (http) router.ts  (modified remotely)

Nothing to push.

$ echo "test" > test
$ vt status
On branch main @5 (local) | remote @10
Your branch is behind remote by 5 versions, and there are also untracked local changes.

Changes you can pull:
  M (http) router.ts  (modified remotely)

Changes you can push:
  A (file) test

$ rm router.ts
$ vt status
On branch main @5 (local) | remote @10
Your branch is behind remote by 5 versions, and there are also untracked local changes.

Changes you can pull:
  M (http) router.ts  (modified remotely)

Changes you can push:
  D (file) router.ts
  A (file) test

Maybe the missing piece is a three-way state model, like git has. The the three trees:

• BASE – the last remote snapshot your local dir acknowledged (tracked in .vt/index.json).
• LOCAL – the current working dir on disk.
• REMOTE – the current remote snapshot (branch head) with metadata for each file: path, type, mtime, size, content hash/etag.

This is then git’s “three-way” diff without commits. Then we ca use git status as a guide for what this should look like.

• Block push if behind, but you can force-push to overwrite remote
• Pull fast-forwards if clean, else marks conflicts

Does this make sense?

[404Wolf]

This is what I'm looking for:

$ vt status
On branch main @5 (local) | remote @9
Your branch is behind remote by 4 versions, and there are no local changes to push.

Nothing to pull.
Nothing to push.

$ vt status
On branch main @5 (local) | remote @10
Your branch is behind remote by 5 versions, and there are no local changes to push.

Changes you can pull:
  M (http) router.ts  (modified remotely)

Nothing to push.

$ echo "test" > test
$ vt status
On branch main @5 (local) | remote @10
Your branch is behind remote by 5 versions, and there are also untracked local changes.

Changes you can pull:
  M (http) router.ts  (modified remotely)

Changes you can push:
  A (file) test

$ rm router.ts
$ vt status
On branch main @5 (local) | remote @10
Your branch is behind remote by 5 versions, and there are also untracked local changes.

Changes you can pull:
  M (http) router.ts  (modified remotely)

Changes you can push:
  D (file) router.ts
  A (file) test

Maybe the missing piece is a three-way state model, like git has. The the three trees:

* BASE – the last remote snapshot your local dir acknowledged (tracked in .vt/index.json).

* LOCAL – the current working dir on disk.

* REMOTE – the current remote snapshot (branch head) with metadata for each file: path, type, mtime, size, content hash/etag.

This is then git’s “three-way” diff without commits. Then we ca use git status as a guide for what this should look like.

* Block push if behind, but you can force-push to overwrite remote

* Pull fast-forwards if clean, else marks conflicts

Does this make sense?

It sounds like you want to take into account changes since the last pull to do push and pull now.

A while back we spent a while discussing this and decided to not design vt like this, because this introduces a substantial amount of complexity. There are no "commits" locally, so there is no good way to track the evolution of changes. Instead, we decided to make vt only do full syncs in either direction.

Some of the questions we thought about a while back:

• What happens if you create a file locally that does not exist remotely? We don't know it wasn't just deleted remotely unless we walk the remote history from the offset stored in ./vt/state.json (which we could do!). Do we offer the user to push a creation and pull a deletion? But nothing was deleted remotely. This is like your example. We could see the mtime locally is more recent than the latest remote change, but maybe remote does some modifications after you create it locally.
• Likewise, if there was a remote modification to a file that doesn't exist locally, do we infer that that file got deleted locally? Maybe it was created remotely. We could walk remote history to see it was modified remotely, so it was just deleted locally. But maybe they renamed it locally and modified it remotely.
• If I edit something remotely to look like something locally, do we offer pulling a rename?
• There's more edge cases like the above two, I can try to think through what they were, but that's the general idea
• We aren't git, so we don't have any way to track changes made locally since there are no commits, we can only look at how it differs from the latest remote state
• We don't have an API endpoint to get a list of versions for a Val, we would have to add one

This is discussed more fully here.

At the time I proposed a sync algorithm that would solve all of this and get what you are expecting. We would offer the user between the pull I define, and summarize the results, or a push, where you can only push if your version is the highest or you force.

We ended up taking max's suggestion here as an effort to avoid this problem and keep things simple. The idea was that we would totally simplify this contract, and just say "push the entire diff and force remote to match local" or "pull the entire diff and force local to match remote" rather than designing a fancy algorithm.

I'm happy to start rethinking this algorithm, I like the delta tracking version, but this is a big change to vt's core logic.

[stevekrouse]

Ok, yes I see there's lots of complexity with the sync-y stuff.

In the meantime, could we make vt status be less misleading? We should at least be honest about how we are only detecting "differences" and can't tell you if you're ahead/behind, etc. Given that vt gives a lot of git vibes, we should really be clear with users that we're not doing git stuff here.

For example, we could say: "There are differences between local and remote. vt doesn't (yet) intelligently track which is ahead/behind, so you have to resolve this, either by vt pulling remote changes locally or vt pushing local changes remotely - depending on your context."

And then when we list the files that have differences we could label them as "differences" and not as "changes", because that's more precise, given that we actually don't know where they were changed.

[404Wolf]

Ok, yes I see there's lots of complexity with the sync-y stuff.

In the meantime, could we make vt status be less misleading? We should at least be honest about how we are only detecting "differences" and can't tell you if you're ahead/behind, etc. Given that vt gives a lot of git vibes, we should really be clear with users that we're not doing git stuff here.

For example, we could say: "There are differences between local and remote. vt doesn't (yet) intelligently track which is ahead/behind, so you have to resolve this, either by vt pulling remote changes locally or vt pushing local changes remotely - depending on your context."

And then when we list the files that have differences we could label them as "differences" and not as "changes", because that's more precise, given that we actually don't know where they were changed.

Cool, I agree! We should be displaying honest output -- to clarify, you'd like it to be like what this PR does now, but "Differences" instead of "Changes," and a disclaimer that it is all pull or all push?

[stevekrouse]

I'm proposing something different than what's in this PR. Here's what I'm proposing we move towards:

$ vt status
On branch main @5 (local) | remote @9
There are differences between local and remote.
vt doesn’t (yet) intelligently track which side is ahead or behind.
You’ll need to resolve this yourself — either by pulling remote changes locally
or pushing your local changes remotely, depending on your context.

Files with differences:
  M (http) router.ts

$ echo "test" > test
$ vt status
On branch main @5 (local) | remote @9
There are differences between local and remote.
vt doesn’t (yet) intelligently track which side is ahead or behind.
You’ll need to resolve this yourself — either by pulling remote changes locally
or pushing your local changes remotely, depending on your context.

Files with differences:
  M (http) router.ts
  A (file) test

$ rm router.ts
$ vt status
On branch main @5 (local) | remote @9
There are differences between local and remote.
vt doesn’t (yet) intelligently track which side is ahead or behind.
You’ll need to resolve this yourself — either by pulling remote changes locally
or pushing your local changes remotely, depending on your context.

Files with differences:
  D (file) router.ts
  A (file) test

What do you think?

[404Wolf]

I'm proposing something different than what's in this PR. Here's what I'm proposing we move towards:

$ vt status
On branch main @5 (local) | remote @9
There are differences between local and remote.
vt doesn’t (yet) intelligently track which side is ahead or behind.
You’ll need to resolve this yourself — either by pulling remote changes locally
or pushing your local changes remotely, depending on your context.

Files with differences:
  M (http) router.ts

$ echo "test" > test
$ vt status
On branch main @5 (local) | remote @9
There are differences between local and remote.
vt doesn’t (yet) intelligently track which side is ahead or behind.
You’ll need to resolve this yourself — either by pulling remote changes locally
or pushing your local changes remotely, depending on your context.

Files with differences:
  M (http) router.ts
  A (file) test

$ rm router.ts
$ vt status
On branch main @5 (local) | remote @9
There are differences between local and remote.
vt doesn’t (yet) intelligently track which side is ahead or behind.
You’ll need to resolve this yourself — either by pulling remote changes locally
or pushing your local changes remotely, depending on your context.

Files with differences:
  D (file) router.ts
  A (file) test

What do you think?

Sure, but the differences are symmetric. If we have a file locally that does not exist remotely, it is a "add" for push and a "remove" for pull. Do we want to only show the dry "push" items (this is what vt status lists out now)? This seems like what you are doing in your example.

In my opinion the current state of this PR (minus the wording) is most honest, since it explicitly says what will happen if you run a pull or a push.

[stevekrouse]

In my opinion the current state of this PR (minus the wording) is most honest, since it explicitly says what will happen if you run a pull or a push.

Ok, this helped. I think "what will happen if you run a pull or a push" is not relevant in a status message. That's what --dry-run is for.

vt status should just describe the state of local vs remote – as best it can, ie "Here are the differences we've detected between local and remote. We can't help you resolve them. You can either push or pull, based on your understanding of what's more up to date."

[404Wolf]

In my opinion the current state of this PR (minus the wording) is most honest, since it explicitly says what will happen if you run a pull or a push.

Ok, this helped. I think "what will happen if you run a pull or a push" is not relevant in a status message. That's what --dry-run is for.

vt status should just describe the state of local vs remote – as best it can, ie "Here are the differences we've detected between local and remote. We can't help you resolve them. You can either push or pull, based on your understanding of what's more up to date."

Okay. The differences between local and remote are relative to whether you would do a pull or do a push, though.

With git, it's relative to local uncommited state, and it's easy to see what's changed since the last commit, but with vt, we have local state and remote state, and it's possible both local state and remote state have changed. There's never a case in git where your previous commit changed, and your current state changed, and you're confused in quite the same way.

For all the reasons above, there's really no easy way in the current model to enumerate every change, and know with total certainty where it happened.

Now, we can see how the local state differs from the eyes of the remote state, or how the remote differs from the eyes of local, which is why we frame it as "what we would pull" or "what we would push." Git has commits, and sort of shows you "what you would commit."

If you don't want to show "what the remote sees is different" (i.e. what you would push) or what you would pull, I'm not really sure what you would like to show. Maybe having a vt status command to begin with isn't useful? Maybe we could only check where the newest modification was made and tell the user to pull if it was remote or push if it was local (this really only works for modifications though)?

[stevekrouse]

and know with total certainty where it happened.

Yes, I am trying to say that we should lean into the fact that we have no idea where the change happened and simply tell users that we know very little, and then list all the differences that we know about.

Assuming the state of the codebase is X and the remote is Y, we should say: "The status is {{ Y - X }}, but we can't tell you what that means. It's up to you to decide if you should push or pull."

Maybe having a vt status command to begin with isn't useful?

Yes! It's not very useful, but we need it because our users are coming from git and expect it to exist.

[stevekrouse]

Clarifying point: when I say {{ Y - X }}, I mean the differences, ie the "absolute value" of the changes, so {{ X - Y }} is fine too. I want to just list the differences we detect without implying a direction because as you said, we don't know that.

[404Wolf]

Clarifying point: when I say {{ Y - X }}, I mean the differences, ie the "absolute value" of the changes, so {{ X - Y }} is fine too. I want to just list the differences we detect without implying a direction because as you said, we don't know that.

I think this may break down since renames will look like modifications. Maybe we just list the names of all files (set addition) that show up in a dry push and pull "stuff that mismatches" without duplicates, and say "you can push or pull to resolve this" and don't tell people whether given files are renames, adds, deletions, or modifications.

[stevekrouse]

Exactly!

[stevekrouse]

@404Wolf - is this a quick thing to ship? Would be awesome if we could get something shipped here so we're being less confusing

[stevekrouse]

Specifically, ship this:

Maybe we just list the names of all files (set addition) that show up in a dry push and pull "stuff that mismatches" without duplicates, and say "you can push or pull to resolve this" and don't tell people whether given files are renames, adds, deletions, or modifications.