GIT information

[IkarusDowned]

This discussion is mostly for those of us with experience using other source control methods such as CVS and SVN, and are now working with Git / Github for Valyria Tear. I'd like to try and pool some resources into here as to the "standard" ways of doing things, and tips and tricks for people who are new to git.

So let me start with what I know so far:

Getting your own "master" repo of the current code:
1) Request a Fork() from github

from Roots:
Pro Git Book
http://git-scm.com/book

Article on GitHub Workflow
http://scottchacon.com/2011/08/31/github-flow.html

a method of forking
https://help.github.com/articles/fork-a-repo

From Bertram; his recommended way of pulling from the master branch
When resyncing with my repo, I'd advise you to use this command:
git pull --rebase 'my_repo_git_address'

Why? Because:
git pull is roughly equivalent to: git fetch + git merge.
And the --rebase option is an awesome one making every unmerged commits made on your own, be re-applied on top of your own git history.
This has several qualities:
1. It prevents the creation of merge points, which make unnecessary noise in your pull requests.
2. It makes sure your commits are always compatible with the reference repo.
3. It doesn't change the content of pull requests while updating your repository branch.

For those Windows users out there
1) I recommend PortableGit as opposed to the standard binary install. Get used to the Bash shell version of this, and I think you'll find it better than the normal version. I recommend it because it doesn't make an nasty changes to your registry hive, and more importantly it allows you to move your local git repo AND the binary around on a USB drive.
http://code.google.com/p/msysgit/downloads/list

Proxy settings for git
1) Determine your proxy information. If you have a standard .pac proxy it is usually port 8080, but that may not neccesarily be the case. You can get most of this info from your Internet Settings or from your local Admin
2) run:
git config --global http.proxy <proxy info>
git config --system http.sslcainfo /bin/curl-ca-bundle.crt (this step may be windows only...not sure)

working with branches
making a local branch:
git checkout -b mybranch

pushing to branch:
git push origin mybranch

and finally, deleting a branch:
git branch -d mybranch

deleting a remote branch (AKA the one you pushed)
git push origin :mybranch

past questions (now answered by members)
Questions:
1) is it best to work directly against my master? Or should I immediately branch so that when Bertram / someone else makes an "upstream" commit, I can pull to the master and trickle down (other wise, it seems I have to roll back commits and looks like a headache...)
2) is git mergetool particulary useful for the above problem
3) are there better ways to get the code and maintain a "my master" vs "global master" ?

[Roots]

I approve of this topic. I've been struggling with git as well. Here are some resources that I've collected but haven't had the time to study yet.

Pro Git Book
http://git-scm.com/book

Article on GitHub Workflow
http://scottchacon.com/2011/08/31/github-flow.html


In regards to your questions, here's my current understanding on them:
1) Bertram told me to always use a branch. For everything. I really dislike this, as if I'm just doing a quick bug fix or something, I don't want to go through the steps of creating a branch, checking it out, making my change, grabbing the latest upstream, merging the branch back into master, and then pushing my commit to the server. I feel like the same thing should be accomplished if I made changes to my master, grabbed the latest upstream, merged my changes with upstream, and then push. Shouldn't it be the same?
2) I keep reading about how the git merge tool is supposed to be some holy power to be worshiped, but I've already had problems with it in the relatively minor changes that I have made.
3) I've been following the guide here for this: https://help.github.com/articles/fork-a-repo . So I made the upstream repository which points to Bertram's master. When I need to make a change, I fetch the latest upstream repository and then merge it into my master. After resolving any merge conflicts, then I push it to my remote master and send a pull request to Bertram.

[IkarusDowned]

Thanks Roots!
I'll top-add to the original post as we get more and more info. hopefully, with enough information we can maybe sticky the thread.

[IkarusDowned]

aaargh, GIT now gives me "unable to determine absolute path of git directory" for my project, which is on a USB external drive...

I have to admit, GIT is giving me way more headaches than I thought it would...

[Bertram]

Hi all,

This topic is a very good initiative, and I must say I was pondering the fact of adding such a thing somewhere seeing all the problems people new to git seem to have.

Let me first react on Roots own reactions:

1) Bertram told me to always use a branch. For everything. I really dislike this, as if I'm just doing a quick bug fix or something, I don't want to go through the steps of creating a branch, checking it out, making my change, grabbing the latest upstream, merging the branch back into master, and then pushing my commit to the server. I feel like the same thing should be accomplished if I made changes to my master, grabbed the latest upstream, merged my changes with upstream, and then push. Shouldn't it be the same?

Short answer: Yes, the same can go for master and for a fix you're quite sure of, BUT:
- It seems you had problems resyncing your master the last time, and using branches is the common way used to avoid this.
It's better to delete/re-create a branch than having a messed up master with nothing easily done to do before resyncing. So you can push to master and do merge requests, but you're warned.
People at ease with history rewriting stuff, I mean interactive rebasing, squashing, high conflicts handling and all that can surely only stay in master. But themselves will use
branches to avoid the pain of that. An I do think you're not at ease with those notions yet, and even if I don't doubt you will anytime soon.
Also, If you've read this article, you might notice they also advise to use branches:
https://help.github.com/articles/using-pull-requests

- A short fix can become more than that, preventing you from resyncing freely until the fix is accepted. An since common merge requests are taking in account all the unmerged commits, using master prevents you from working on several stuff at a time without getting into troubles. Even when at ease with git.
I mean, working on a project alone or with a low-level of code review is fine, and corresponds well to the SVN times. Yet, working as a community and thrive for a quality project requires much more discipline for me and for you all, as we must all try to keep the reference repository code changes readable as much as the code itself is, so that people can understand easily what is happening when reading the log and the diffs.
I you prefer to let people push, and then do fixes afterwards, it's your choice. But I prefer having the classic merge-request/review method that prevents certain easy bugs to go in, to have a clearer history for everyone, and permit everyone to get to know about the code common practices before pushing stuff to the repository.
If you're not happy with it, well, c'est la vie, but I can swear I'm actually not the most pedantic about this. I do think I'm actually quite gentle and do compromises when I can.

2) I keep reading about how the git merge tool is supposed to be some holy power to be worshiped, but I've already had problems with it in the relatively minor changes that I have made.

I won't go into that. Some people don't like git, some does, etc...
The fact is that GIT is a more evolved tool than SVN is, more evolved and thus more complex as it is taking into account more complex needs.
I told you would have some hard times with git the first times. It's no surprise.
Yet, the github awesome interface, and the tools offered with git for my own management of your pull requests and other stuff is something I absolutely appreciate.
I'm not gonna change that.

3) I've been following the guide here for this: https://help.github.com/articles/fork-a-repo . So I made the upstream repository which points to Bertram's master. When I need to make a change, I fetch the latest upstream repository and then merge it into my master. After resolving any merge conflicts, then I push it to my remote master and send a pull request to Bertram.

Ah, speaking of which, i'd like to take the opportunity to point out the method I'd advise people to use when doing this:

When resyncing with my repo, I'd advise you to use this command:
git pull --rebase 'my_repo_git_address'

Why? Because:
git pull is roughly equivalent to: git fetch + git merge.
And the --rebase option is an awesome one making every unmerged commits made on your own, be re-applied on top of your own git history.
This has several qualities:
1. It prevents the creation of merge points, which make unnecessary noise in your pull requests.
2. It makes sure your commits are always compatible with the reference repo.
3. It doesn't change the content of pull requests while updating your repository branch.

So, please use it, and get used to it.

[Bertram]

I'll top-add to the original post as we get more and more info. hopefully, with enough information we can maybe sticky the thread.

Yep, good idea. Let's keep to beat it while it's hot, so the necessary troubleshooting is good enough to makes this sticky.

aaargh, GIT now gives me "unable to determine absolute path of git directory" for my project, which is on a USB external drive...

Arg, mapping and remapping stuff. Btw, I wonder what tool are you using, and on what os?

It's just an supposition, but did you try to run 'git init' on the repository making troubles?
That put me out of troubles the last times, without messing up the repo data.

[IkarusDowned]

Heya Bert,

I'm gonna play around / look around at all the methods provided by both of you, and I'll annotate the top post with both info.

just FYI, I'm using Win7 64bit (at home). I'm completely comfortable with Linux as well and do back-end linux programming as my day job, but my home rig is windows based since i'm also a gamer

I've finally got a decent build of linux I like so I'm working on bootin that on the laptop as well, but yeah, i figure it can't hurt to have a a windows weirdo like me too

Also, as for the crash, I had upgraded some drivers to Beta versions and this seems to have crashed a whole bunch of files. Was recovering it when I got home, looks more stable now.
In light of that, for anyone ELSE out there using windows, I HIGHLY recommend Portable Git
http://code.google.com/p/msysgit/downloads/list
as opposed to the standard binary install

[Bertram]

Hi Nik,

I used msysgit in the past, and it saved my life for some other things but I didn't know it helped with a portable setup. Good to know, though.

Just in case, and since you're on windows, github has got its own suite of integrated tools (git for windows) which is quite nice when dealing with commit management.
Having people on several OS can't hurt at all, I think. And I've got also Win7 at home, so I won't call you a weirdo.

Regards,

[Bertram]

Hi,

I'll try to give my opinion on the questions form the first post:

1) is it best to work directly against my master? Or should I immediately branch so that when Bertram / someone else makes an "upstream" commit, I can pull to the master and trickle down (other wise, it seems I have to roll back commits and looks like a headache...)

My advice: A branch per feature / bugfix type. I do know it might sound too much for a quick fix.
Why-oh-why? Because:
- You can easily create a branch and delete it once the work is done or if you want to try again, or copy parts from one to another using cherry-pick, for instance. You cannot delete master without deleting your fork.
You can always brute force git push -f anything to anywhere but I'm not sure it will help to stay in sync with the upstream.
- Even quick fixes sometimes turn into several patches after review. And if your own master gets messy, your pull request using master as the commit list
gets messy.
- Using a branch for each feature type permits to have clean pull requests, and permits also to work on several things while the others review your changes
peacefully.

2) is git mergetool particulary useful for the above problem

If you use branches and pull --rebase wisely, the conflicts won't be that enormous most of the time. mergetool comes in handy when you want to make tries on big merge operations.
Yet, as every tool, it is automated and could do silly things. Push only what you understand and seems relevant to you! Use git diff and look at whether it looks fine, even roughly, before pushing.

3) are there better ways to get the code and maintain a "my master" vs "global master" ?

By using branches, you always have the master pristine, and easily pulled from upstream.
Using pull --rebase is also the best way to get your commits cleanly reviewable, would you pull from local master or global master.

Best regards,

[IkarusDowned]

Just collecting thoughts and commands as well...

once going through the nice tutorial Roots provided, I created a branch using:
git branch working
#working is the name of the branch I made
git checkout working
git pull --rebase upstream master
#this pulls all the stuff from Bertram's master. I had used git config as per tutorial to set Bertram's git as the upstream
#I was up to date at this point, but it is good practice
git push origin working
#add the branch to github
#now, in theory I can have a "working" branch I can use when I am both at home and at office / out on the road somewhere, without
#disturbing the master branch.
#????
#profit

[IkarusDowned]

for undoing commits, it seems the thing to remember:
if the HEAD is "commit" deadbeaf
and the commit you want to remove is beafdead

git rebase -i HEAD^^
or
git rebase -i deadbeaf

and remove the beafdead line

[Bertram]

I'll add that deadbeaf is the commit hash of the last good commit, the last commit you don't want to touch.

rebase -i permits to reorder also the commit order, by copy/pasting the lines proposed if more than one commits is listed.
but also to squash, reword, and some other things I've forgotten.

Another thing that you might fall onto:
I made a commit but I have to split it in several parts. You might want to know about git soft reset and git add -p:
http://stackoverflow.com/questions/1440 ... two-in-git

note that when the split can be done on different files, you just need to do a soft reset 'git reset HEAD^' (not --hard)
and git commit only-the-files-from-first-point
and then git commit the-rest.
Et voilà !

[IkarusDowned]

I noticed that if i do a rebase on my own branch,
even if i do a defensive git pull before hand, i still get a "non-fast-forward" rejection on a git push
I get around it by simply doing a git push -f

but that doesn't seem "right."
I checked the man page but it was generally pretty useless to me; I had trouble understanding the real point.
looks like, instead of -f i can do a --no-ff and that will work?

[Bertram]

@Ikarus,

Nope, it is normal.

As long as your remote commit history and your local history have unmergeable differencies, you have to tell git that your (local) history is the one to use.
Thus, git push -f is the right command to do that while keeping a clean git history remotely without annoying merge points.

Just make sure everything unmerged with upstream remotely is there locally or you'll lose work.

Best regards,

[Roots]

I have a git comment/concern/questio related to this issue:

https://github.com/Bertram25/ValyriaTear/pull/36

So what happened here was I made a commit, pushed it to my remote, submitted a pull request, made another commit, pushed it to my remote (which automatically added it to the existing pull request). And then you appear to have accepted it as it was, without adding any changes of your own. But then, Bertram, you said I had to delete my own commits. That makes no sense to me, and seems like a bad idea if I'm having to delete things that I have already pushed out to my repository.

I discussed this with some folks on IRC, and I was told that I have to delete it because if my commit doesn't match with his, I wouldn't be able to pull from his repository in the future. -BUT- if that's not the case (the commits do match) then I do not have to delete my commits.

So, what is it that I need to do? I'm scared to try anything after the last git headache I suffered. I'd like to get all the latest changes from your repository, but I'm not making a move until I understand A) what I need to do (if anything) and B) why I need to do it.

[Bertram]

Hi Roots,

I discussed this with some folks on IRC, and I was told that I have to delete it because if my commit doesn't match with his, I wouldn't be able to pull from his repository in the future. -BUT- if that's not the case (the commits do match) then I do not have to delete my commits.
So, what is it that I need to do? I'm scared to try anything after the last git headache I suffered. I'd like to get all the latest changes from your repository, but I'm not making a move until I understand A) what I need to do (if anything) and B) why I need to do it.

You might have noticed that I closed the pull request without merging it. Meaning there is no merge point, like this one for instance:
https://github.com/Bertram25/ValyriaTea ... 76eb8b3bcd
As there is none, git doesn't know the commits are the same or not.

Note that it also has mechanisms to recognize when a commit is 100% the same than an unknown one and deal with it as the known one.
But in this case, I've squashed (merged) your two commits into one before pushing and added my own fix afterwards. This means the overall change matches, but not the commits since there is only one commit corresponding to your changes, not two.
Hence, to me, you'd better delete them before pulling/rebasing.

Note again that I don't do that to make your life difficult, but rather to have cleaner commits in the reference repository when possible.
you can simply git reset --hard your own commits if you haven't any other changes in master of course, and pull again.

Then, to make your remote repository in sync, you'll have to do: git push -f.

Now, for the case where I accept your pull request as is, you'll see a merge point in my commit history, meaning you can surely pull without doing all that.
If you want me to accept your pull requests more "as is", I can also tell you what to change before I accept it if you want, and if you're enough at ease with interactive rebasing. (git rebase -i)

I hope it answers your questions.

Regards,

[charlie]

Bertarm: on "cleaner" commits, I'm pretty sure that one of the explicit drivers of Git's design was to make sure you commit often. Commits were not intended to be big and clean (as in "finish feature X"). They were intended to be small and atomic.

http://sethrobertson.github.com/GitBestPractices/

Just pull or ask to rebase next time. Don't create problems for other developers by refactoring their commits because you are OCDing on the commit log. Don't confuse commit log and changelog.

If you have a minor issue with a developer's change, either pull and then commit your fix and ask them to pull that, or direct them to fix it themselves accordingly.

Changing somebody's commits is a Git worst practice and creates work for you and for them and tension in the project.

[charlie]

Just to add to that; if you want Roots to rebase his commits to make them more changelog friendly, then ask him, don't do it for him.

[Bertram]

Charlie,

Bertarm: on "cleaner" commits, I'm pretty sure that one of the explicit drivers of Git's design was to make sure you commit often. Commits were not intended to be big and clean (as in "finish feature X"). They were intended to be small and atomic.

Hmm, that's what you say. Yet, I'm perfectly sure it's false in many places.

"Best Practices vary from environment to environment, and there is no One True Answer", right?

Just pull or ask to rebase next time. Don't create problems for other developers by refactoring their commits because you are OCDing on the commit log. Don't confuse commit log and changelog.

The one OCDing here isn't me. I'm just pushing in my repository what I find relevant.

If you have a minor issue with a developer's change, either pull and then commit your fix and ask them to pull that, or direct them to fix it themselves accordingly.

I would, if Roots was at ease to do it. I'm just trying to be proactive here. And I don't see the difficulty in erasing two obsolete commits one's repo before pulling.

Changing somebody's commits is a Git worst practice and creates work for you and for them and tension in the project.

Just to add to that; if you want Roots to rebase his commits to make them more changelog friendly, then ask him, don't do it for him.

Once again, I'm doing it only when people are not at ease to do it themselves.

Charlie, I don't like the tone of those two last comments. In the manaworld project, we kept having problems with people which were not willing or not at ease with git to refactor their commits
before inclusion. By only pointing out problems in the commits and waiting for their fixes, stuff wouldn't never be pushed before weeks. That's what I'm trying to avoid here.
If Roots wants to do stuff himself, then fine. But he could just have said it himself or you could have asked what why I did what I did before playing the boss with me.

You say I'm creating tensions, fine. But you've just created one here.

and my nickname is Bertram, not Bertarm.

[Bertram]

Charlie, I don't like the tone of those two last comments. In the manaworld project, we kept having problems with people which were not willing or not at ease with git to refactor their commits
before inclusion. By only pointing out problems in the commits and waiting for their fixes, stuff wouldn't never be pushed before weeks. That's what I'm trying to avoid here.
If Roots wants to do stuff himself, then fine. But he could just have said it himself or you could have asked what why I did what I did before playing the boss with me.

You say I'm creating tensions, fine. But you've just created one here.

Charlie and me just mailed each other. I think I created a problem where there weren't. My public apologize for such a thing.

Hence, on Charlie's advice, I'll let developers rebase themselves from now on and if necessary, it was my goal from the start, I swear it, as long as people is ready and at ease to do it themselves, so let's do it.

[Roots]

You might have noticed that I closed the pull request without merging it. Meaning there is no merge point, like this one for instance:
https://github.com/Bertram25/ValyriaTea ... 76eb8b3bcd
As there is none, git doesn't know the commits are the same or not.

Okay, that makes sense. Now I understand why the delete is needed, thanks. I had looked at the changes and I didn't see anything that you added/removed/changed from my original commits, so I guess I just missed something that you changed.

Now, for the case where I accept your pull request as is, you'll see a merge point in my commit history, meaning you can surely pull without doing all that.
If you want me to accept your pull requests more "as is", I can also tell you what to change before I accept it if you want, and if you're enough at ease with interactive rebasing. (git rebase -i)

In the future, I'd greatly prefer that you accept pull requests is there's nothing terribly wrong with them. If you want to make a minor logical change or whatever, please be my guest. Just accepte the pull request, then make your own changes in an additional individual commit. That way you get the code you want and at the same time I don't have to jump through hoops and delete my commit history. Not to mention that the project history won't reflect that I worked on X when you delete my work and resubmit it as your own.

Now if you want to deny a pull request because there's a serious bug or flaw that would break the compilation or otherwise make the game unplayable, that I would totally understand. As a rule of thumb you never want to make such commits to your master repository.

Just pull or ask to rebase next time. Don't create problems for other developers by refactoring their commits because you are OCDing on the commit log. Don't confuse commit log and changelog.

If you have a minor issue with a developer's change, either pull and then commit your fix and ask them to pull that, or direct them to fix it themselves accordingly.

Changing somebody's commits is a Git worst practice and creates work for you and for them and tension in the project.

I agree with the above, especially the last sentence (and I've been told the same by other git users on IRC). It has been frustrating. You did ask me to make a change to my last pull request and I did as was asked, yet you made additional changes on top of what was asked and what I said I would do.

If you have a minor issue with a developer's change, either pull and then commit your fix and ask them to pull that, or direct them to fix it themselves accordingly.

I would, if Roots was at ease to do it. I'm just trying to be proactive here. And I don't see the difficulty in erasing two obsolete commits one's repo before pulling.

It is difficult because I'm doing more work while I'm waiting for my pull requests to be accepted/merged/rejected/whatever. Yes, I'm doing my work in branches so not all of it is affected, but often I go on to work on things that build upon the commits that are in my submitted pull request. If I have to wait a day or more every time I submit a pull request because there's a chance that I may have to delete those commits and re-pull, that's a big hassle (as I've already experienced it). It also slows down the rate at which I (or another) can contribute to the project if I'm paranoid that I'm going to have to delete previous commits and don't want to continue working on that which was already submitted.

In the manaworld project, we kept having problems with people which were not willing or not at ease with git to refactor their commits
before inclusion. By only pointing out problems in the commits and waiting for their fixes, stuff wouldn't never be pushed before weeks. That's what I'm trying to avoid here.
If Roots wants to do stuff himself, then fine. But he could just have said it himself or you could have asked what why I did what I did before playing the boss with me.

For the record, I'm fine with rebasing/refactoring commits. You just need to explain to me what to do, and linking that article like Charlie did there would have been enough for me to figure out how to do it on my own. I am new to git so I'm ignorant, but I'm not stupid. As for Manaworld, I understand your past frustrations with the way that project was loosely run and resulted in a lot of sloppy commits and code. So I think it's a very good thing that you care about code quality. But I'd argue that you're almost now on the other extreme, ruling over the codebase with an iron fist. From my experience, there is a happy medium you need to find between the two ends of the spectrum (between letting anyone commit whatever whenever, and maintaining absolute and total authority over every single line of code that is committed).

Keep in mind that about 98% of the VT code came from Allacrost, which used a CVS and later a SVN repository and commits were largely unmonitored after it was established that someone new to the project knew what they were doing and understood good software development practices. It wasn't always perfect and sometimes buggy code snuck in, but when it did there was always someone around to get on top of it and fix the problem, even if the original author of the change was unavailable. And through those practices, we got a really solid, well designed, well documented code base (for the most part anyway).

And it's okay to be OCD about VT. I fully admit that I am OCD when it comes to Allacrost. When you "own" a project, you get really protective of it and you want everything to be perfect about it. But the key to happy development and developers is to not allow your OCD to cause grief among those that work with you (it's not easy, and I've caused grief for my team in the past).


In closing, I just want to clarify that I'm not angry or upset at anyone and I'm not taking any sides in this debate. I'm simply expressing that the current way we've been doing things has been frustrating for me. I don't mean to create a conflict or tension. I simply want to figure out the best practices that we can adopt so that we all spend more time doing actual development work and less time dealing with repository micromanagement.

[Bertram]

Right,

I'm glad there is no actual anger out there, the frustration I've caused apart.

Let's do it that way, then.

When you make a pull request, I'll ask for an additional fix or refactor if really needed, but will overall accept without redoing it, except when asked for.

Would it be fine for everyone around?

[charlie]

Anger? Upset? Intrigue? Will Bertram butcher more commits? Will Roots understand Git? Will Charlie ever program something? Tune in to next week's FreeGameDev forum episode to find out - a forum.freegamedev.net exclusive!

[Bertram]

This is indeed becoming some kind of Gossip Girl episode

[IkarusDowned]

lol, alright, i want a good clean fight. no low blows and "yo mamma" jokes..

drama aside , this thread is really helpful for me since its a nice way for me to keep track of what commands are doing what.