How to use GitHub (and Bitbucket)

last update:

I keep some notes relevant to bitbucket.org in here. Actually, I use Bitbucket for my personal stuff while GitHub in most professional settings I've been in.

I had to write this up after abandoning gitolite-admin for GitHub. It was nice not to have to do all that work myself anymore.


Steps to setting up a new account in GitHub
  1. Go to www.github.com.
  2. Sign in.
  3. Click on tools icon (Account settings).
  4. Fill in Public Profile, leave e-mail blank.
  5. Click SSH Keys in left-hand navigation pannel.
  6. Click "generating SSH keys" and follow the instructions to create and copy an SSH key to your clipboard. (There is an example below on this page.)
  7. Click Add SSK key.
  8. Give the key a suitably descriptive name.
  9. Paste the key on your clipboard into the space provided.
  10. Click on Add Repository button (book with a plus sign).
  11. Change owner to "powerful" owner (acme).
  12. Fill in repository name.
  13. Give repository a description.
  14. Make repository private (or public).
  15. Click green Create repository button.

The steps for Bitbucket are similar to those above.


Steps to setting up a new repository in Bitbucket

The steps are outlined at the website when you create a new repository. Here they are anyway because this can also be used for other Git hosting unless you insist on creating, then cloning the new repository.

  1. At the Bitbucket website, logged into your personal account, create the new repository.
  2. Using the wizard, give as much information as you'd like, but at least give it a name (duh), a short description, set the respository type (to Git, of course), and the principal language of expression (Java, Python, etc.).
  3. (For the host you're running on, you'll need an RSA key; I'm not covering that here.)
  4. When the respository page comes up, it will give you some commands to set up the respository. The ones you really need are:
    $ git init
    $ git remote add origin [email protected]:yourname/projectname.git
    
  5. Then add your new files (assuming you know how to use Git: git add .).
  6. Commit (git commit -m "Initial commit").
  7. And push (git push origin master).

How to clone a repository on a new host in Bitbucket
  1. Assuming git installed on the new host...
  2. In your filesystem, go to the parent subdirectory where you'd like your cloned project to live.
  3. If you don't already know the command, go to the repository page at Bitbucket; at the top, you'll see SSH and, to the right, a path. Copy that path to use below.
  4. Clone with this command:
    $ git clone [email protected]:yourname/projectname.git
    
  5. You should see your new project code, which will be the latest, ready to use.

How to migrate existing repository contents...

There are instructions, however, they're somewhat difficult to pursue. The following are easier, but they require that you be in your master branch with your local updated to everything remote has. username is the "power" username, like acme.

  1. $ git remote add github [email protected]:acme/projectname.git
  2. $ git push --all -f github
  3. $ git push --tags github
  4. Look at Github in browser to see that all is there.

Here are some console scrapes...

test-client:
master ~/dev/test-client $ git remote add github [email protected]:acme/test-client.git
master ~/dev/test-client $ git push --all github
To [email protected]:acme/test-client.git
 ! [rejected]        master -> master (non-fast-forward)
error: failed to push some refs to '[email protected]:acme/test-client.git'
To prevent you from losing history, non-fast-forward updates were rejected
Merge the remote changes before pushing again.  See the 'Note about
fast-forwards' section of 'git push --help' for details.
master ~/dev/test-client $ git push --all -f github
Counting objects: 553, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (441/441), done.
Writing objects: 100% (553/553), 1.77 MiB, done.
Total 553 (delta 218), reused 0 (delta 0)
To [email protected]:acme/test-client.git
 + 5f5f421...ad462aa master -> master (forced update)
master ~/dev/test-client $ git push --tags github
Everything up-to-date
master ~/dev/test-client $ git remote
github
origin
master ~/dev/test-client $ git remote set-url origin [email protected]:acme/test-client.git
master ~/dev/test-client $ git remote
github
origin
master ~/dev/test-client $ git remote -v
github	[email protected]:acme/test-client.git (fetch)
github	[email protected]:acme/test-client.git (push)
origin	[email protected]:acme/test-client.git (fetch)
origin	[email protected]:acme/test-client.git (push)
test-server:
master ~/dev/test-server $ git remote add github [email protected]:acme/test-server.git
master ~/dev/test-server $ git push --all -f github
Counting objects: 1848, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (1431/1431), done.
Writing objects: 100% (1848/1848), 26.73 MiB | 322 KiB/s, done.
Total 1848 (delta 855), reused 0 (delta 0)
To [email protected]:acme/test-server.git
 + db1dd06...efc4025 master -> master (forced update)
master ~/dev/test-server $ git push --tags github
Everything up-to-date

master ~/dev/test-server $ git remote set-url origin [email protected]:acme/test-server.git
master ~/dev/test-server $ git remote
github
origin
master ~/dev/test-server $ git remote -v
github	[email protected]:acme/test-server.git (fetch)
github	[email protected]:acme/test-server.git (push)
origin	[email protected]:acme/test-server.git (fetch)
origin	[email protected]:acme/test-server.git (push)

Migration command summary...
$ git remote add github [email protected]:username/projectname.git
$ git push --all -f github
$ git push --tags github
$ git remote set-url origin [email protected]:username/projectname.git

$ git remote -v

Once you've finished, you'll want to use the last command just to verify that when you start working again in the filesystem, your fetches and pushes will be going where you think they should (i.e.: to GitHub instead of to your old Git server).


Getting started: cloning a new repository...

Once you've created a new repository (or migrated an old one), you begin working in it by cloning it. (Of course, whether for GitHub or Bitbucket, you'll need to install appropriate ssh keys. See notes above.)

~/dev $ git clone [email protected]:acme/test-server.git
Initialized empty Git repository in /home/russ/dev/test-server/.git/
warning: You appear to have cloned an empty repository.
~/dev $ cd test-server
master ~/dev/test-server $ ll
total 0
drwxr-xr-x  3 russ russ  72 2012-10-11 15:08 .
drwxr-xr-x 10 russ russ 296 2012-10-11 15:08 ..
drwxr-xr-x  7 russ russ 248 2012-10-11 15:08 .git

Here's how I got Jenkins and GitHub wired up...

There's a lot of help out there for this, most of it complex pratting that doesn't work.

When you set up a Jenkins job, you tell it to kick off based on Git. You have to set up the Git plug-in in Jenkins, but you get the following error:

  Failed to connect to repository : Command "git ls-remote -h [email protected]:acme/test-server.git HEAD"
  returned status code 128:
  stdout:
  stderr: ERROR: Repository not found.
  fatal: The remote end hung up unexpectedly

Here's what I did to wire up Jenkins and GitHub.

  1. Fetched git.hpi and github.hpi plug-ins for Jenkins and dropped them on the path /var/lib/jenkins/plugins.
  2. Bounced Jenkins: sudo /etc/init.d/jenkins restart.
  3. Became user jenkins and generated a key pair with no passphrase:
    [email protected]:~/.ssh$ ssh-keygen
    Generating public/private rsa key pair.
    Enter file in which to save the key (/home/jenkins/.ssh/id_rsa):
    Enter passphrase (empty for no passphrase):
    Enter same passphrase again:
    Your identification has been saved in /home/jenkins/.ssh/id_rsa.
    Your public key has been saved in /home/jenkins/.ssh/id_rsa.pub.
    The key fingerprint is:
    7e:c7:9a:8a:90:ce:17:2c:b5:dd:28:3d:53:fa:03:d9 [email protected]
    The key's randomart image is:
    +--[ RSA 2048]----+
    |                 |
    |                 |
    |                 |
    |      .   .      |
    |     o +SB       |
    |    ..=.X E.     |
    |    o. o.=. o    |
    |   o ... .o+     |
    |    o.. ..o.     |
    +-----------------+
    
  4. Copied the resulting key pair to my Linux development host via scp.
  5. Copied the public key to the clipboard (because this is how to get it into GitHub):
    ~/Downloads/jenkins-deploy-keys $ xclip -selection c -i ./id_rsa.pub
    
  6. Logged into GitHub and navigated to my project. Clicked on the Admin tab, then clicked on Deploy Keys (left-hand navigational thumb), then Add deploy key. Gave the deploy key a descriptive name to do with "Jenkins deploy key for <server-name> and copied the key (already on the clipboard) into the space provided.
  7. Returned to console on my server to see if this was successful from the jenkins user:
    [email protected]:~$ ssh -T [email protected]
    Hi acme/test-client! You've successfully authenticated, but GitHub does not provide shell access.
    
  8. Return to the Jenkins dashboard, click on the project (if not created, do create it), then Configure, then scroll down to Source Code Management where Git should be checked with a Repository URL of [email protected]:username/projectname.git. There should be no red error message there.

Don't stop here; read the next section.


Here's how I really got Jenkins and GitHub wired up...

The problem with the method above, I quickly found out, is that it only works for one repository. GitHub won't let you add that RSA key for any other repositories since it's already been used.

So, I did an end-run...

  1. Created a new GitHub free user, Acme-Jenkins with a bogus e-mail address (waiting to see if this fails because I have no way to answer any confirmation e-mail if GitHub sends one and insists on it being answered).
  2. Logged into GitHub and established an RSA key, the one created for user jenkins in the previous section.
  3. (I removed this key from the repository I'd put it into.)
  4. From user jenkins on the server, I attempted to ssh into [email protected] to ensure that worked again.
  5. I logged in to GitHub as my poweruser self, clicked the Teams tab, created new team Acme-pull only. I added user Acme-Jenkins as a team member, then two repositories already created.
  6. I configured the jobs for both repository projects to use [email protected]:acme/projectname and refreshed them. There was no error about using Git from either one.

Unless GitHub doesn't insist upon a confirmation step as noted above, I should be home free.


Migration accomplished, how to cut over to GitHub

You must log into GitHub and add an RSA key. There's information elsewhere on this page on how to copy your RSA key to the Linux clipboard such that you can paste it accurately into GitHub.

If the migration was accomplished with everything committed and pushed to origin master, then the following commands should suffice to accomplish an individual's cutover.

    $ git checkout master
    $ git remote set-url origin [email protected]:acme/projectname.git

As I haven't migrated any repositories with branches, I don't yet know what will become of branch russ after the second command. I'll do that and find out then, if there's anything noteworthy, I'll let you know. I know that the migration actually covers branches too, but as I say, I haven't actually seen it work yet.


Error and hang for git push origin master

You've just committed some code, maybe (and especially) a lot of it, and are pushing it to the master remote. You find it hangs and, if you squint hard, that there was an error.

    master ~/bitbucket/zMongoTryStuff $ git push origin master
    Password:
    Counting objects: 31, done.
    Delta compression using up to 4 threads.
    Compressing objects: 100% (27/27), done.
    error: RPC failed; result=56, HTTP code = 0
    Writing objects: 100% (31/31), 2.97 MiB, done.
    Total 31 (delta 0), reused 0 (delta 0)
    ^C

Had to use Ctrl-C to get out. What's happened is that the push buffer isn't big enough. The solution is a change in git configuration, an arbitrary new size for the HTTP POST buffer.

    master ~/bitbucket/zMongoTryStuff $ git config http.postBuffer 524288000
    master ~/bitbucket/zMongoTryStuff $ git push -u origin master
    Password:
    Counting objects: 31, done.
    Delta compression using up to 4 threads.
    Compressing objects: 100% (27/27), done.
    Writing objects: 100% (31/31), 2.97 MiB, done.
    Total 31 (delta 0), reused 0 (delta 0)
    remote: bb/acl: russellbateman is allowed. accepted payload.
    To https://[email protected]/russellbateman/zMongoTryStuff.git
     * [new branch]      master -> master
    Branch master set up to track remote branch master from origin.

Bitbucket

Just a recap in case it's not obvious. Assuming I'm russellbateman, but have been granted rights to my friend's projects, I clone one of the latter using the second command below.

~/dev/bitbucket $ git clone [email protected]:russellbateman/zMongoTryStuff
~/dev/bitbucket $ git clone [email protected]:jsevison/java-learn

Jenkins and Bitbucket

Check here for hooking the two up:

In the end, it was much easier than I thought although I'm doing this very privately and so didn't care about others being able to reach Jenkins for my project. I used a Jenkins I had installed privately that's not visible on the Internet.

It was not the nightmare that GitHub was in that Bitbucket conveniently allows me to tell Jenkins a URL of

    https://russellbateman:<password>@bitbucket.org/russellbateman/<repositor>.git

...for the Source Code Management -> Git -> Repositories -> Repository URL field.


GitHub pull-request paradigm

The whole "Pull-request" paradigm was new to me as I'd always used the "Shared-repository" model. Here I'm following GitHub's Fork a Repository instructions in order to fork a repository (to contribute to a project). Despite the comment, you do not have to be new to Git and GitHub to need to read this page. I've used Git and GitHub for years, but always used a different model (approach) than what I'm learning to do here.

What's going on here: there's an existing repository to which I need to make a contribution. I need to change a small thing in a file. I don't have ownership over that repository, but my product consumes what it creates. In order to do this, I have to make a fork of that repository, make my changes, then plead with the owners to accept them. First, I need to fork the repositior and make the changes. Here are the steps:

- Step 1: Fork the repository you wish to contribute to.

- Step 2: Clone your fork (and make it your current working directory):

$ git clone https://github.com/russellbateman/tomcat-default.git
$ cd tomcat-default

- Step 2bis: Configure your forked repository (if you like—not in article)

$ git config --global user.email "[email protected]"
$ git config --global user.name  "Russell Bateman"
$ git config color.ui true

- Step 3: Configure your remotes:

$ git remote add upstream https://github.com/fs-eng/tomcat-default.git
$ git fetch upstream

Note: If you're reading this, the upstream remote is probably something you've never created or done, etc. It constitutes a salient difference with your previous Git experience.

- Step 4: Make your change(s):

$ cd src/main/conf
$ gvim server.xml
$ git diff server.xml
# On branch master
# Changes not staged for commit:
#   (use "git add ..." to update what will be committed)
#   (use "git checkout -- ..." to discard changes in working directory)
#
#	modified:   server.xml
#
no changes added to commit (use "git add" and/or "git commit -a")

- Step 4bis: Commit and push your changes:

$ git commit -a server.xml
[master 7603346] Spoke to Stephen Kinser; Awaken Tomcat access logging and
set to 1-second granularity. (Future: We'll be going for millisecond
granularity in Tomcat 7.)
 1 file changed, 18 insertions(+), 3 deletions(-)
$ git push origin master

Now I'm following Using Pull Requests, in particular, the "Fork & Pull" model. I have to propose the change I made to tomcat-default's owners for incorporation into the official source base. To do this, I have to initiate a pull request, then send it to them.

Initiate and review the pull request

- Step 1: Switch to your (forked) branch. I clicked "tomcat-default", noted by the arrow below:


- Step 2: Click the Compare & review button.

...to get the diff and be ready to create the request.

As explained in Changing the Branch Range and Destination Repository, your pull request will be based on the parent repository's default branch (master). Here, russellbateman/tomcat-default was forked from fs-eng/tomcat-default.


Send the pull request

- Step 1: Click on the Create Pull Request button. You'll get a chance to entitle the pull request and add a comment:


- Step 2: Click the Send pull request button to see the result; you'll await some response from the owning team:

Cancelling a pull request

You can't cancel a pull request, but you can close it. To do this, you go to the discussion page associated with the request.

To get to the page to do that, follow these steps:

  1. Get to your GitHub page.
  2. Click on the relevant repository near the bottom of the rightmost column.
  3. Click the green Compare, review, create a pull request button at the upper left of the page content region.
  4. Click the green View Pull Request button.
  5. Near the bottom, click the Close button next to the green Comment button.


GitHub simple pull-request

—directly in the browser. These steps to modify a version of a JAR in a pom.xml file:

  1. Go to github.com and locate the repostiory.
  2. Locate the file to change.
  3. Click Edit and make the change.
  4. Below Propose file change, edit the change message.
  5. Click big, green Propose file change button. This makes a fork named patch-1.
  6. If there's nothing more to say (nothing to add to the existing message that's needed as the message to the pull request), then just click the big, green Create pull request button.
  7. Done! You should see something like this:


How to stop getting notification for a repository
  1. Go to the repository top page.
  2. Click the Unwatch button.
  3. Choose either Not watching (you only get notified if you're actually involved somehow) or Ignoring (you won't get any notifications at all).

Agent admitted failure...key...Permission denied (publickey)
"Cloning into ...
Agent admitted failure to sing using the key.
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

This happened even after I recreated the public key, etc. What I neglected to do, and this something I never had to do before around 2012, is add the key:

~/.ssh $ ssh-add
Identity added: /home/russ/.ssh/id_rsa (/home/russ/.ssh/id_rsa)

How to remove or change remote origin from repository

You've been working on a project committed to an existing repository, then want to rename that project and the repository it will be held in. You create the new repository (in Bitbucket, GitHub, etc.). You rename the subdirectory. You want to shift over to the new respository.

If it were a new repository and new in your project (what would be the case if you did $ rm .git ; git init), you would do this:

master ~/dev/cda-filter $ git remote add origin [email protected]:russellbateman/cda-filter.git
fatal: remote origin already exists.

but, as noted, you already have a remote (recorded down inside .git/config). So, instead, you change the URL to the new repository you've created:

master ~/dev/cda-filter $ git remote set-url origin [email protected]:russellbateman/cda-filter.git

And, yes, this retains all the original history from when the repository was otherwise named!