Tuesday 26 June 2012

HOWTO: Simple Git Repository

This is by no means a comprehensive guide to the Git version control system but is rather a few of the basic operations that I use to maintain a few small projects and scripts.

It is also worth understanding that unlike svn, git is a peer to peer based version control system where every "client" is a repository in its own right which can also push and pull from a "master" repository as required.

The first thing to do of course is to install git on your server. In my case this will be a PC named "jupiter". You will also require ssh server so we will also install that;

For Debian/Ubuntu/Mint;

$ sudo apt-get install git openssh-server

or for Redhat/CentOS/Mandriva/Suse

$ sudo yum install git openssh-server

I will be assuming that the user account that you logged in as will be the account that has access to the repository. Fine grained user control is not covered in this document.

Before we begin building a repository, we should configure git for our user account (change the following commands to suit own your user details).

$ git config --global user.name "brett"
git config --global user.email "brett_AT_tuxnetworks.com"
git config --global core.autocrlf input
git config --global core.safecrlf true


This will create a ".gitconfig" file in your home directory. Edit this file;

$ vi ~/.gitconfig

Add some aliases to the end of the file;

[alias]
  co = checkout
  ci = commit
  st = status
  br = branch
  hist = log --pretty=format:\"%h %ad | %s%d [%an]\" --graph --date=short
  type = cat-file -t
  dump = cat-file -p


Let's create a test project directory for our base repository and change to it;

$ mkdir -p ~/repository/project

# cd ~/repository/project


OK, now we will create a "bare" repository. You can think of this as the master "shared" repository.

$ git --bare init
Initialized empty Git repository in /home/brett/repository/project/.git/


Do a directory listing and you will see this;

$ ls -al
total 40
drwxrwxr-x 7 brett brett 4096 Jun 26 11:24 .
drwxrwxr-x 3 brett brett 4096 Jun 26 11:24 ..
drwxrwxr-x 2 brett brett 4096 Jun 26 11:24 branches
-rw-rw-r-- 1 brett brett   66 Jun 26 11:24 config
-rw-rw-r-- 1 brett brett   73 Jun 26 11:24 description
-rw-rw-r-- 1 brett brett   23 Jun 26 11:24 HEAD
drwxrwxr-x 2 brett brett 4096 Jun 26 11:24 hooks
drwxrwxr-x 2 brett brett 4096 Jun 26 11:24 info
drwxrwxr-x 4 brett brett 4096 Jun 26 11:24 objects
drwxrwxr-x 4 brett brett 4096 Jun 26 11:24 refs


Surprisingly, that is all that is required on the master repository.

Now, we move to the client which will do the initial code push to the repository.

Note: This can be the same host as the server or a different one altogether. In this case I will log on to a different host which is configured with the same user account details as well as ssh key authorization. 

To save configuring git again on this host, you can simply copy the git config file over from my server "jupiter";

scp jupiter:.gitconfig ~

Create a folder for your project in your home directory;

$ mkdir ~/project

$ cd ~/project


Initialize a local repository, this time without the "--bare" option

$ git init
Initialized empty Git repository in /home/brett/project/.git/


This time if we take a look we will see the directory is totally different than before (on the master);

t$ ls -al
total 12
drwxrwxr-x  3 brett brett 4096 Jun 26 11:23 .
drwxr-xr-x 16 brett brett 4096 Jun 26 11:24 ..
drwxrwxr-x  7 brett brett 4096 Jun 26 11:27 .git


Now we tell our local repository to track the master repository on jupiter;

$ git remote add --track master repo brett@jupiter:repository/project

Note: This command tells git that the branch is the master, and assigns the repository the name "repo". Other guides and examples you come across will generally use the name "origin" in place of "repo" by convention. I use "repo" but you can also have multiple remotes and give them all different names like "staging", "testing", "production" etc.

Of course we will need to have a file to be version managed by git. Create one now;

$ echo "This is my source file" > source.file

Tell git to manage the source file;

git add source.file

The "status" command can be used to see what state our local repository is in;

$ git status
# On branch master
#
# Initial commit
#
# Changes to be committed:
#   (use "git rm --cached ..." to unstage)
#
#    new file:   source.file

We can see that source.file is a new, uncommited file.

Let's commit it to the local repository now;

$ git commit -m "This is my first commit"
[master (root-commit) 4714b45] This is my first commit
 1 file changed, 1 insertion(+)
 create mode 100644 source.file


At the moment our file is only commited to the local repository. We need to push it to the master;
$ git push --tags repo master
Counting objects: 3, done.
Writing objects: 100% (3/3), 250 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To jupiter:repository/project
 * [new branch]      master -> master

Note: For subsequent pushes and pulls, you may omit the "master" parameter as we configured the repository to track the master branch using "--track master" when we set it up. It is required for the first push though I'm not sure why.

We can take a look at the history using the "hist" alias (we configured that in .gitconfig earlier)

$ git hist
* 4714b45 2012-06-26 | This is my first commit (HEAD, repo/master, master) [brett]


Cool. If we want to "pull" the source out of the repository then use the "pull" command insted of "push"

$ git pull repo

So there you have it, a rudimentary git setup.

Here are a few other commands that are useful;

Cancel an uncommited change;
$ git reset HEAD
$ git checkout


Add a tag;
$ git tag "v1.0"

Push tags;
$ git push --tags repo
Total 0 (delta 0), reused 0 (delta 0)
To jupiter:repository/project
 * [new tag]         v1.0 -> v1.0


Remove a tag;
$ git tag -d "v1.0"
Deleted tag 'v1.0' (was 4714b45)


Push deleted tags;
$ git push repo :refs/tags/"v1.0"
To jupiter:repository/project
 - [deleted]         v1.0





Thursday 21 June 2012

HOWTO: Subversion 1.7.x on Centos 6

Howto: Subversion 1.7.x on Centos 6

Prerequisites:
* Minimal CentOS 6 installation
* SELinux & Firewall disabled

First, I'm going to start off with a bit of a mini-rant.

As usual, installing stuff on RHEL/CentOS is much harder than it is on Debian based systems. For one, the repositories are far more limited and what packages there are  are hopelessly outdated. Of course I understand that the RHEL philosophy is to freeze packages for a particular major version (6 in this case) and only provide security and bug fixes to these packages because this makes sense when running servers in a business environment, which is, of course, their target market.

This is a good thing.

However sometimes you want a newer version of something for whatever reason. Debian manages this by having a backports repository which can be optionally enabled to allow easy access to newer packages from the Debian testing branch. From what I can tell RHEL/CentOS do not have an equivalent option. Of course there are third party repositories that provide access to newer packages (to a degree), but coverage is sporadic at best.

In this case we will be required to resort to manually downloading third party RPM packages from WANDisco because they are not provided via any repository I could find. Apparently you are meant to fill out some sort of webform where you have to provide them with your personal details and "request" the packages along with an installer script that will allegedly install all the dependancies.

Feel free to go and fill out that form, however, I wasn't willing to go that route, it evoked too many memories of when I used to be a Windows user, where everything comes with strings attached.

The good news is that the RPMs are available without having to request them by going directly to here. The bad news is that these packages are for RHEL/CentOS 5 and I have been unable to find the equivalent packages for CentOS 6. This crucial difference will cause us to briefly flirt with the dreaded dependancy hell later on.

So, the first step is to obtain the following packages (64 bit links to my site below);

subversion-1.7.5-1.x86_64.rpm
mod_dav_svn-1.7.5-1.x86_64.rpm
neon-0.25.5-10.el5_4.1.x86_64.rpm

Note: If you don't want to trust my linked packages (and why should you?) or you require 32 bit versions then the first two are available from the WANDisco website mentioned above, the third I found at rpm.pbone.net.

OK, with the three files in hand, we probably should ensure our system is up to date before we proceed.

yum update

Subversion requires Apache web server, let's install it now;

yum install httpd

You probably want Apache to start automatically after a reboot;

chkconfig httpd on

We should also start it now;

service httpd start

These dependencies are required for SVN 1.7.x and thankfully they are all available in the standard CentOS repository;

yum install openssl098e compat-db43 compat-expat1 compat-openldap

Tip: In cases like this you can use something like "yum whatprovides ./libldap-2.3.so.0" to find where your dependencies live

Here's where we hit a minor snag. If we try and install the subversion rpm at this point it will complain that it depends on neon-0.25 but the CentOS6 repository provides v0.29.

Another rant: This is another bugbear I have with the yum/rpm system. It seems to be much more finicky than Debian and backwards compatibility is often non-existent. In this case the WANDisco RPM has been built to explicitly require neon 0.25 even though I am pretty sure that v 0.29 is fully backwards compatible and would work. It's a stupid situation and one that I honestly can't remember finding in Debian/Ubuntu over nearly 10 years of using that distro. Maybe that is because you are not forced to rely on dodgy third party compiled packages on a regular basis I suppose.

Anyway, luckily for us neon does not have any onerous dependency requirements of it's own so we can go ahead and install the older version manually without falling into dependency hell, which is a bit of luck.

rpm -i neon-0.25.5-10.el5_4.1.x86_64.rpm

With neon v0.25 installed, we can go ahead and install subversion and mod_dav_svn;

rpm -i subversion-1.7.5-1.x86_64.rpm
rpm -i mod_dav_svn-1.7.5-1.x86_64.rpm


Note: If you see something like this;

warning: mod_dav_svn-1.7.5-1.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 3bbf077a: NOKEY

you can ignore it, unless you want to obtain and configure the appropriate validation keys for these files which is outside the scope of this document.

We can confirm that subversion is now installed;

# rpm -qa | grep subversion
subversion-1.7.5-1.x86_64


OK, all is good, right? Well, yes and no. Everything is OK right now but if you try and do a yum update, it will fail like this;

--> Finished Dependency Resolution
Error: Package: subversion-1.7.5-1.x86_64 (installed)
           Requires: libneon.so.25()(64bit)
           Removing: neon-0.25.5-10.el5_4.1.x86_64 (installed)
               libneon.so.25()(64bit)
           Updated By: neon-0.29.3-1.2.el6.x86_64 (base)
               Not found
You could try using --skip-broken to work around the problem
You could try running: rpm -Va --nofiles --nodigest


Aaargh!

To workaround this, we can exclude neon from being updated;

vi /etc/yum.conf

Add this line somewhere in the file;

exclude=neon*

Now our yum update won't try and upgrade neon and therefore complain about dependency problems;

# yum update
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: ftp.swin.edu.au
 * extras: ftp.swin.edu.au
 * updates: ftp.swin.edu.au
base                                                                                                 
extras    updates                                                                                              Setting up Update Process
No Packages marked for Update


And that's it, go grab yourself a beverage for a job well done!

Tuesday 5 June 2012

Monday 4 June 2012

Passwordless SSH login fails

If you are attempting to log in to an openssh server using public key authorisation and it keeps asking for your password anyway then check the permissions on the ssh directory for the user account you are trying to log in as;

ls -al ~/.ssh
drwx------ 2 brett brett 4096 Jun  4 13:40 .
drwx------ 6 brett brett 4096 Jun  4 13:37 ..
-rwx------ 1 brett brett  398 Jun  4 13:40 authorized_keys



If the permissions are anything other than those shown above then you need to fix that;

chmod 700 ~/.ssh
chmod 700 ~/.ssh/*




Friday 1 June 2012

Steam is coming to Linux

Wow! Native Steam is coming to Linux within months.

http://www.escapistmagazine.com/news/view/116943-Steam-Coming-to-Linux-Soon

Git Aliases "cannot exec"

I'm playing around with git and following the tutorial here.

I ran into a problem when attempting to use git aliases.

[git@git hello]$ git hist
fatal: cannot exec 'git-hist': Permission denied


Turns out the issue was because I had logged in as user "git" via the su command but had neglected to use the - (minus)

When I logged in directly as user git or from root using 'su -l git' everything worked as advertised.

I got the clue to the problem from running strace.

[git@git hello]$ strace -f -e execve git hist
execve("/usr/bin/git", ["git", "hist"], [/* 21 vars */]) = 0
Process 1560 attached
[pid  1560] execve("/usr/libexec/git-core/git-hist", ["git-hist"], [/* 21 vars */]) = -1 ENOENT (No such file or directory)
[pid  1560] execve("/usr/local/sbin/git-hist", ["git-hist"], [/* 21 vars */]) = -1 ENOENT (No such file or directory)
[pid  1560] execve("/usr/local/bin/git-hist", ["git-hist"], [/* 21 vars */]) = -1 ENOENT (No such file or directory)
[pid  1560] execve("/sbin/git-hist", ["git-hist"], [/* 21 vars */]) = -1 ENOENT (No such file or directory)
[pid  1560] execve("/bin/git-hist", ["git-hist"], [/* 21 vars */]) = -1 ENOENT (No such file or directory)
[pid  1560] execve("/usr/sbin/git-hist", ["git-hist"], [/* 21 vars */]) = -1 ENOENT (No such file or directory)
[pid  1560] execve("/usr/bin/git-hist", ["git-hist"], [/* 21 vars */]) = -1 ENOENT (No such file or directory)
[pid  1560] execve("/root/bin/git-hist", ["git-hist"], [/* 21 vars */]) = -1 EACCES (Permission denied)
fatal: cannot exec 'git-hist': Permission denied


The fact that it was trying to execute the command in /root/bin (wtf is that? Some sort of hardcoded path? Further investigation is required) indicated there was some sort of user identity problem.