How is the Git hash calculated? How is the Git hash calculated? git git

How is the Git hash calculated?


As described in "How is git commit sha1 formed ", the formula is:

(printf "<type> %s\0" $(git cat-file <type> <ref> | wc -c); git cat-file <type> <ref>)|sha1sum

In the case of the commit 9eabf5b536662000f79978c4d1b6e4eff5c8d785 (which is v2.4.2^{}, and which referenced a tree) :

(printf "commit %s\0" $(git cat-file commit 9eabf5b536662000f79978c4d1b6e4eff5c8d785 | wc -c); git cat-file commit 9eabf5b536662000f79978c4d1b6e4eff5c8d785 )|sha1sum

That will give 9eabf5b536662000f79978c4d1b6e4eff5c8d785.

As would:

(printf "commit %s\0" $(git cat-file commit v2.4.2{} | wc -c); git cat-file commit v2.4.2{})|sha1sum

(still 9eabf5b536662000f79978c4d1b6e4eff5c8d785)

Similarly, computing the SHA1 of the tag v2.4.2 would be:

(printf "tag %s\0" $(git cat-file tag v2.4.2 | wc -c); git cat-file tag v2.4.2)|sha1sum

That would give 29932f3915935d773dc8d52c292cadd81c81071d.


There's bit of confusion here. Git uses different types of objects: blobs, trees and commits.The following command:

git cat-file -t <hash>

Tells you the type of object for a given hash.So in your example, the hash 9eabf5b536662000f79978c4d1b6e4eff5c8d785 corresponds to a commit object.

Now, as you figured out yourself, running this:

git cat-file -p 9eabf5b536662000f79978c4d1b6e4eff5c8d785

Gives you the content of the object according to its type (in this instance, a commit).

But, this:

git hash-object fi

...computes the hash for a blob whose content is the output of the previous command (in your example), but it could be anything else (like "hello world!"). Here try this:

echo "blob 277\0$(cat fi)" | shasum

The output is the same as the previous command. This is basically how Git hashes a blob. So by hashing fi, you are generating a blob object. But as we have seen, 9eabf5b536662000f79978c4d1b6e4eff5c8d785 is a commit, not a blob. So, you cannot hash fi as it is in order to get the same hash.

A commit's hash is based on several other informations which makes it unique (such as the committer, the author, the date, etc). The following article tells you exactly what a commit hash is made of:

The anatomy of a git commit

So you could get the same hash by providing all the data specified in the article with the exact same values as those used in the original commit.

This might be helpful as well:

Git from the bottom up


Reimplementing the commit hash without Git

To get a deeper understanding of this aspect of Git, I reimplemented the steps that produce a Git commit hash in Rust, without using Git. It currently works for getting the hash when committing a single file. The answers here were helpful in achieving this, thanks.

These are the individual pieces of data we need to compute to arrive at a Git commit hash:

  1. The object ID of the file, which involves hashing the file contents with SHA-1. In Git, hash-object provides this ID.
  2. The object entries that go into the tree object. In Git, you can get an idea of those entries with ls-tree, but their format in the tree object is slightly different: [mode] [file name]\0[object ID]
  3. The hash of the tree object which has the form: tree [size of object entries]\0[object entries]. In Git, get the tree hash with: git cat-file commit HEAD | head -n1
  4. The commit hash by hashing the data you see with cat-file. This includes the tree object hash and commit information like author, time, commit message, and the parent commit hash if it's not the first commit.

Each step depends on the previous one. Let's start with the first.

Get the object ID of the file

The first step is to reimplement Git's hash-object, as in git hash-object your_file.

We create the object hash from our file by concatenating and hashing these data:

  • The string "blob " at the beginning, followed by
  • the size of the file, followed by
  • a null byte, expressed with \0 in printf and Rust, followed by
  • the file content.

In Bash:

file_name="your_file";printf "blob $(wc -c < "$file_name")\0$(cat "$file_name")" | sha1sum

In Rust:

// Get the object IDfn git_hash_object(file_content: &[u8]) -> Vec<u8> {    let file_size = file_content.len().to_string();    let hash_input: Vec<u8> = vec![        "blob ".as_bytes(),        file_size.as_bytes(),        b"\0",        file_content,    ]    // Flatten the Vec<&[u8]> to Vec<u8> with concat    .concat();    to_sha1(&hash_input)}

I'm using crate sha1 version 0.6.0 in to_sha1:

fn to_sha1(hash_me: &[u8]) -> Vec<u8> {    let mut m = Sha1::new();    m.update(hash_me);    m.digest().bytes().to_vec()}

Get the object entry of the file

Object entries are part of Git's tree object. Tree objects represent files and directories.

Object entries for files have this form: [mode] [file name]\0[object ID]

We assume the file is a regular, non-executable file, which translates to mode 100644 in Git. See this for more on modes.

This Rust function takes the result of the previous function git_hash_object as the parameter object_id:

fn object_entry(file_name: &FileName, object_id: &[u8]) -> Vec<u8> {    // It's a regular, non-executable file    let mode = "100644";    // [mode] [file name]\0[object ID]    let object_entry: Vec<u8> = vec![        mode.as_bytes(),        b" ",        file_name.as_bytes(),        b"\0",        object_id,    ]    .concat();    object_entry}

FileName is a new type for String to avoid mixing up arguments, nothing fancy:

pub struct FileName(pub String);impl std::fmt::Display for FileName {    fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {        self.0.fmt(f)    }}impl FileName {    fn as_bytes(&self) -> &[u8] {        self.0.as_bytes()    }}

I tried to write the equivalent of object_entry in Bash, but Bash variables cannot contain null bytes. There are probably ways around that limitation, but I decided for now that if I can't have variables in Bash, the code would get quite difficult to understand. Edits providing a readable Bash equivalent are welcome.

Get the tree object hash

As mentioned above, tree objects represent files and directories in Git. You can see the hash of your tree object by running, for example, git cat-file commit HEAD | head -n1.

The tree object has this form: tree [size of object entries]\0[object entries]

In our case we only have a single object_entry, calculated in the previous step:

fn tree_object_hash(object_entry: &[u8]) -> String {    let object_entry_size = object_entry.len().to_string();    let tree_object: Vec<u8> = vec![        "tree ".as_bytes(),        object_entry_size.as_bytes(),        b"\0",        object_entry,    ]    .concat();    to_hex_str(&to_sha1(&tree_object))}

Where to_hex_str is defined as:

// Converts bytes to their hexadecimal representation.fn to_hex_str(bytes: &[u8]) -> String {    bytes.iter().map(|x| format!("{:02x}", x)).collect()}

In a Git repo, you can look at the contents of the tree object with ls-tree. For example, running git ls-tree HEAD will produce lines like these:

100644 blob b8c0d74ef5ccd3dab583add7b3f5367efe4bf823    your_file

While those lines contain the data of an object entry (the mode, the string "blob", the object ID, and the file name), they are in a different order and include a tab character. Object entries have this form: [mode] [file name]\0[object ID]

Get the commit hash

The last step creates the commit hash.

The data we hash using SHA-1 includes:

  • Tree object hash from the previous step.
  • Hash of the parent commit if the commit is not the very first one in the repo.
  • Author name and authoring date.
  • Committer name and committing date.
  • Commit message.

You can see all of that data with git cat-file commit HEAD, for example:

tree a76b2df314b47956268b0c39c88a3b2365fb87ebparent 9881a96ab93a3493c4f5002f17b4a1ba3308b58bauthor Matthias Braun <m.braun@example.com> 1625338354 +0200committer Matthias Braun <m.braun@example.com> 1625338354 +0200Second commit (that's the commit message)

You might have guessed that 1625338354 is a timestamp, the number of seconds since the Unix epoch. You can convert from the date and time format of git log, such as "Wed Jun 23 18:02:18 2021", with date:

date --date='Wed Jun 23 18:02:18 2021' +"%s"

The timezone is denoted as +0200 in this example.

Based on the output of cat-file, you can create the Git commit hash using this Bash command (which uses git cat-file, so it's no reimplementation):

cat_file_output=$(git cat-file commit HEAD);printf "commit $(wc -c <<< "$cat_file_output")\0$cat_file_output\n" | sha1sum

The Bash command illustrates that—similar to the steps before—what we hash is:

  • A leading string, "commit " in this step, followed by
  • the size of a bunch of data. Here it's the output of cat-file which is detailed above. Followed by
  • a null byte, followed by
  • the data itself (output of cat-file) with a line break at the end.

In case you kept score: Creating a Git commit hash involves using SHA-1 at least three times.

Below is the Rust function for creating the Git commit hash. It uses the tree_object_hash produced in the previous step and a struct CommitMetaData which contains the rest of the data you see when calling git cat-file commit HEAD. The function also takes care of whether the commit has a parent commit or not.

fn commit_hash(commit: &CommitMetaData, tree_object_hash: &str) -> Vec<u8> {    let author_line =        commit.author_name_and_email.to_owned() + " " +        commit.author_timestamp_and_timezone;    let committer_line =        commit.committer_name_and_email.to_owned() + " " +        commit.committer_timestamp_and_timezone;    // If it's the first commit, which has no parent,    // the line starting with "parent" is omitted    let parent_commit_line = match &commit.parent_commit_hash {        Some(parent_commit_hash) => "\nparent ".to_owned() + parent_commit_hash,        None => "".to_string(),    };    let git_cat_file_str = format!(        "tree {}{}\nauthor {}\ncommitter {}\n\n{}\n",        tree_object_hash,        parent_commit_line,        author_line,        committer_line,        commit.commit_message    );    let git_cat_file_len = git_cat_file_str.len().to_string();    let commit_object: Vec<u8> = vec![        "commit ".as_bytes(),        git_cat_file_len.as_bytes(),        b"\0",        git_cat_file_str.as_bytes(),    ].concat();    // Return the Git commit hash    to_sha1(&commit_object)}

Here's CommitMetaData:

#[derive(Debug, Copy, Clone)]pub struct CommitMetaData<'a> {    pub(crate) author_name_and_email: &'a str,    pub(crate) author_timestamp_and_timezone: &'a str,    pub(crate) committer_name_and_email: &'a str,    pub(crate) committer_timestamp_and_timezone: &'a str,    pub(crate) commit_message: &'a str,    // All commits after the first one have a parent commit    pub(crate) parent_commit_hash: Option<&'a str>,}

This function creates CommitMetaData where author and committer info are identical, which will be convenient when we run the program later:

pub fn simple_commit<'a>(    author_name_and_email: &'a str,    author_timestamp_and_timezone: &'a str,    commit_message: &'a str,    parent_commit_hash: Option<&'a str>,) -> CommitMetaData<'a> {    CommitMetaData {        author_name_and_email,        author_timestamp_and_timezone,        committer_name_and_email: author_name_and_email,        committer_timestamp_and_timezone: author_timestamp_and_timezone,        commit_message,        parent_commit_hash,    }}

Putting it all together

As a summary and reminder, creating a Git commit hash consists of getting:

  1. The object ID of the file, which involves hashing the file contents with SHA-1. In Git, hash-object provides this ID.
  2. The object entries that go into the tree object. In Git, you can get an idea of those entries with ls-tree, but their format in the tree object is slightly different: [mode] [file name]\0[object ID]
  3. The hash of the tree object which has the form: tree [size of object entries]\0[object entries]. In Git, get the tree hash with: git cat-file commit HEAD | head -n1
  4. The commit hash by hashing the data you see with cat-file. This includes the tree object hash and commit information like author, time, commit message, and the parent commit hash if it's not the first commit.

In Rust:

pub fn get_commit_hash(    file_name: &FileName,    file_content: &[u8],    commit: &CommitMetaData,) -> String {    let file_object_id = git_hash_object(file_content);    let object_entry = object_entry(file_name, &file_object_id);    let tree_object_hash = tree_object_hash(&object_entry);    let commit_hash = commit_hash(commit, &tree_object_hash);    to_hex_str(&commit_hash)}

With the functions above, you can create a file's Git commit hash in Rust, without Git:

use std::fs::File;use std::io;use std::io::prelude::*;fn main() -> io::Result<()> {    let file_name = FileName("your_file".to_string());    let file_content = read_all_bytes(&file_name)?;    let first_commit = simple_commit(        "Firstname Lastname <test@example.com>",        // Timestamp calculated using: date --date='Wed Jun 23 18:02:18 2021' +"%s"        "1625338354 +0200",        "Message of first commit",        // No parent commit hash since this is the first commit        None,    );    let first_commit_hash = get_commit_hash(&file_name, &file_content, &first_commit);    Ok(println!("Git commit hash: {} ", first_commit_hash))}fn read_all_bytes(file_name: &FileName) -> io::Result<Vec<u8>> {    let mut f = File::open(file_name.to_string())?;    let mut file_content = Vec::new();    f.read_to_end(&mut file_content)?;    Ok(file_content)}

To create the hash of the second commit, you take the hash of the first commit and put it into the CommitMetaData of the second commit:

let second_commit = simple_commit(    "Firstname Lastname <test@example.com>",    "1625388354 +0200",    "Message of second commit",    // The first commit is the parent of the second commit    Some(first_commit_hash),);

Apart from the other answers here and their links, these were some useful resources in creating my limited reimplementation:

  • Reimplementation of git hash-object in JavaScript.
  • Format of a Git tree object, this is the next place I'd look if I wanted to make my reimplementation more complete: To work with commits involving more than one file.