Security best practices for git users
In recent years git has become one of most popular SCM/Version Control systems. Usage in some high-profile open-source projects like Linux or Raspberry Pi and support from vendors like GitHub and GitLab definitively helped to gain fame. As particular technology gets widely used, it becomes a high profile attack target. SCM software and services are one of the best high-profile targets as unauthorized change within few lines of code may result in half of the world with the same vulnerability or backdoor installed. This article provides minimal git security best practices. Practices presented below can easily be integrated into your Secure Development Lifecycle process if you deployed one already.
Identity
Proper user identification and authorization is one of most fundamental security requirements. Git tries its best to detect user identity when committing changes. The problem is that this process is based on data gather from an operating system based on user name and account configuration. In reality, anyone can create a user account with data imitating legal user account. This brings us to the first best practices:
Two year's worth of NIST-aligned training
Deliver a comprehensive security awareness program using this series' 1- or 2-year program plans.
Practice #1: never use shared, or system accounts like root, nobody, www, etc. for committing changes
Practice #2: never use privileged account to develop code and commit (you can use system facilities like sudo, does or runs depending on operating system you are using)
There are some cases when git can't "detect" your identity or guess it wrongly. For such case (or for malicious user) git allow to enforce identity using following commands for example:
Git config –global user.name "John Doe."
Git config –global user.email j.doe@nosuchsite.com
The same result can be achieved by editing git configuration files which are plain text. Git also allows defining an identity locally for a particular repository using –local option. This feature brings another question: should you allow users to use different identities per project. The quick answer will be no, but if you think about it for a while it may turn out that answer will depend on project type and requirements. Hence, I do not define it as a best practice but rather a point to be addressed in you security policy or Secure Development Lifecycle process.
Practice #3: never assume identity-based on username / email. Enforce signed commits.
Signing commits and tags
Fortunately, for some time, git supports the cryptographic signing of commits and tags. Following Practice #3 you should use both options.
Before any user can sign commit or tag, he needs to have his pair of PGP private and public keys. You can use gpg to generate those using:
gpg –gen-key
Practice #4: Do not use short keys for PGP/GPG.
Practice #5: Do not trust a PGP/GPG key by default – be sure you've got a trusted public key from a right person.
Practice #6: Protect you private key. Report if it gets compromised and revoke the key immediately from key servers. Right after generating your private key, generate revocation certificate (gpg –gen-revoke option) and store it in a safe place. In case of compromise, you will be able to use it to inform others that your compromised key isn't valid any longer.
Practice #7: Do not generate PGP/GPG keys with an infinite validity period. Such keys are harder or impossible to "revoke" in case of compromise, especially when no revocation certificate is available, or you've lost access to your private key.
Practice #8: Assign strong password to protect private keys.
After your key gets created you have to tell about it git using following commands:
Git config --global user.signingkey [key id]
Now you can sign every commit and tag. To sign a commit you can use –S switch like below:
git commit –S –m "Signed commit message"
To sign a tag enter:
git –s [tag name] "Signed tag name"
Verifying commits and tags is also very straightforward. Verify tag using:
git tag –v [tag name]
In case of commits you need to use:
git log --show-signature
You can also use signing mechanism – starting from git 1.8.3 version – during marge and pull operations. In such case commits without trusted and valid PGP/GPG signature will be rejected. Here are proper examples:
git merge --verify-signatures
You should use –S switch when using git "merge" command to sign resulting merge commit.
Practice #9: All committers must sign – otherwise this mechanism makes no sense.
Practice #9 is great for newly created projects but how it can be implemented in case of already existing repositories with millions of unsigned commits and tags? The answer is simple: introduce signing as a requirement and create a signed tag marking a point from which all commits must be signed. Review unsigned commits to assure there is no unauthorized one.
Single secure repository
Git is distributed revision control system which means there can be an unlimited number of repositories of the project at any particular time. While it is great from a developer's perspective, it doesn't sound so good from a security point.
Practice #10: always have at least one copy of repository that you can trust and treat as secure.
Securing access
There are number of different access paths you need to be concerned from security standpoint:
- Secure access to developer systems: if it is remote, provide both strong authentication (two-factor authentication may be a very good idea) and cryptography for connection (properly configured SSHv2 server like OpenSSH with key authentication is one example)
- Secure access to git server protocols including limited access to those (not always applicable, especially if you run community services like GitHub, or you are using one of many other cloud services)
- Secure and limit access to you Continuous Integration (CI) systems including runners. Deploy code using secure protocols offering transmission encryption and strong authentication like SSH (for example Fabric is using SSH for remote code deployment)
- Not every project can be / must be public – assign proper access rights to it, this is especially true when using services like GitHub or GitLab for example (Fig 1).
Figure 1: Project repository-global access rights (Visibility Level) configuration when creating a new project in GitLab. Don't forget to define proper access rights per user.
Protected branches
Protected branches (see Fig. 2) is a functionality not available directly from git but rather git based repository management solutions including web ones.
The idea behind a protected branch is a fact that git makes it fairly easy to enforce conflicting commits. Anyone who can complete:
git push –force
The operation can destroy whole repository content. Please note that from git perspective this is perfectly valid and legal operation.
Figure 2: List of protected branches in GitLab
Since git doesn't provide branch protection, the details of its implementation may vary depending on the 3rd party software enabling such functionality (even name of it can vary). It should protect from:
- Deleting branch
- Forced pushing to the branch
- Preventing pushes or mergers from unprivileged users
For availability and further information regarding functionality, please consult the documentation for your git repository management software. Both GitHub and GitLab support branch protection.
Practice #11 Use SSH key authentication where possible and where applicable.
Practice #12 Use only SSHv2 protocol and explicitly disable SSHv1 protocol support.
Practice #13 Use https or SSH only to access git repositories.
Practice #14 Configure access rights to your projects.
Practice #15 Review access rights to your projects periodically.
Practice #16 Isolate test, preproduction and production environments from each other.
Practice #16 Isolate CI and runners from other environments.
Practice #17 Deploy code remotely in a safe manner.
When talking about securing access, services like GitLab or GitHub use https connection by default to access their sites. However, this may not be true when you are deploying such service internally (so-called Enterprise versions). Therefore, reconfigure those applications to use https instead of http – otherwise, there will be no protection for administrator passwords and CI runners tokens for example.
Practice #18 Enforce https access to git repository management software, CI systems and ticketing / bug tracking systems.
Git and integrity
Internally git is using an SHA-1 function to provide integrity checks both for packfiles (committed files are stored in compress format) and index. Furthermore packfile name contains an SHA-1 checksum. You can run integrity check on repository any time issuing:
git fsck
Git can also be forced to check integrity of all incoming objects using the following command:
git config --global transfer.fsckObjects true
As always you can also edit the .gitconfig file to achieve the same result. Besides, transfer.fsckObjects there are also two more options you can enable:
- receive.fsckObjects
- transfer.fsckObjects
Practice #19 Enforce integrity checks on all incoming objects by setting transfer.fsckObjects, fetch.fsckObjects and receive.fscObjects to true.
Git and information leak / malware prevention
Using .gitignore files you can tell git which files must be ignored during "add" command. This feature can be used both to prevent information leak and as a part of malware protection policy.
You wouldn't like to leak your test or even worst your production databases with customer data violating privacy and data protection acts, would you? The .gitignore functionality allows you disable tracking of such files by git and publishing those further.
If you add executable files like *.exe or *.dll or Linux ELF executable that are common vectors for infection by malware you are also limiting the possible chances of spreading an infection. However, this may not always be possible or applicable depending on your project objectives.
Practice #20 Enforce usage of .gitignore files by providing a proper .gitignore file content to all current and future projects. Add .gitignore to your repositories, so anyone cloning from it will get the same file.
Practice #21 Be careful when using 3rd party .gitignore files including those created automatically by services like GitHub when creating a repository. Review those first and assure they are correct before accepting those into your repositories.
Practice #22 Review periodically and after every technology change correctness of your .gitignore files.
Practice #23 Add intermediate, temporary and executable files to .gitignore that aren't needed for your project.
Also, take a note that when using Test Driven Development (TDD) both your unit and functional tests may be a perfect source of information leak. For example, when testing web apps, you may query the database or access protected areas of web app you are testing, using hardcoded credentials in your tests. Do not forget to review those sources and add them to .gitignore or simply modified those appropriately. Regarding databases, some frameworks, like Django for example, not only provides own API for testing purposes but also additionally supports this process by creating separate databases for testing period. Be sure to use such features by enabling them whenever possible.
Source code review
Practice #24 Perform manual source code review at least for every branch before doing a release. In reality, you may do it more often – some organizations are using tags as triggers for audits, but that depends on how you are using tags or other git functionality in your project.
Practice #25 Hook your git repository to automatic static source code analysis tools. You may consider using hooks mechanism to monitor every commit but also make sure that developers are using static analysis tools at their local repositories as well.
Change default passwords & keys
Some applications and services assign default passwords. A great example can be GitLab CE with default "root" username and "5iveL!fe" password. Not every system can be integrated with Active Directory or other directory services enforcing password policies, so you have to deal with it manually.
Practice #26 Change default passwords by assigning strong passwords and disable unneeded accounts if possible.
Practice #27 Regenerate default SSH keys by deleting old and generating new ones.
Practice #28 Change passwords according to you password management policy.
SSH key regeneration may require a few words of explanation. If you enable SSH access to git repositories and development hosts, the keys must be protected. However if you are using 3rd party virtual machine, virtual appliance or container keys can be multiplicated and are well known allowing for MitM attacks. Generating own pair of keys is therefore highly recommended best practice. The key (re)generation process depends on your system and SSH software installed. For example in Debian / Ubuntu systems with OpenSSH installed you can issue following commands:
sudo rm /etc/ssh/ssh_host_*
sudo dpkg-reconfigure openssh-server
If for some reason dpkg-reconfigure throws an error and fails to generate new keys, you can do it manually using ssh-keygen. For other systems, please consult the proper documentation. Keep in mind that some systems. Theoretically, regenerate keys during boot process but this may not happen due to some reasons. Therefore manually forcing key regeneration is the only acceptable solution unless you control the environment you are working with.
Keep critical parts of your git infrastructure secure
Last but not least let's not forget to apply security fixes and patches to you git repository. The minimal set of upgrades should include both ssh client and server and git (considered as the whole package). If you are using some git repository management software, it also must be included on your list. The list must also include all required for successful installation packages. Further the list should be extended by your CI management software, CI runners and their installation requirements.
If you are using additional client tools like GitHub Desktop, your list must also include those. In the case of GitHub Desktop it is easy to check what has been fixed by either clicking Settings -> About GitHub Desktop -> Release notes in GitHub Desktop window (see Fig. 3) or visiting its website [3].
Figure 3. GitHub Desktop release notes
If your IDE comes with git integration, be sure to include it on your list as well. The same rules apply to all kinds of plugins, extensions and services used across your development, build and test environments. The case of plugins and external services could be quite a challenge to manage from a security point of view as most IDEs, CI and web-based git managing software have such functionality.
Two examples of why you need to track vulnerabilities for your dev tools
Recent git vulnerabilities: CVE-2016-2315, CVE-2016-2324 clearly demonstrate that you have to patch your dev tools. Those vulnerabilities can be triggered during both clone and push operation by construing long name or long path inside the repository. This problem applies to git versions before 2.7.3 unless patches have been applied.
Another recent vulnerability CVE-2016-3115 in OpenSSH due to lack of sanitization of untrusted input allows an authenticated user who can request X11 forwarding to inject commands to south. While there is no need for X Windows usually on git servers, many Linux vendors enable X11 forwarding by default. To mitigate this vulnerability you need to disable X11 forwarding by setting X11Forwarding=no option in /etc./ssh/sshd_config file. All OpenSSH versions before 7.2p2 are vulnerable.
Please note that in the case of recent git vulnerability the time window between releasing publicly the information and availability of binary patches has been quite long. In such case, you may consider patching by building git from source. The process is well described in git documentation. The most basic workflow is presented below:
git clone https://github.com/git/git
cd ./git
make
make install
This will install git for the current user. To install it system-wide after cloning the repository enter following commands:
cd ./git
make prefix=/usr all doc info
sudo make prefix=/usr install install-doc install-html install-info
Summary
Demonstrated best practices are only a baseline – you should develop on them your policy to secure source code in your project(s). Please note that some areas aren't covered – one is your backup procedure and backup procedure testing for example. Due to its distributed nature git, repositories are easier to backup but nevertheless you should have a document and test the process.
Phishing simulations & training
We've also barely scratched the attack surface of CI and ticketing / bug tracking systems. The same applies to web base git repository management software. Securing only your repository infrastructure while forgetting about those components is nonsense, and it is a very bad practice. Unfortunately due to some different solutions, accessibility models, supported operating systems it is not possible to even barely cover that topics regarding most popular tools like GitLab, GitHub, Jenkins, Travis to name a few.
Sources
- "Pro Git," Scott Chacon and Ben Straub https://git-scm.com/book/en/v2
- "Software Configuration Management (SCM) Security," David A. Wheeler http://www.dwheeler.com/essays/scm-security.html
- GiHub Desktop release description https://desktop.github.com/release-notes/windows/