Open .git global scan

Hello dear visitor,

the reason you are visiting this site is you probably received my security notice. Just for clarification, I didn’t hack your site. I’m a security researcher/white hat/ethical hacker and I only detected a security problem on your website. No sensitive data was downloaded from your site except for your e-mail address which will be forgotten after the research. I won’t store it or use it for any other purposes.

Go to the mitigation

Go to the results

Go to related issue with misconfigureated webservers

 

The Scan

As I have said, I’m a security researcher and computer security is my hobby. I found your site while scanning the Internet for some particular vulnerabilities. So you are just one from many I notified – nothing personal here. I started with the scan of the Czech Internet and when I realized how big the problem is, I decided to repeat it on a global scale. These scans are non-commercial activities, but you can make a small donation for the future scans if you consider them helpful.

Details of the Vulnerability

The problem is you have a publicly available git repository on your website. You can check it by visiting /.git/HEAD. When you visit the directory /.git you usually get 403 error because there is no index.html/.php file and you don’t allow to show the directory listing/autoindex (if you can see the directory structure you have a misconfigured webserver – it is another type of vulnerability). Despite 403 it is possible to access the files directly. This is the way I found your e-mail address – from the /.git/logs/HEAD – it is the list of commits with details about commiteers. The structure of git repository is well known, so it is possible to found references to the objects/packs in the repository, download them via direct requests and reconstruct the repository and obtain your files – not only the current ones, but also the past files. There are tools which enable to automate the download.

It is a bad idea to store DB credentials and various API tokens in the repository, but many developers don’t follow the best practices and the vulnerability is really serious in this case.

It is not always possible to download the complete repo, but there are many other interesting information still, e.g. /.git/index – it is a binary file and it reveals the structure of your application, libs used, endpoints, inernal files etc. Sometimes you can find the address of unsecured WYSIWYG editors with the file uploader – unfortunatelly it is really common. You can see readable paths strings in the index file or you can use GIN to extract them to human readable format.

git-autoindex
accessible .git + autoindex enabled

Mitigation

It is a good practice to disallow the access to all files and folders with dot at the beginning, except for .well-known folder (it is used for certificates verification and for security.txt file).

For Apache and Nginx you can use the following rules:

Even better way to mitigate this problem is to change your deployment workflow and don’t have .git in the same folder with your public files, of course. If you don’t use the repository anymore, you can safely delete it. Some deployment workflow tips:

I published the details and statistics about this scan on our company blog (in Czech and English) on 2018-08-29. If you want to be notified about it you can follow my twitter or subscribe here for news.

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 43 other followers

This site was modified for purposes of my recent scan – you can find older posts here.

124 thoughts on “Open .git global scan

  1. Great service – thanks for the help. Fixing the problem and sent a donation to encourage more of this kind of research.

  2. Appreciated, however am curious.. How did you come about your domain list? My domain is rather long and has little to no traffic. Just curious.

  3. Thanks, man. I appreciate the concern and followed according to your suggestions. Don’t worry, I didn’t think you hacked anything. Your clear and concise and error-free message was immediately identifiable as non-spam. 🙂 Keep up the good work

  4. I always thought servers automatically handled security of the hidden stuff. Thanks for the heads up and the solution.

  5. This was helpful to us. Where one has to manage a bunch of low-priority legacy websites and in-house resources are very limited it is not viable to security-test all sites. This was a helpful “heads-up” in a situation where the initial deployment was not clear and development of a site had gone cold.
    Thank you,

  6. Thanks! You wrote that you’d publish details about this scan. So will you be publishing the website names? I don’t think you’d be doing that.

    1. Hi, I won’t publish the website names, of course. The post will be about following topics: how did I prepare the data for scan; what technologies I used; what problems I had to solve; how long did it take + plus some statistics: how many sites was affected; what technologies they use etc.

    1. I think the editor removed the <Directory .git > tags, of cause you don’t want to deny the entire website.A

      1. You are right, thank you for notice. I have added examples for the Apache config into the main article.

  7. Many thanks Vladimir. Despite the comments made my some of the good folk above, I certainly don’t want my repository to be public and it’s a stark warning about exposing sensitive data in the repo . Good work man and thanks again.

  8. Thank you very much. Found an even more serious exposed .svn thanks to your advice. I owe you a beer.

  9. Thank you for the notification.
    To prevent it, I made following in the global apache config:

    Order deny,allow
    Deny from all

    You can use it in .htaccess too and for other tools like svn, …

  10. Security clickbait.

    1. Here are some other public repos: https://github.com/lynt-smitka?tab=repositories

    – I have scanned them.
    – They all contain public-readable git contents. 🙀
    – This is non-commercial work but paypal me $5 ples

    2. >I only detected a security problem on your website.

    No, you detected git repos. To claim anything more is absurd fearmongering. You are both annoying professional site operators and actively discouraging amateur site operators from using version control and/or self-hosting their own stuff. You know, all that small, bloggy, static, eminently-self-hostable stuff that you just told them is dangerous to expose on absolutely no grounds.

    At best, this is low-grade nonsense on your part and you (should) know it. Of course you will select for a comment section full of trite “thanks!” and the occasional misconfiguration tip-off. That doesn’t change the _blatant_ chilling effect of masscanning for vulns that aren’t vulns and spamming people about it.

    3. >No sensitive data was downloaded from your site except for your e-mail address

    No, you downloaded *an* email address. You then sent a security notice directly to that address without knowing anything about the site host’s relationship to that recipient. To call this behavior stupidly unconsidered is overly charitable. A more-calibrated characterization would be “disqualifying”, given your self-styled whitehat label.

    1. I think you’re being a bit hard on him. While it’s probably true that a lot of code is open source and this doesn’t do that much in some cases, it is a good thing to keep in mind that exposing .git could make it easier to take advantage of your site. I mean at least some of the various thank yous have probably gained something from making the suggested changes. He also expressed interest in improving the script to not include open source code, and potentially could improve it to only apply to non-static websites as well.

    2. Hi, thank you for your opinion.

      Let me reply to your comment as there are some things I can’t agree with.

      1 – there is certainly a huge difference between intentional publishing your own sources on the github and if you ununtentionally make your private files public. I have already made a similar scan in a smaller scale before and examined the content of repositories – lots of them contained data you don’t want to share.

      You are absolutelly right, that some repositories are only a copy of public available repos and it is my fault not to exclude them from the scan – it is easy to recognize them from /.git/config file. Therefore it is possible that some of my alerts were false positive. It is a good point and I will implement it in my future scans.

      The scan is really non-commercial, I don’t force anyone to donate. It is a voluntary acitivity and the purpose is only to cover some expenses of the scan.

      2 – I definitely do not discourage anyone from using VCS or open-source. On the contrary, I highly recommend to use them, but I only urge the users to have them properly configured. In my opinion, if a user doesn’t expect certain behaviour and leaves their repository open, it is when it becomes a serious vulnerability.

      3 – the emails I found in the repository are directly connected to the user/system who/which commited the changes to the repository, so the relationship is pretty clear.

      Cheers,
      VS

  11. At first I thought this was some spam gimmic, but sure enough you were dead right and thanks very much. I have since patched and notified other sites that I know of. Sent you a small thanks – enough for a beer or two. Much appreciated

  12. Thank you very much for the email!

    I’ve applied your fix, however, one thing I’ve realised is that depending on your .htaccess, users will still be able to directly access the files such as /.git/HEAD

    I’ve resolved this by manually adding a .htaccess in the .git directory for now with a “Deny from all”.

    Regards

  13. Thank you! I’ve occasionally put ‘.git’ location to a wrong host and may never notice this without your email!

  14. Thanks for your work!
    Closed it, though I do not consider it a security risk in my case – all the repos you found on my sites are public on github. But it’s a good reminder to think twice before you put anything into a document root.

    1. It is a good point to exclude public repos from the notices, it is easy to recognize them from /.git/config file. I will implement it to the next scans.

      1. Hi, we discussed it on twitter and the problem is more complex than it seems. I’m not able to recognize “full public repositories” without deeper data examination. There are many private repos on the github and even the local copy of the public repo can contain private commits. The best option (I think) will be to add a friendly apology to the alert message for the truly opensource project owners.

    1. Hi, it depends on your server confiuration. If the user who own your file is the same as the webserver user, it won’t be working.

  15. Thank you! I was performing some tests necessitating the git info, and I did not realize the .git folder was not at least 644… (As you mentioned there are better deployment workflows, though).

  16. Another solution for .htaccess (if you don’t use .well-known folder), is:
    RedirectMatch 404 /\..*$
    This will return 404 instead of 403.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s