.gitignore Is Inherently Sisyphean

(rgbcu.be)

45 points | by RGBCube 195 days ago

23 comments

  • yencabulator 193 days ago
    This is a bad idea and if this seems necessary you are interacting with very low quality committers. People who don't even bother reading their own commits do not deserve to get their commits merged.

    Also, that gitignore doesn't even work. man gitignore:

           •   An optional prefix "!" which negates the pattern; any
               matching file excluded by a previous pattern will become
               included again. It is not possible to re-include a file
               if a parent directory of that file is excluded. Git
               doesn’t list excluded directories for performance
               reasons, so any patterns on contained files have no
               effect, no matter where they are defined. [...]
    • cstrahan 193 days ago
      > Also, that gitignore doesn't even work.

      It's not terribly far off. This works:

        diff --git a/.gitignore b/.gitignore.new
        index 7dc7aea..fc9ebfe 100644
        --- a/.gitignore
        +++ b/.gitignore.new
        @@ -3,17 +3,21 @@
         !.gitignore
         
         # whitelist `src` directories and their children, regardless of place
        +!src
         !src/**/
         !src/**/*.rs
         !Cargo.{toml,lock}
         
         # whitelist root `pysrc` directory
        +!/pysrc
         !/pysrc/*.py
         !pyproject.toml
         !poetry.lock
         
        +!/cmd
         !/cmd/*.go
         !main.go
         !go.{mod,sum}
         
        +!/docs
         !/docs/*.md
      
      This is how I've been managing my dotfiles for over a decade (https://github.com/cstrahan/dotfiles/blob/master/.gitignore).

      This is the email thread I started back on 2016/03/3 to pin down the gitignore behavior that we now have today (there was a regression between git version 2.6.0 and 2.7.0):

      https://lore.kernel.org/git/1457057516.1962831.539160698.3C8...

      (For posterity: the subject line was "Change in .gitignore handling: intended or bug?")

    • arcfour 193 days ago
      I have never once experienced this problem. Mostly because I don't work with people who blindly commit everything without even looking once to see what's in their commit.
      • pxc 193 days ago
        I use gitnr[1] to add fairly comprehensive `.gitignore` files to my repos at creation time.

        Ultimately, it's the responsibility of each developer to determine whether or not the files they've added to their local clone really belong in source control! I've never encountered this issue, either, and I think I'd be unhappy working on a team where it was a serious problem.

        That said, the whitelist approach does seem kinda interesting. It's really what should be present from the start, since Git had a manual staging process. But maybe this gap highlights a legit usabiiity issue. Maybe "add everything" plus an explicit, version-controlled whitelist is somehow easier or more intuitive than the "add nothing" default.

        --

        1: https://github.com/reemus-dev/gitnr

        • arcfour 193 days ago
          In the Go world, the whitelist approach is a popular alternative to the standard gitignore, since Go projects can kind of take any structure the developer sees fit.

          To me the normal approach makes equal sense. I create a file, but I'm not ready for other people to see it yet, because it's just a placeholder. So I don't add it. But another set of files are ready, and are part of one logical step of the change, so I add them specifically and commit.

        • plorkyeran 193 days ago
          Interns and new hires sometimes need a quick chat about actually looking at their changes before asking for a review, but that's about it in my experience. I guess it could be a problem for a project which accepts a large number of public PRs from mostly non-technical or beginner users?
          • vdfs 193 days ago
            It's not a problem with PRs if you use Squash merge, the developer can make any number of changes to that PR but in the end, only the final version (in this case it doesn't include .DS_Store) will be commited. even the .DS_Store file will not be in the git index
            • matheusmoreira 192 days ago
              That sacrifices the git history though. It turns the PR into a huge atomic change instead of a series of step-by-step changes that are easy to review.
            • arcfour 193 days ago
              Yes, I have run into that once or twice but never to the extent that I would call it "sisyphean", haha.
            • wpollock 193 days ago
              I had this issue all the time. Until I retired as a CS college professor. An accept-list is the way to go when you have lots of people making commits.
            • edflsafoiewq 193 days ago
              Ideally people who have .DS_Store, .vscode, etc. would put them into their own $GIT_DIR/info/exclude files and not pollute the shared .gitignore with stuff from their personal setup.
              • nomel 193 days ago
                I've found that reducing the friction, as much as possible, is the only way to get people to actually contribute.

                Excluding things like .vscode and .DS_store, which is a common and universal "don't need that in my repo", is a trivial way to reduce the burden for everyone (the goal of software, imo), maintainer included.

                • eadmund 193 days ago
                  > Excluding things like .vscode and .DS_store

                  I’m old and cranky enough to want to believe that excluding contributions from VS Code and macOS users is a trivial way to reduce the maintainer’s burden and improve software quality.

                  That’s completely unfair, of course, and probably false too.

                  edit: s/maybe/probably/

                  • Ferret7446 193 days ago
                    I don't think you need to fencesit that hard. VS Code and macOS users *that don't know how to double check their commit or ignore their junk files locally will definitely cause extra burden and lower software quality. If they can't check their commits for .vscode, do you really think they tested their code properly or thought hard about the code that they wrote? If they don't know abut local git ignore files, do you think they know the language and libraries well enough? If they didn't read the Git docs, then did they read the docs that said to not use a specific function in this way?
                  • Ferret7446 193 days ago
                    If you want to attract novice contributors, sure. This is as much give-and-take as anything else; if you want these kind of people, then you're going to have to throw a bone to these people.

                    Now, if you don't want contributors, or at least novice contributors that don't know to proofread their commits or ignore their own IDE files, then this actually helps you screen out the contributors you don't want to engage with early.

                  • mook 193 days ago
                    Just use core.excludesFile, once per machine.

                    https://git-scm.com/docs/git-config#Documentation/git-config...

                    • gruez 193 days ago
                      [deleted]
                      • do_not_redeem 193 days ago
                        This post is about unwanted files ending up in git. So if you don't use git and you collaborate by swapping floppy disks or whatever, you won't have this problem. (But you'll have other problems, of course)
                      • meatmanek 193 days ago
                        What project is using .gitignore but not git?
                        • gpm 193 days ago
                          Lots of non-git tools use them. Some that come to mind are ripgrep (by default ignores .gitignore'd files), some editors (greying them out in the file selector, not searching them, etc), rustic (not backing them up), probably more.
                          • weikju 193 days ago
                            For one thing, many AI code helpers use the contents of .gitignore as a suggestion for files to not include in their context or changes.
                            • gruez 193 days ago
                              My bad, I misinterpreted the proposal.
                          • adastra22 193 days ago
                            TIL
                          • accoil 193 days ago
                            I think people should be responsible for the litter of their own tools. If their commits have include any litter it should be rejected as they clearly aren't paying attention. The local gitignore is for project specific files.
                            • sneak 193 days ago
                              The fact that toil can be pushed off onto other people (the pushing and feedback and re-review also being additional time and effort) doesn’t change the fact that it’s toil and it’s stupid and wasteful.
                              • Brian_K_White 193 days ago
                                No. Wrong take. Wrong location for obligation. This absolute collective toil argument would imply getting these into some kind of default baked into the tools, not something every new project should go out of their way to do something only for the benefit of everyone else but themselves. That is the pushed off work.
                            • bravetraveler 193 days ago
                              Common pattern/solution, not a common problem. Feels equal-if-not-more Sisyphean. Also: "Oh no, my collaborative maintenance requires both collaboration and maintenance"

                              Don't accept people playing with shotgun commits; there, problem(s) solved.

                              • dogleash 193 days ago
                                > "Oh no, my collaborative maintenance requires both collaboration and maintenance"

                                chef's kiss This is replacing one of the memes on my whiteboard rn

                              • hansvm 193 days ago
                                This isn't a terrible way to go about things. I normally add

                                  .*/
                                
                                to the gitignore. Most crap I don't want is a hidden directory. It's easy to whitelist a few bad actors, and the gitignore stays pretty short.
                                • tonymet 193 days ago
                                  this really should be git's default, with a warning on git init .

                                  They decided to waste the default warning with a wrist-slap for using "master"

                                  • eadmund 193 days ago
                                    > They decided to waste the default warning with a wrist-slap for using "master"

                                    I think that one can eliminate that by adding this to /etc/gitconfig, ~/.config/git/config or ~/.gitconfig as appropriate:

                                        [init]
                                                defaultBranch = master
                                    • tonymet 193 days ago
                                      that's right but often CI systems are ephemeral or locked down so the logs get cluttered with this noise
                                      • eadmund 193 days ago
                                        That’s fair. They really should revert that nonsense.
                                        • Brian_K_White 192 days ago

                                            $ git init
                                            hint: whoa whoa whoa my dude "master" no...
                                          
                                            $ git config --global init.defaultBranch aids_is_gods_punishment_for_depravity
                                            $ git init
                                            ok
                                          • tonymet 192 days ago
                                            my favorite song from team america!
                                • kazinator 192 days ago
                                  Or don't have people in your project that add random files or directories, and can't even explain what they are.

                                  I don't use .gitignore files in projects that I control. A few times I've been tripped up by these things ignoring something that should not have been.

                                  I feel that these files should stay local. I don't want your .gitignore any more than I want your .vimrc or whatever. If you maintain a very messy local repo with lots of extraneous files, and find that this file helps you when you're using git, by all means create one for yourself.

                                  • ivanjermakov 193 days ago
                                    I think .gitignore design is flawed. If I don't want file to be versioned, I don't want its _mention_ in VCS either.

                                    Of course there is .git/info/exclude, but I'm not sure it's working an already cached (present on remote) files. This is another common use case git handles badly - local changelist. For example having .env file on remote with variable names only to denote structure and filling those out in local without showing these modifications in staging.

                                    • Arrowmaster 193 days ago
                                      .git/info/exclude and .gitignore are processed the exact same way.

                                      In the very first line of the description on https://git-scm.com/docs/gitignore it states "A gitignore file specifies intentionally untracked files that Git should ignore. Files already tracked by Git are not affected; see the NOTES below for details."

                                      This is a core behavior of git that has existed since creation. The gitignore is really only there to keep git status tidy and let people blindly use git add -A. Once a file is added to a repo it is never ignored until it is removed. This is why you should should rename your example .env to .env.example.

                                    • davydm 195 days ago
                                      Cool post.

                                      I highly suggest using templates from https://github.com/github/gitignore for your project as they tend to include a lot of files commonly found in them - eg .idea folders and so on.

                                      • FrenchyJiby 193 days ago
                                        Yes-anding this to say:

                                        For per-user gitignore, one can set the following config in ~/.gitconfig:

                                            # System-wide gitignore file
                                            # To avoid in-repo repetition for system-specifc garbage files
                                            [core]
                                                excludesfile = ~/.config/git/gitignore
                                        
                                        
                                        Then your file at ~/.config/git/gitignore can be a normal format, and not need to be stored in repo. Perfect for system files (.DS_Store...) or editor swap files etc.
                                        • Baarsgaard 193 days ago
                                          Git will per default read ~/.config/git/ignore No need to configure it :)
                                          • tonymet 193 days ago
                                            great tip!
                                          • RGBCube 195 days ago
                                            They're still not enough. Even with those, it's really easy for stuff to slip in.

                                            You should just pick the nearly exhaustive option instead

                                          • elijahwright_ 193 days ago
                                            there's a good reason why this isn't done this way. Git is very open ended but there are .gitignore templates out there. ideally anyone submitting a pull request should be looking at what they're adding. but the real issue here is this:

                                            > Luckily, you realize that you can turn the blacklist of files (the gitignore) into a whitelist, by just ignoring everything and manually un-ignoring desired files.

                                            this will not work and tbh it's fitting a square peg in a round hole. it technically might fit if you shave off the square enough but large repos don't do this, they use a round peg: https://github.com/github/gitignore

                                            • zzo38computer 193 days ago
                                              I did not know that it can be used for whitelist (although I probably would have figured it out if I use mainly git for version control, which I don't), although I had not used .gitignore and I used "fossil add" to add the files that I wanted instead, which is also a kind of whitelist.

                                              I did think the blacklist for this purpose was a bad idea anyways, before I saw this, so I used whitelist anyways already. (However, you can use which way you prefer, in case you do not like this, either.)

                                              • thunderbong 193 days ago
                                                Fossil by default ignores files and directories starting with a dot.

                                                So, even of you explicitly add a directory, any files or directories starting with a dot inside it gets skipped.

                                                • zzo38computer 190 days ago
                                                  Normally I think it is useful to ignore them in that case. Nevertheless you can (and I have done) explicitly added file names even if the directory name starts with a dot, such as ".fossil-settings/encoding-glob" and ".github/workflows/auto-assignment.yml".
                                                  • thunderbong 190 days ago
                                                    We do the same thing when specific files need to be added even though they are either dot files or within dot directories.
                                              • etbebl 193 days ago
                                                This is like an infomercial contrived situation... Sure you have to update the .gitignore sometimes, but Sisyphean? Nahh. It takes 30 seconds.
                                                • seeknotfind 193 days ago
                                                  I didn't see the con explicitly in the comments. When someone adds a file, they'll have trouble figuring it out why it's not visible in git, or they may not realize an important file is missing if they are adding multiple.
                                                  • karunamurti 193 days ago
                                                    Somebody didn't know about https://github.com/github/gitignore
                                                    • Brajeshwar 195 days ago
                                                      Yes, the time I used to hook up "rm `find ./ | grep DS_Store`" before anyone do a `git add`
                                                      • latexr 193 days ago
                                                        That command doesn’t feel safe. It’ll delete anything with `DS_Store` in the name (even if it has other text). Does that even work properly? You’re relying on newlines to separate the files. I can imagine a different IFS easily screwing that up. In the end, you’re invoking three different utilities to do a janky operation which can be done properly with a single one:

                                                          find . -name '.DS_Store' -delete
                                                        • arcfour 193 days ago
                                                          You can create a global gitignore when forced to work on a Mac that excludes these.
                                                        • nickcw 193 days ago
                                                          This story definitely resonates with me.

                                                          While I don't think we've ever merged a pr with ide files, we do merge the .gitignore part when people put them in their commits which has lead to growth over time...

                                                          https://github.com/rclone/rclone/commits/master/.gitignore

                                                          • pigbearpig 193 days ago
                                                            Interesting. Makes sense for open source. In the workplace for those using common IDEs things like .vscode or .idea can definitely help with consistency or shared project setup. Each has docs which mention which files should or shouldn't be committed. Personally, I just use gitignore.io to generate the file based on my company's tooling and call it good enough.
                                                            • tom_ 193 days ago
                                                              That's what I've always seen done with git or Perforce. The initial set of all likely IDE-related files is known (just talk to the people you work with...), so you just add all of them to the ignore list (of whatever kind) and off you go. Then over time, additional entries might get added, as new platforms become supported and/or new team members join who use other IDEs. And the easiest thing is to add them to the list, because if one person is using this or that tool then there's a fair chance that somebody else will be using it in future too.

                                                              One way of thinking about this is to note that there are an awful lot of these tools, and so each person should look after their own shit rather than fucking it up for everybody else. Another is to note that the number of these tools is finite, and team members won't be using all of them, and so over time the centralized ignore file will tend towards a useful degree of completeness.

                                                            • singingfish 193 days ago
                                                              If you want to just stop yourself from shooting yourself in the foot in your current checkout and maintain privacy around your own stupidity, you can add patterns to `.git/info/exclude`
                                                              • milesrout 193 days ago
                                                                No. IDE-specific ignore lists should go in $CONFIG/git or .git/info/exclude (or whatever the path is).

                                                                (git config --global core.excludesfile "$CONFIG/git/ignore")

                                                                .gitignore is committed to the repository and should be for files created by the repository that shouldn't appear in 'git status' like build/, etc.

                                                                • callc 193 days ago
                                                                  Alternatively, ignore EVERYTHING except the files you need.

                                                                  Puts the onus on the committer

                                                                • paulyy_y 193 days ago
                                                                  Allow/deny list, hope that helps.
                                                                  • zzo38computer 193 days ago
                                                                    This list is not really to allow permissions; it is the including and excluding list.