Menu icon Foundation
HUGE Node_Modules Folder Sizes

So I've been using foundation for years now, and just started with F6. After a couple site designs, I noticed that my cloud services (GDrive and Dropbox where I store my repos) were crashing like crazy.

So I used one of my file scanning apps to see what was happening, and for some reason, ALL my sites now that are built using the manual zurb install via terminal instructions (from the online docs) are around 1GB per-site! - and there were SO MANY files being created in the back end inside the node_modules folder, I'm not sure what to do. 

Is this normal? If so, there's no way I can continue to use F6 if each site I build is going to have 1GB+ of random compiler files. Please help!! Maybe I'm just doing something wrong? All my sites pre-F6 combined (the top right corner of the image) over YEARS and dozens of sites don't compate to just one F6 site I'm working on.

 

File example

node moduleslarge filesSizelarge file sizes

So I've been using foundation for years now, and just started with F6. After a couple site designs, I noticed that my cloud services (GDrive and Dropbox where I store my repos) were crashing like crazy.

So I used one of my file scanning apps to see what was happening, and for some reason, ALL my sites now that are built using the manual zurb install via terminal instructions (from the online docs) are around 1GB per-site! - and there were SO MANY files being created in the back end inside the node_modules folder, I'm not sure what to do. 

Is this normal? If so, there's no way I can continue to use F6 if each site I build is going to have 1GB+ of random compiler files. Please help!! Maybe I'm just doing something wrong? All my sites pre-F6 combined (the top right corner of the image) over YEARS and dozens of sites don't compate to just one F6 site I'm working on.

 

File example

Ondřej Vašíček over 3 years ago

The same „size problem“ here. I just wanted to add, that the empty project is about 200 Mb big, which is also “too much” in my opinion. The issue is the “node_modules” folder. The dependencies are nested and nested and nested … almost infinitely. I’m a bit afraid, that they could be duplicated.

But anyway – is it possible to install dependencies via your foundation-cli globally (like the installation of foundation itself)? I guess it would solve the problem. I’ve tried “foundation new –global” but it didn’t work.

Even your foundation-cli says “Let's set up a new project. It shouldn't take more than a minute.” Well, it takes definitely more, cause it has to download 200 MB :)

 

Thank you for your answer and your work.

bill over 3 years ago

The "It shouldn't take more than a minute." is a bit of a joke, at least on my pc. I was wondering if it was just me. 

I'm not an expert on npm and node, but infinitely nested node_modules seems to be a fundamental part of it. Unfortunaltely, if you're on windows then it can cause big issues. (I just had loads of hassle deleting an F6 project because windows can't delete files with a path of over 260 characters). 

So it's not a fundamental issue with Foundation, but with the way node and npm work. 

That said, there might be some things the Foundation team could do to reduce the issue.

As I said, I'm not a node expert, but it sounds like something like DeDupe https://docs.npmjs.com/cli/dedupe might be one solution, possibly combined with Shrinkwrap. 

 

That said, my sites aren't close to 1Gb, so something may be going on there. Mine are around 200Mb, and one I've been working on for a while still hasn't gone over 215mb. Mind you, that is in 73,331 Files in 10,949 Folders, which seems a little excessive. 

For the OP, I'm not sure if dedupe will help or will mess things up. I may try running it on a fresh project to see what happens. I'm not sure how you use the cloud services, but another option would be to exclude the node modules folder from upload and just keep a local copy... if you aren't doing everything directly on the cloud. 

Ondřej Vašíček over 3 years ago

It’s true about the Windows. I’m forced to delete Foundation projects in total commander via shift+del.

I tried the dedupe and it’s quite power consuming. I took a lot of time even on powerful CPU, but in the end, it has almost a half of the size – thanks!
But one would wonder, why NPM doesn’t do this automatically. I naively assumed, that it’s a standard part of every package system.

But I think that global installation of packages would definitely solve the problem, because every package would be installed globally, not in the folder of the project.

 

In many cases, it’s probably not god idea to install every package globally, but here, if we use Foundation over and over, it’s pretty neat I suppose. I just don’t know how to force Foundation CLI to install npm packages globally.

 

Dave Daverson over 3 years ago

Updating node/npm helped me with the nested dependency issues on Windows. It significantly cut down the size of my node_modules directory and eliminated the frustrating 260 character path issue.

npm -v

3.8.0

 

node -v

v4.3.2

Dominik Pieper over 3 years ago

As Dave said its a thing with the npm version. Older versions of npm saved the dependencies as a (sometimes very deep) tree. Since npm v3 npm tries to build a flat node_modules folder. The tree should only go deeper if several libraries need the same dependency in various versions.

 

More: https://docs.npmjs.com/how-npm-works/npm3

Ondřej Vašíček over 3 years ago

Well something must be wrong. Few months ago, before installing W10, after update node, I remember that happened what you said - the structure got flat. But now, on W10 I have npm 3.8.6. and node 4.4.6. and the structure is deep like eyes of …. some girl with deep eyes :)

Dominik Pieper over 3 years ago

Mhh, maybe they have changed something again.

Alex Kempton over 3 years ago

I'm all for keeping things modern and using these new technologies but when an empty startup project takes up over 200MB, it's a bit ridiculous! Please work to fix this Zurb.

bill over 3 years ago

FYI, running npm dedupe on a fresh foundation 6 site reduced it from 200Mb to 85Md and from 73,000 files to 21,000 files. 

It did take bloody ages though. seems to be working ok so far, but I haven't tested extensively.  

I'm on NPM v > 3 so it seems like 200mb is the already optimised by npm size. 

Not strictly relevant, but I came across useful stuff for anyone who's on windows, since node on windows is full of gotchas, and even upgrading versions can be a risk:

MS Guidelines for using node: https://github.com/Microsoft/nodejs-guidelines

(especially https://github.com/Microsoft/nodejs-guidelines/blob/master/windows-environment.md#configuring-your-windows-development-environment )

Update node safely: https://github.com/coreybutler/nvm-windows

Update npm safely: https://github.com/felixrieseberg/npm-windows-upgrade

 

Maurice Stuffer almost 3 years ago

I came across the same issue. After upgrading to NPM V3.x the manuel installation -> cloning from git and then installation of node and bower manually - doesn´t throw indefinite folder structures but a nice NPM 3 flat design. However if you use foundation new on the console it will deliver the NPM2 nested structure. I have no idea why though.

Bernardo Presser about 2 years ago

Solved it by using npm pack. The folder went from 200mb to 3mb!