Drop-in Replacement for fs to prevent too many open files
As of version 0.4.0,
Filequeue supports Node 0.10.x, and as of version 0.5.0, it has basic Streams support.
Filequeue was born out of my encounter with
Error: EMFILE, too many open files, which occurs when you try to open too many files at once on your system. Due to Node's asynchronous nature, if you perform a lot of
fs.readFile or similar operations in quick succession, you can easily hit your system's
maxfiles limit, usually set to 256 on a dev box.
Filequeue creates a replacement for
fs, that I use as
fq with many of the same operations. However, it keeps track of how many files are open at once, and queues them if there are too many.
$ npm install filequeue
or using Git
$ git clone git://github.com/treygriffith/filequeue.git node_modules/filequeue/
var FileQueue = require'filequeue';var fq = 100;// additional instances will attempt to use the same instance (and therefore the same maxfiles)var FileQueue2 = require'filequeue';var fq2 = 100;console.logfq === fq2; // => true// you can force a new instance of filequeue with the `newQueue` parametervar fq3 = 100 true;console.logfq === fq3; // => false
forvar i=0; i<1000; i++fqreadFile'/somefile.txt' encoding: 'utf8'console.log"data from somefile.txt without crashing!" somefile;;
Adding a new
fs method is simple, just add it to the
methods.js file following the conventions therein.
Pull requests to add other fs methods with tests exercising them are welcome.