You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
“Unknown system error -75” nodejs fs.stat large files #19455
I don't know what happened on my system (raspberry pi, armhf, latest nodejs version) but since a few days I cannot fs.stat a file that is larger than 2gb
Error: Unknown system error -75: Unknown system error -75, stat 'myfile'
at Object.fs.statSync (fs.js:972:11)
at Object. (/home/pi/nodeapps/fapp_node_pwa/test.js:2:22)
at Module._compile (module.js:649:30)
at Object.Module._extensions..js (module.js:660:10)
at Module.load (module.js:561:32)
at tryModuleLoad (module.js:501:12)
at Function.Module._load (module.js:493:3)
at Function.Module.runMain (module.js:690:10)
at startup (bootstrap_node.js:194:16)
at bootstrap_node.js:666:3
`
I know for sure that it worked a few days ago. it's like I'm suddenly on a 32bit system (or node or whatever).
I thought it was the current nodejs version 9.8.0, but i've tested it with 9.7.1 and 9.7.0 and it's the same.
EOVERFLOW pathname or fd refers to a file whose size, inode number, or number of blocks cannot be represented in, respectively, the types off_t, ino_t, or blkcnt_t. This error can occur when,
for example, an application compiled on a 32-bit platform without -D_FILE_OFFSET_BITS=64 calls stat() on a file whose size exceeds (1<<31)-1 bytes.
(not sure about arm, but on x64 Linux EOVERFLOW is 75, so that checks out).
So, yes, this is the most likely cause… it might be a good idea to check whether your node executable is built for 64 bit support? file $(which node) would be a good starting point
i still not get it. it worked fine until it didn't. the only things i am doing on this pi is updating it from time to time (the usual apt-get update/upgrade/dist-upgrade)
@filly82 Did you (intend to) install Node via your package manager or another way? (I think dpkg -S /usr/bin/node would tell you that in the case of apt-get).
In any case, the BuildID from that output matches the one for the official node-v9.8.0-linux-armv7l download from https://nodejs.org/, so somehow that binary made its way from there onto your machine.
i do not really care how to install node. i chose the easiest way back then with the script from deb.nodesource.com
i've also tried the binaries from https://nodejs.org a few minutes ago, the same problem
the only binaries i can execute on my raspberry pi are the armv7 ones. what do you mean with your last sentence that that binaries made their way onto my machine? i think that binaries were always on my machine since they are the only ones which work. and i know for sure that i had not have the problem with large files until a few days ago
the only binaries i can execute on my raspberry pi are the armv7 ones
Those are 32-bit binaries, though. If you have an armv7 machine, you won’t be able to run 64-bit Node binaries, as I understand it. So I’m really not sure how this could have worked in the first place then…
I suppose we could build Node with -D_FILE_OFFSET_BITS=64 to make sure that we support this, though? /cc @bnoordhuis
what i can say for sure is that my node app has been running for a few months and that i always had files bigger than 2gb. i am 100% sure that those files were correctly "stat"-ed until a few days ago. they must have, because I've worked with the results every day for the past few months
i'll try to install an arm64 OS (unfortunately there are no stable distributions for the raspberry pi with 64bit support, i did not really knew that until that problem popped out). that would fix my problem, i think. but i am still wondering how it could work with the current constellation for a few months
/cc @nodejs/libuv we had to stop at 1.19.2 due to incompatible changes in 1.20.0... tbh I can't remember exactly what they are but I think @cjihrig would know.
Would you like to open another issue to track this?
I think you're referring to #22814 and #21203. Both of those issues are fixed. A regression test for #22814 landed in #23051. #23053 fixed #21203, but it's marked as don't land on v10 and earlier. Now, that might be backportable since it reverts a semver major and the version of libuv can now be updated on those older branches (cc: @addaleax since that is her PR).
Activity
addaleax commentedon Mar 19, 2018
From
stat(2):(not sure about arm, but on x64 Linux
EOVERFLOWis75, so that checks out).So, yes, this is the most likely cause… it might be a good idea to check whether your
nodeexecutable is built for 64 bit support?file $(which node)would be a good starting pointfilly82 commentedon Mar 19, 2018
that is the output
file $(which node)/usr/bin/node: ELF 32-bit LSB executable, ARM, EABI5 version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.26, BuildID[sha1]=46883207b843e59d049607d8bdfefb8de2bcf7a9, strippedi still not get it. it worked fine until it didn't. the only things i am doing on this pi is updating it from time to time (the usual apt-get update/upgrade/dist-upgrade)
addaleax commentedon Mar 19, 2018
@filly82 Did you (intend to) install Node via your package manager or another way? (I think
dpkg -S /usr/bin/nodewould tell you that in the case ofapt-get).In any case, the
BuildIDfrom that output matches the one for the officialnode-v9.8.0-linux-armv7ldownload from https://nodejs.org/, so somehow that binary made its way from there onto your machine.filly82 commentedon Mar 19, 2018
i do not really care how to install node. i chose the easiest way back then with the script from deb.nodesource.com
i've also tried the binaries from https://nodejs.org a few minutes ago, the same problem
the only binaries i can execute on my raspberry pi are the armv7 ones. what do you mean with your last sentence that that binaries made their way onto my machine? i think that binaries were always on my machine since they are the only ones which work. and i know for sure that i had not have the problem with large files until a few days ago
addaleax commentedon Mar 19, 2018
Those are 32-bit binaries, though. If you have an armv7 machine, you won’t be able to run 64-bit Node binaries, as I understand it. So I’m really not sure how this could have worked in the first place then…
I suppose we could build Node with
-D_FILE_OFFSET_BITS=64to make sure that we support this, though? /cc @bnoordhuisfilly82 commentedon Mar 19, 2018
what i can say for sure is that my node app has been running for a few months and that i always had files bigger than 2gb. i am 100% sure that those files were correctly "stat"-ed until a few days ago. they must have, because I've worked with the results every day for the past few months
i'll try to install an arm64 OS (unfortunately there are no stable distributions for the raspberry pi with 64bit support, i did not really knew that until that problem popped out). that would fix my problem, i think. but i am still wondering how it could work with the current constellation for a few months
bnoordhuis commentedon Mar 20, 2018
It is - at least, libuv is, and that's what makes the system call.
@filly82 Can you post the output of
strace node -p 'fs.statSync("myfile")'?filly82 commentedon Mar 20, 2018
the output is massive
Details
bnoordhuis commentedon Mar 20, 2018
@filly82 It failed with
ReferenceError: test is not defined. Can you try again and make sure it fails with that -75 errno?filly82 commentedon Mar 20, 2018
sorry, i am not familiar with strace
when i do
node test.jsit fails with -75but when i do
strace node -p test.jsit does not
am i doing it right? test.js is the test node script with just fs.statSync('myfile')
filly82 commentedon Mar 20, 2018
i tried
strace node test.jsand got thisDetails
build: build with -D_FILE_OFFSET_BITS=64 again
bnoordhuis commentedon Mar 20, 2018
Thanks, I found the issue. Fix in libuv/libuv#1779.
6 remaining items
bnoordhuis commentedon Mar 20, 2018
It'll auto-close when the libuv PR is merged.
build: build with -D_FILE_OFFSET_BITS=64 again
deps: upgrade libuv to 1.20.0
deps: upgrade libuv to 1.20.0
deps: upgrade libuv to 1.20.0
mempf commentedon Oct 23, 2018
Is it possible to get this fix back-ported to Node 8.X LTS? We are still using node in a 32 bit linux environment that has to deal with files >2GB.
Thanks!
MylesBorins commentedon Oct 23, 2018
/cc @nodejs/libuv we had to stop at 1.19.2 due to incompatible changes in 1.20.0... tbh I can't remember exactly what they are but I think @cjihrig would know.
Would you like to open another issue to track this?
cjihrig commentedon Oct 23, 2018
I think you're referring to #22814 and #21203. Both of those issues are fixed. A regression test for #22814 landed in #23051. #23053 fixed #21203, but it's marked as don't land on v10 and earlier. Now, that might be backportable since it reverts a semver major and the version of libuv can now be updated on those older branches (cc: @addaleax since that is her PR).
MylesBorins commentedon Oct 23, 2018
I've opened a tracking issue #23833