Repository navigation
Segfaults when writing to Buffer using UCS2 encoding #2457
Description
The following code causes a segfault when write is called:
var text = '1234567890123456789012345678901234567890123456789';
var len = Buffer.byteLength(text, 'ucs2');
var buf = new Buffer(len);
var bytesWritten = buf.write(text, 0, 'ucs2');The odd thing about this bug is that it is intermittent, but consistent. Meaning, the code above always causes a segfault in my tests. However, when I change text to have an extra 0 at the end (12345678901234567890123456789012345678901234567890), it stops causing segfaults.
Even more confusing is that if I take that same piece of text with a 0 at the end, the variant that doesn't cause a segfault, and use it in the following code, it also always causes a segfault.
This code is also the code I've been using to test this bug. Basically, I've been calling this script with a bunch of sample text files, and the only encoding that ever causes a segfault is ucs2.
Test script:
var text = '12345678901234567890123456789012345678901234567890';
if (process.argv.length > 2) {
var lastArg = process.argv[process.argv.length - 1];
if (lastArg.indexOf('-') !== 0) {
var fs = require('fs');
text = fs.readFileSync(lastArg).toString();
}
}
console.log('Text to write:', text);
console.log();
console.log(text.length + ' characters');
function write(encoding) {
console.log();
console.log('=== Writing using ' + encoding + ' encoding');
console.log();
try {
var len = Buffer.byteLength(text, encoding);
console.log('Required buffer length: ' + len);
var buf = new Buffer(len);
var bytesWritten = buf.write(text, 0, encoding);
console.log('Wrote ' + bytesWritten + ' bytes');
console.log(buf);
} catch (err) {
console.log('Error during write:', err);
}
};
write('base64');
write('binary');
write('utf8');
write('hex');
write('ascii');
write('ucs2');Example output:
Text to write: 12345678901234567890123456789012345678901234567890
50 characters
=== Writing using base64 encoding
Required buffer length: 37
Wrote 37 bytes
<Buffer d7 6d f8 e7 ae fc f7 4d 76 df 8e 7a ef cf 74 d7 6d f8 e7 ae fc f7 4d 76 df 8e 7a ef cf 74 d7 6d f8 e7 ae fc f7>
=== Writing using binary encoding
Required buffer length: 50
Wrote 50 bytes
<Buffer 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30>
=== Writing using utf8 encoding
Required buffer length: 50
Wrote 50 bytes
<Buffer 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30>
=== Writing using hex encoding
Required buffer length: 25
Wrote 25 bytes
<Buffer 12 34 56 78 90 12 34 56 78 90 12 34 56 78 90 12 34 56 78 90 12 34 56 78 90>
=== Writing using ascii encoding
Required buffer length: 50
Wrote 50 bytes
<Buffer 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30 31 32 33 34 35 36 37 38 39 30>
=== Writing using ucs2 encoding
Required buffer length: 100
[1] 3798 segmentation fault (core dumped) node main.js
Backtrace:
#0 0x0000000000b6c595 in void v8::internal::String::WriteToFlat<unsigned short>(v8::internal::String*, unsigned short*, int, int) ()
#1 0x000000000085124a in v8::String::Write(unsigned short*, int, int, int) const ()
#2 0x0000000000d9980a in node::StringBytes::Write(v8::Isolate*, char*, unsigned long, v8::Local<v8::Value>, node::encoding, int*) ()
#3 0x0000000000d74de8 in void node::Buffer::StringWrite<(node::encoding)3>(v8::FunctionCallbackInfo<v8::Value> const&) ()
#4 0x00000000008653f7 in v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) ()
This bug has acted very oddly, but yet I've managed to replicate it both on my current machine and on a fresh install of io.js on a separate computer. It affects versions 3.0.0 and 3.1.0, but I haven't tested any older versions.
Activity
ChALkeR commentedon Aug 20, 2015
I can't reproduce it using either of those two supplied scripts for some reason.
alexlamsl commentedon Aug 20, 2015
Couldn't reproduce with
iojs 2.5.0on Windows Server 2012 R2However, on an unrelated note, when I made a typo with
encoding,Buffer.byteLength()does not err:Whereas
buf.write()would throwTypeError: Unknown encoding: usc2skomski commentedon Aug 20, 2015
Seems like a compiler optimization bug: I can only reproduce it on master with gcc (5.2.0) if I compile with -O3 not -O2 and with clang it works fine on -O3.
ChALkeR commentedon Aug 20, 2015
@vsimonian Did you build the binaries (using which you could reproduce the bug) yourself?
If yes, what was the compiler version and what were the options?
bnoordhuis commentedon Aug 20, 2015
Can someone post a stack trace (from gdb or lldb) of the crash?
adalinesimonian commentedon Aug 20, 2015
@ChALkeR I'm using binaries from the Nodesource Debian/Ubuntu repository. Should I give compiling io.js a try?
skomski commentedon Aug 20, 2015
Only crashes with
gcc -O3hence this all can be fixed by specifying-fno-tree-loop-vectorize.The problem is maybe undefined unaligned access which results with the vectorizer into a crash like https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58039
indutny commentedon Aug 20, 2015
@bnoordhuis could be alignment problem?
kzc commentedon Aug 20, 2015
char* buf arg is not aligned:
buf=0x00000000011cd90bsrc/string_bytes.cc:
https://github.com/nodejs/node/blob/master/src/string_bytes.cc#L337
indutny commentedon Aug 20, 2015
Yay, bullseye!
@kzc may I ask you to give a try to this patch?
kzc commentedon Aug 20, 2015
@indutny - I didn't run it. Just inspected the source code after looking at the stack trace.
indutny commentedon Aug 20, 2015
Oh, right! cc @skomski
kzc commentedon Aug 20, 2015
regarding:
did you mean:
indutny commentedon Aug 20, 2015
Yikes, sure! Thanks for pointing out.
6 remaining items
indutny commentedon Aug 20, 2015
@kzc this is correct... I guess I should do a memcpy instead.
bnoordhuis commentedon Aug 20, 2015
@indutny I fixed a similar bug a while ago in commits 535fec8 and 56fde66, but for StringBytes::Encode(). There's probably a way to factor out the common code. You may have to take endianness into account.
kzc commentedon Aug 20, 2015
@indutny - the logic regarding
ncharsand thememmoveseems not quite right to me, but I may be mistaken. I think you'll need a number of test cases for unalignedbufUCS2 string conversions of varying lengths to prove it is correct.indutny commentedon Aug 21, 2015
Here is the fix: #2480 cc @kzc @bnoordhuis
adalinesimonian commentedon Aug 21, 2015
Not sure how much help this may be, but building io.js 3.1.0 with
results in the code triggering a segfault, but building with @indutny's latest version of patch #2480 with the same options fixes the segfault bug.
indutny commentedon Aug 21, 2015
Thank you @vsimonian !
shernshiou commentedon Aug 21, 2015
+1
string_bytes: fix unaligned write in UCS2
indutny commentedon Aug 21, 2015
Fixed!
adalinesimonian commentedon Aug 21, 2015
Fantastic work, @indutny! Thank you! 👍
string_bytes: fix unaligned write in UCS2