You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Large strings decoded from latin1 and then encoded to utf8 has wrong size #22728
Platform: Linux xyz 4.17.5-200.fc28.x86_64 deps: update openssl to 1.0.1j #1 SMP Tue Jul 10 13:39:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Subsystem:
Decoding a latin1 buffer larger than about 1MB to a string and encoding that string into utf-8 gives a buffer with the same number of bytes as the latin1 input even though more are required for characters that use more space in utf-8.
This seems to work properly on v10.x but not v8.x or v9.x.
Code that demonstrates the problem:
constfs=require('fs');consts='Räksmörgås';letss='';constSIZE=(1024*1024);// works://const SIZE = (1024 * 512);while(ss.length<SIZE){ss=ss+ss.length+' '+s+'\n';}// create latin1 buffer we can decodeletl1Buffer=Buffer.from(ss,'latin1');letl1String=l1Buffer.toString('latin1')// also fixes it:// l1String = ('x' + l1String).substring(1, l1String.length + 1);// create utf8 buffer from decoded latin1 stringletu8Buffer=Buffer.from(l1String,'utf8')console.log(l1Buffer.length);console.log(u8Buffer.length);if(l1Buffer.length===u8Buffer.length){console.log('failed, should be different size');}else{console.log('works');}
Decoding a latin1 buffer larger than about 1MB to a string and encoding that string into utf-8 gives a buffer with the same number of bytes as the latin1 input even though more are required for characters that use more space in utf-8.
This seems to work properly on v10.x but not v8.x or v9.x.
Code that demonstrates the problem: