MantisBT - v2.3 Release (Closed)
View Issue Details
0000603v2.3 Release (Closed)[All Projects] Generalpublic2010-10-14 04:522011-03-24 09:51
Reportergajdusek 
Assigned Tocaseydk 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target VersionFixed in Version2.3 
Summary0000603: Translation editor: backslashes in newline character (\n) are removed from both english and localized strings
DescriptionHi,

If you edit translations via Translation Management in System admin the slashes are removed from both english and localized strings. This happens both in loading and saving process and results in damaged strings stored in .inc files.

There are two strings in common.inc that contain new line character (\n):

common.inc:178:'sendpass1'=>'has this email associated with it.\nA web user from'
common.inc:179:'sendpass2'=>'has just requested that a new password be sent.\n\nYour New Password is:'

Currently localization in po/ directory is affected.

Problem is in using w2PformSafe(...,true) in modules/system/translate.php and calling stripslashes in translate_save.php.

Petr Gajdusek
TagsNo tags attached.
Attached Filespatch issue603_inc_files_fix.patch (4,685) 1969-12-31 16:00
https://bugs.web2project.net/file_download.php?file_id=141&type=bug
patch issue603.patch (4,201) 1969-12-31 16:00
https://bugs.web2project.net/file_download.php?file_id=142&type=bug

Notes
(0001313)
gajdusek   
2010-10-19 13:57   
(Last edited: 2010-10-19 18:50)
If you save translation all strings are passed through addslashes(stripslashes()) function. This lead to (1) inconsistency between manual and web-based edited localization files and (2) "\n" becomes "n".

i.e string
'Some "quoted string" and \nnew line'
from localization file will become
'Some \"quoted string\" and nnew line'
if you save it in web-based translation manager.

Whenever localization files are included and evaluated stripslashes() is applied so backslashes are removed and the string will become
'Some "quoted string" and nnew line' for both cases. The \n becomes pure n in both cases.

I see solution in escaping only sequences with special meaning in single quoted strings when saving translation changes and do not apply stripslashes() when loading them.

This can be the function used for escaping strings when saving them:

<code>
function export_single_quoted_string($str, $return = false) {
    $patterns = array(
        // escape backslashes of special meaning in single quoted strings (\\, \') and
        // last character if it is a backslash.
        "@(?| ((?:\\\\{2})*) | (\\\\)(') | (\\\\)($))@x",
        // escape apostrophes
        "@'@"
        );
    $replaces = array(
        '$1$1$2',
        '\\\\\''
    );
    $result = "'" . preg_replace($patterns, $replaces, $str) . "'";
    if ($return) return $result;
    print ($result);
}
</code>

Next step will be repair already damaged localization files, I found only polish localization affected, but did not make sure others are not.

And finally don't strip slashes while loading localization files. Maybe check for magic_quotes_gpc and magic_quotes_runtime. But these should be off because of .htaccess and because they are deprecated.

Petr

(0001351)
gajdusek   
2010-11-18 00:27   
Please see https://github.com/gajdusek/web2project/commits/issue603 with my patch fixing this.

Petr
(0001352)
gajdusek   
2010-11-18 00:50   
(Last edited: 2010-11-18 00:53)
Added patches that applies cleanly against SVN trunk (commit 1489).

issue603_inc_files_fix.patch fixes Polish locales: some strings were wrong escaped as result of using web2project translation manager suffering this bug

issue603.patch fixes the bug itself:

Fixed escaping of (single quoted) l10n strings in the web2project code
    
    This fixes http://bugs.web2project.net/view.php?id=603. Though, already
    existed errors in .inc files must be fixed first (previous commit
    does it).
    
    Details:
    
    Translation manager:
    * Escape only characters that must be escaped in single quoted strings
    before writing translation strings out to the .inc files.
    * Do not deslash translation strings while loading .inc files to
    the translation manager.
    
    ui.class.php:
    * Do not deslash translated strings returned by _() and __() methods
    of CAppUI class and replace '\n' with "\n" (UI_OUTPUT_RAW and UI_OUTPUT_JS
    flags) or with '&<br /&>' (UI_OUTPUT_HTML).

(0001686)
caseydk   
2011-02-26 22:48   
Can you file a pull request on 3453548?
(0001688)
caseydk   
2011-02-26 23:01   
Actually, after examining this one a more closely, the issue is a little different.

If you insert a linebreak via the translation interface in w2p, it preserves the linebreaks you've inserted. It loses the \n that are included in single quotes because the slashes are stripped as it comes in (as you note).

It looks like we have a trio of options here:
- continue using single quotes in translation files and insert real (not escaped) breaks as needed;
- use double quotes and use either escaped or real linebreaks as desired.
- use your solution to catch mis-escaped linebreaks and transform as needed.

Your solution - although it would work - seems like we're patching over something that is obviously a mistake and a problem.. a cleaner solution seems like the second option with properly escaped linebreaks.

Thoughts?
(0001732)
caseydk   
2011-03-20 17:05   
Fixed a bunch of places where newlines in translation strings were breaking the output;
Resolved in r1754, will be in v2.3 release;
(0001774)
caseydk   
2011-03-24 09:51   
Closed in preparation for v2.3 release.

Issue History
2010-10-14 04:52gajdusekNew Issue
2010-10-19 13:57gajdusekNote Added: 0001313
2010-10-19 13:58gajdusekNote Edited: 0001313
2010-10-19 18:50gajdusekNote Edited: 0001313
2010-11-18 00:27gajdusekNote Added: 0001351
2010-11-18 00:42gajdusekFile Added: issue603_inc_files_fix.patch
2010-11-18 00:43gajdusekFile Added: issue603.patch
2010-11-18 00:50gajdusekNote Added: 0001352
2010-11-18 00:52gajdusekNote Edited: 0001352
2010-11-18 00:53gajdusekNote Edited: 0001352
2010-12-18 21:20caseydkProjectv2.2 Release (Closed) => v2.3 Release (Closed)
2011-02-26 22:48caseydkNote Added: 0001686
2011-02-26 23:01caseydkNote Added: 0001688
2011-03-20 17:05caseydkNote Added: 0001732
2011-03-20 17:05caseydkStatusnew => resolved
2011-03-20 17:05caseydkResolutionopen => fixed
2011-03-20 17:05caseydkAssigned To => caseydk
2011-03-24 09:51caseydkNote Added: 0001774
2011-03-24 09:51caseydkStatusresolved => closed
2011-03-24 09:51caseydkFixed in Version => 2.3