Anonymous Login
2019-02-23 05:32 PST

View Issue Details Jump to Notes ]
IDProjectCategoryView StatusLast Update
0000603v2.3 Release (Closed)[All Projects] Generalpublic2011-03-24 09:51
Reportergajdusek 
Assigned Tocaseydk 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionfixed 
Product Version 
Target VersionFixed in Version2.3 
Summary0000603: Translation editor: backslashes in newline character (\n) are removed from both english and localized strings
DescriptionHi,

If you edit translations via Translation Management in System admin the slashes are removed from both english and localized strings. This happens both in loading and saving process and results in damaged strings stored in .inc files.

There are two strings in common.inc that contain new line character (\n):

common.inc:178:'sendpass1'=>'has this email associated with it.\nA web user from'
common.inc:179:'sendpass2'=>'has just requested that a new password be sent.\n\nYour New Password is:'

Currently localization in po/ directory is affected.

Problem is in using w2PformSafe(...,true) in modules/system/translate.php and calling stripslashes in translate_save.php.

Petr Gajdusek
TagsNo tags attached.
Attached Files
  • patch file icon issue603_inc_files_fix.patch (4,685 bytes) 1969-12-31 16:00 -
    diff --git a/locales/po/common.inc b/locales/po/common.inc
    index b226203..73599e7 100644
    --- a/locales/po/common.inc
    +++ b/locales/po/common.inc
    @@ -24,7 +24,7 @@
     'All Modules'=>'Wszystkie moduły',
     'All Projects'=>'Wszystkie projekty',
     'All Users'=>'Wszyscy użytkownicy',
    -'All w/o in progress'=>'Wszystko poza \"w toku\"',
    +'All w/o in progress'=>'Wszystko poza "w toku"',
     'Allocated'=>'Przydzielone',
     'Allocated hours'=>'Przydzielone godziny',
     'Allowed Companies'=>'Dostępne firmy',
    @@ -372,8 +372,8 @@
     'search'=>'szukaj',
     'send password'=>'wyślij hasło',
     'sendpass0'=>'The user account',
    -'sendpass1'=>'has this email associated with it.nA web user from',
    -'sendpass2'=>'has just requested that a new password be sent.nnYour New Password is:',
    +'sendpass1'=>'has this email associated with it.\nA web user from',
    +'sendpass2'=>'has just requested that a new password be sent.\n\nYour New Password is:',
     'sendpass3'=>'If you didn\'t ask for this, don\'t worry. You are seeing this message, not them. If this was an error just login with your new password and then change your password to what you would like it to be.',
     'sendpass4'=>'New password for',
     'show all'=>'pokaż wszystko',
    diff --git a/locales/po/public.inc b/locales/po/public.inc
    index f1aa34e..7a00a51 100644
    --- a/locales/po/public.inc
    +++ b/locales/po/public.inc
    @@ -17,6 +17,6 @@
     'chgpwUpdated'=>'Twoje hasło zostało zaktualizowane',
     'chgpwWrongPW'=>'Stare hasło jest niepoprawne',
     'colorLegend'=>'Wybierz kolor, by wyróżnić twoje zaznaczenie i zamknij okno.',
    -'nodepartments, companies, permissions'=>'\"nodepartments\", firmy, zezwolenia',
    +'nodepartments, companies, permissions'=>'"nodepartments", firmy, zezwolenia',
     'notasks'=>'Zadania niedostępne',
     'submit'=>'wyślij',
    diff --git a/locales/po/system.inc b/locales/po/system.inc
    index ac6f16d..6624993 100644
    --- a/locales/po/system.inc
    +++ b/locales/po/system.inc
    @@ -25,7 +25,7 @@
     'Bind Password'=>'Powiąż hasło',
     'Checkbox'=>'Checkbox',
     'Could not open locales file to save.'=>'Nie można otworzyć pliku lokalizacji do zapisu.',
    -'Couldnt load the Custom Field, It might have been deleted somehow.'=>'Nie można wczytać \"własnego pola\", mogło zostać usunięte.',
    +'Couldnt load the Custom Field, It might have been deleted somehow.'=>'Nie można wczytać "własnego pola", mogło zostać usunięte.',
     'Currency Format'=>'Format waluty',
     'Custom Field Editor'=>'Edycja własnych pól',
     'Custom Fields - Add/Edit'=>'Własne pola - dodaj/edytuj',
    @@ -199,7 +199,7 @@
     'cal_day_view_show_minical_title'=>'Pokaż minikalendarze w widoku dnia',
     'cal_day_view_show_minical_tooltip'=>'Wyświetl minikalendarz w widoku dnia.',
     'cal_working_days_title'=>'Godziny robocze w kalendarzu',
    -'cal_working_days_tooltip'=>'Ustaw wymagane dni robocze dla twojej firmy jako listę cyfr oddzielonych przecinkami, gdzie 0 to niedziela. Jeśli pracujesz od poniedziałku do piątku, twoja lista to \"1,2,3,4,5\". Jeśli pracujesz od piątku do niedzieli, twoja lista to \"5,6,0\".',
    +'cal_working_days_tooltip'=>'Ustaw wymagane dni robocze dla twojej firmy jako listę cyfr oddzielonych przecinkami, gdzie 0 to niedziela. Jeśli pracujesz od poniedziałku do piątku, twoja lista to "1,2,3,4,5". Jeśli pracujesz od piątku do niedzieli, twoja lista to "5,6,0".',
     'calendar_group_title'=>'Kalendarz',
     'check_overallocation_title'=>'Sprawdzanie zawyżonych przydziałów',
     'check_overallocation_tooltip'=>'Komponent zawyżonych przydziałów dla web3project jest niekompletną funkcją, która wciąż jest rozwijana. To baza systemu, który pozwoli menadżerom projektu ustawić maksymalny procent czasu, dla któego zasób może zostać przydzielony do pojedynczego zadania dla określonego okresu czasu.',
    @@ -286,7 +286,7 @@
     'ldap_version_title'=>'Wersja LDAP',
     'ldap_version_tooltip'=>'Wersja LDAP, która hosty LDAP rozumieją. Jeśli masz wątpliwości, zostaw domyślne 3.',
     'link_tickets_kludge_title'=>'Zezwól na relinkowanie biletów',
    -'link_tickets_kludge_tooltip'=>'Włączenie tej opcji spowoduje relinkowanie biletów do bezwzględnego rodzica w \"biletowni\". Może okazać się użyteczne przy biletach generowanych e-mailem.',
    +'link_tickets_kludge_tooltip'=>'Włączenie tej opcji spowoduje relinkowanie biletów do bezwzględnego rodzica w "biletowni". Może okazać się użyteczne przy biletach generowanych e-mailem.',
     'locale_alert_title'=>'Łańcuch znaków alarmu tłumaczenia',
     'locale_alert_tooltip'=>'Łańcuch będzie dodawany do nieprzetłumaczonych wyrażeń lub nieodnalezionych kluczy, jeśli Ostrzeżenia tłumaczeń są włączone.',
     'locale_warn_title'=>'Ostrzeżenia tłumaczeń.',
    
    patch file icon issue603_inc_files_fix.patch (4,685 bytes) 1969-12-31 16:00 +
  • patch file icon issue603.patch (4,201 bytes) 1969-12-31 16:00 -
    diff --git a/classes/ui.class.php b/classes/ui.class.php
    index cb35e9a..cd91270 100644
    --- a/classes/ui.class.php
    +++ b/classes/ui.class.php
    @@ -556,13 +556,15 @@ class CAppUI {
     
     		switch ($flags & UI_OUTPUT_MASK) {
     			case UI_OUTPUT_HTML:
    -				$str = htmlspecialchars(stripslashes($str), ENT_COMPAT, $locale_char_set);
    +				$str = htmlspecialchars($str, ENT_COMPAT, $locale_char_set);
    +				$str = str_replace('\n', '<br />', $str);
     				break;
     			case UI_OUTPUT_JS:
    -				$str = addslashes(stripslashes($str)); //, ENT_COMPAT, $locale_char_set);
    +				$str = str_replace('\n', "\n", $str);
    +				$str = addslashes($str); //, ENT_COMPAT, $locale_char_set);
     				break;
     			case UI_OUTPUT_RAW:
    -				$str = stripslashes($str);
    +				$str = str_replace('\n', "\n", $str);
     				break;
     		}
     		return $str;
    @@ -1619,4 +1621,4 @@ class CTitleBlock_core {
     		}
     		return false;
     	}
    -}
    \ No newline at end of file
    +}
    diff --git a/includes/main_functions.php b/includes/main_functions.php
    index fdef244..906fb38 100644
    --- a/includes/main_functions.php
    +++ b/includes/main_functions.php
    @@ -1386,4 +1386,28 @@ function w2p_textarea($content)
       }
     
       return $result;
    -}
    \ No newline at end of file
    +}
    +
    +/**
    + * Returns single quoted string representation of $str. 
    + * Usefull for writing config files with content to be evaluated as php code. 
    + * @param string $str String to export
    + * @param boolean $encapsulate Whether encapsulate the returned string with apostrophes, defaults to true.
    + * @return string
    + */
    +function w2p_export_single_quoted_string($str, $encapsulate = true) {
    +    $patterns = array(
    +               // escape backslashes with special meaning in single quoted strings (\\, \') and
    +               // last character if it is a backslash.
    +       "@(?| ((?:\\\\{2})*) | (\\\\)(') | (\\\\)($))@x",
    +       // escape apostrophes  
    +       "@'@"
    +       );
    +    $replaces = array(
    +       '$1$1$2', 
    +       '\\\\\''
    +    );
    +    $result = preg_replace($patterns, $replaces, $str);
    +    if ($encapsulate) $result = '\''. $result . '\'';
    +    return $result;
    +}
    diff --git a/modules/system/translate.php b/modules/system/translate.php
    index 0dff292..0993b35 100644
    --- a/modules/system/translate.php
    +++ b/modules/system/translate.php
    @@ -108,7 +108,7 @@ foreach ($trans as $k => $langs) {
     		$s .= '&nbsp;';
     	}
     	$s .= '</td><td>';
    -	$langs['english'] = w2PformSafe($langs['english'], true);
    +	$langs['english'] = w2PformSafe($langs['english']);
     	if ($lang == 'en') {
     		if (mb_strlen($langs['english']) < 40) {
     			$s .= '<input type="text" name="trans[' . $index . '][english]" value="' . $langs['english'] . '" size="40" class="text" />';
    @@ -122,7 +122,7 @@ foreach ($trans as $k => $langs) {
     	}
     	$s .= '</td><td>';
     	if ($lang != 'en') {
    -		$langs['lang'] = w2PformSafe($langs['lang'], true);
    +		$langs['lang'] = w2PformSafe($langs['lang']);
     		if (mb_strlen($langs['lang']) < 40) {
     			$s .= '<input type="text" name="trans[' . $index . '][lang]" value="' . $langs['lang'] . '" size="40" class="text" />';
     		} else {
    diff --git a/modules/system/translate_save.php b/modules/system/translate_save.php
    index 249e5f0..da68d35 100644
    --- a/modules/system/translate_save.php
    +++ b/modules/system/translate_save.php
    @@ -60,7 +60,7 @@ if ($lang == 'en') {
     	foreach ($trans as $langs) {
     		if (($langs['abbrev'] || $langs['english']) && empty($langs['del'])) {
     			$langs['abbrev'] = addslashes(stripslashes($langs['abbrev']));
    -			$langs['english'] = addslashes(stripslashes($langs['english']));
    +			$langs['english'] = w2p_export_single_quoted_string($langs['english'], false);
     			if (!empty($langs['abbrev'])) {
     				$txt .= '\'' . $langs['abbrev'] . '\'=>';
     			}
    @@ -71,8 +71,8 @@ if ($lang == 'en') {
     	// editing the translation
     	foreach ($trans as $langs) {
     		if (empty($langs['del'])) {
    -			$langs['english'] = addslashes(stripslashes($langs['english']));
    -			$langs['lang'] = addslashes(stripslashes($langs['lang']));
    +			$langs['english'] = w2p_export_single_quoted_string($langs['english'], false);
    +			$langs['lang'] = w2p_export_single_quoted_string($langs['lang'], false);
     			$txt .= '\'' . $langs['english'] . '\'=>\'' . $langs['lang'] . '\',' . "\n";
     		}
     	}
    
    patch file icon issue603.patch (4,201 bytes) 1969-12-31 16:00 +

-Relationships
+Relationships

-Notes

~0001313

gajdusek (reporter)

Last edited: 2010-10-19 18:50

If you save translation all strings are passed through addslashes(stripslashes()) function. This lead to (1) inconsistency between manual and web-based edited localization files and (2) "\n" becomes "n".

i.e string
'Some "quoted string" and \nnew line'
from localization file will become
'Some \"quoted string\" and nnew line'
if you save it in web-based translation manager.

Whenever localization files are included and evaluated stripslashes() is applied so backslashes are removed and the string will become
'Some "quoted string" and nnew line' for both cases. The \n becomes pure n in both cases.

I see solution in escaping only sequences with special meaning in single quoted strings when saving translation changes and do not apply stripslashes() when loading them.

This can be the function used for escaping strings when saving them:

<code>
function export_single_quoted_string($str, $return = false) {
    $patterns = array(
        // escape backslashes of special meaning in single quoted strings (\\, \') and
        // last character if it is a backslash.
        "@(?| ((?:\\\\{2})*) | (\\\\)(') | (\\\\)($))@x",
        // escape apostrophes
        "@'@"
        );
    $replaces = array(
        '$1$1$2',
        '\\\\\''
    );
    $result = "'" . preg_replace($patterns, $replaces, $str) . "'";
    if ($return) return $result;
    print ($result);
}
</code>

Next step will be repair already damaged localization files, I found only polish localization affected, but did not make sure others are not.

And finally don't strip slashes while loading localization files. Maybe check for magic_quotes_gpc and magic_quotes_runtime. But these should be off because of .htaccess and because they are deprecated.

Petr

~0001351

gajdusek (reporter)

Please see https://github.com/gajdusek/web2project/commits/issue603 with my patch fixing this.

Petr

~0001352

gajdusek (reporter)

Last edited: 2010-11-18 00:53

Added patches that applies cleanly against SVN trunk (commit 1489).

issue603_inc_files_fix.patch fixes Polish locales: some strings were wrong escaped as result of using web2project translation manager suffering this bug

issue603.patch fixes the bug itself:

Fixed escaping of (single quoted) l10n strings in the web2project code
    
    This fixes http://bugs.web2project.net/view.php?id=603. Though, already
    existed errors in .inc files must be fixed first (previous commit
    does it).
    
    Details:
    
    Translation manager:
    * Escape only characters that must be escaped in single quoted strings
    before writing translation strings out to the .inc files.
    * Do not deslash translation strings while loading .inc files to
    the translation manager.
    
    ui.class.php:
    * Do not deslash translated strings returned by _() and __() methods
    of CAppUI class and replace '\n' with "\n" (UI_OUTPUT_RAW and UI_OUTPUT_JS
    flags) or with '&<br /&>' (UI_OUTPUT_HTML).

~0001686

caseydk (administrator)

Can you file a pull request on 3453548?

~0001688

caseydk (administrator)

Actually, after examining this one a more closely, the issue is a little different.

If you insert a linebreak via the translation interface in w2p, it preserves the linebreaks you've inserted. It loses the \n that are included in single quotes because the slashes are stripped as it comes in (as you note).

It looks like we have a trio of options here:
- continue using single quotes in translation files and insert real (not escaped) breaks as needed;
- use double quotes and use either escaped or real linebreaks as desired.
- use your solution to catch mis-escaped linebreaks and transform as needed.

Your solution - although it would work - seems like we're patching over something that is obviously a mistake and a problem.. a cleaner solution seems like the second option with properly escaped linebreaks.

Thoughts?

~0001732

caseydk (administrator)

Fixed a bunch of places where newlines in translation strings were breaking the output;
Resolved in r1754, will be in v2.3 release;

~0001774

caseydk (administrator)

Closed in preparation for v2.3 release.
+Notes

-Issue History
Date Modified Username Field Change
2010-10-14 04:52 gajdusek New Issue
2010-10-19 13:57 gajdusek Note Added: 0001313
2010-10-19 13:58 gajdusek Note Edited: 0001313
2010-10-19 18:50 gajdusek Note Edited: 0001313
2010-11-18 00:27 gajdusek Note Added: 0001351
2010-11-18 00:42 gajdusek File Added: issue603_inc_files_fix.patch
2010-11-18 00:43 gajdusek File Added: issue603.patch
2010-11-18 00:50 gajdusek Note Added: 0001352
2010-11-18 00:52 gajdusek Note Edited: 0001352
2010-11-18 00:53 gajdusek Note Edited: 0001352
2010-12-18 21:20 caseydk Project v2.2 Release (Closed) => v2.3 Release (Closed)
2011-02-26 22:48 caseydk Note Added: 0001686
2011-02-26 23:01 caseydk Note Added: 0001688
2011-03-20 17:05 caseydk Note Added: 0001732
2011-03-20 17:05 caseydk Status new => resolved
2011-03-20 17:05 caseydk Resolution open => fixed
2011-03-20 17:05 caseydk Assigned To => caseydk
2011-03-24 09:51 caseydk Note Added: 0001774
2011-03-24 09:51 caseydk Status resolved => closed
2011-03-24 09:51 caseydk Fixed in Version => 2.3
+Issue History