# RFC: Multibyte Char Handling

- Status: Declined
- Authors: Yasuo Ohgaki, yohgaki@ohgaki.net
- Date: 2014-02-03
- Version: 1.3
- Original: https://wiki.php.net/rfc/multibyte_char_handling
- Page: https://externals.io/rfc/multibyte_char_handling
- More for agents: search with https://externals.io/search.md?q={query} · RFCs at https://externals.io/rfc.md · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt

## Votes

### Add required mb_*() functions to fix vulnerability (closed)

Yes: 2 · No: 10

Voters: aharvey (No), brianlmoon (Yes), chobieeee (No), derick (No), klaussilveira (No), krakjoe (No), levim (No), mbeccati (No), nikic (No), peehaa (No), treffynnon (No), yohgaki (Yes)

## Discussions on the mailing list

- [\[VOTE\] RFC: Multibyte Char Handling](https://externals.io/message/71572.md) — started on 2014-01-26, 20 messages
- [\[VOTE\] Multbye char handling - Remove vulnerability related to multibyte short and long term](https://externals.io/message/72427.md) — started on 2014-02-10, 11 messages
- [Resolution for ver_export()/addslashes() encoding based script execution attack?](https://externals.io/message/72788.md) — started on 2014-02-24, 20 messages

---

## Introduction

For example, addslashes()/stripslashes() has vulnerability that allows PHP script execution and DoS under certain environment via char encoding based attack. Counter measure using php\_mblen() is proposed at first in security ML. However, php\_mblen() is locale dependent and unreliable. Therefore, this is proposed and created RFC to decide.

Proper and secure char encoding handling is mandatory for application to work correctly. However, current PHP lacks default multibyte char handling feature. As a result, escapeshellarg()/escapeshellcmd()/fgetcsv() rely on locale based multibyte char handling for security and proper operation. Currently, addslahses()/ver\_export()/stripslashes() lacks proper multibyte char handling.

escapeshellarg()/escapeshellcmd()/fgetcsv() uses locale based multibyte char handling. i.e. php\_mblem() Although this approach works mostly, it has several disadvantages.

- Locale is not reliable and has issues on some locale, such as Turkish locale.
- Locale is not visible for application programmers. Most developers just ignores locale.

New mbstring functions are added for users that are affected by this issue so that they can simply rename functions.

## Proposal

PHP, including released versions, needs secure addslashes()/var\_export()/stripslashes().

### Add mb\_addslashes()/mb\_var\_export()/mb\_stripslashes() to released versions

For PHP 5.4 and up, add mb\_add\_slashes()/mb\_var\_export()/mb\_strip\_slashes() has encoding option.

```php
  string mb_add_slashes(string $str [, string $encoding=internal_encoding])
  string mb_strip_slashes(string $str [, $encoding=internal_encoding])
  string mb_add_cslashes(string $str [, string $encoding=internal_encoding])
  string mb_strip_cslashes(string $str [, $encoding=internal_encoding])
  string mb_var_export(mixed $var [, bool $return=FALSE [, string $encoding=internal_encoding]])
```

addcslashes()/stripcslashs() needs to be multibyte aware for the same reason addslashes()/stripslashes().

### Add mb version of function uses php\_mblen()

For PHP 5.4 and up, add mb\_escape\_shell\_arg()/mb\_secape\_shell\_cmd()/mb\_fget\_csv()/etc that have extra encoding parameter like mb\_add\_slashes().

- mb\_escape\_shell\_arg()
- mb\_escape\_shsell\_cmd()
- functions/methods use [php\_fgetcsv API](http://lxr.php.net/search?q=php_fgetcsv&project=PHP_5_5)
    - mb\_file\_get\_csv()
    - mb\_file\_put\_csv()

Reference

- [http://lxr.php.net/search?q=php\_mblen&amp;defs=&amp;refs=&amp;path=&amp;hist=&amp;project=PHP\_5\_5](http://lxr.php.net/search?q=php_mblen&defs=&refs=&path=&hist=&project=PHP_5_5)

We still needs php\_mblen() for command line output since it's not good idea to detect locale and using internal\_encoding opens new vulnerability.

Functions that should use locale are

- mb\_escape\_shell\_arg()
- mb\_escape\_shell\_cmd()

These function may override locale by encoding parameter. Since fgetcsv() uses locale now, do the same for fgetcsv().

### mbstring.func\_overload

Some users are annoyed by sloppy multilingual implementations using this option. There is feature request from user who want to remove mbstring.func\_overload INI option.

<https://bugs.php.net/bug.php?id=65785>

However, func\_overload is extended for now.

### mbstring usage and implementation

For PHP 5.4 and up, all changes done in mbstring.

mbstring functions have history of remain insecure when single byte version of function's issue has been fixed. e.g. mb\_prase\_str(), mb\_send\_mail() Refactoring is preffered to avoid this issue, but refactoring is postponed until PHP6. i.e. There would be 2 codes that are mostly the same.

In short, if some one fixes related function, do not forget to update mbstring code also.

## Note about short and long term resolution

- **Short term resolution**: Add required function to mbstring
- **Long term resolution**: Replace mbstring with mbstring-ng to provide multibyte aware functions by default. mbstring-ng does not have license issue.

This RFC is for short term resolution.

Main objective is to remove vulnerability like CVE-2014-1239. To accomplish this objective, we need multibyte aware function by default which we don't have it now.

To remove vulnerability like CVE-2014-129 from user scripts, there must be multibyte aware functions by default. We may compile current mbstring by default, but there is license issue for some users. mbstring-ng does not have such issue and it is preferred to use it as default with respect to license, but it's far from complete.

Since there is no feasible option right now, short and long term resolution is needed.

## Backward Incompatible Changes

None. (Adding functions to mbstring)

## Proposed PHP Version(s)

- PHP 5.4 and up - Introduce additional mb\_\*() functions

## Future Scope

- mbstring may be replaced by mbstring-ng in future release and mbstring may be moved to PECL.

There is other RFC for introducing mbstring-ng as a EXPERIMENTAL module.

When mbstring-ng development is finished, there will be a vote whether mbstring is replaced by mbstring-ng or not.

## Open Issues

## Proposed Voting Choices

Yes/No

## Patches and Tests

- Prepared for review after vote.

## Vote

VOTE: 2014/02/10 - 2014/02/17

This vote is only for adding new mb\_\*() functions to released versions.

**Vote: Add required mb\_\*() functions to fix vulnerability** — Yes: 2 · No: 10 (closed)

Thank you for voting.

**If you vote No for this, please provide alternative short term resolution for CVE-2014-1239.**

## Implementation

After the project is implemented, this section should contain

1. the version(s) it was merged to
2. a link to the git commit(s)
3. a link to the PHP manual entry for the feature

## References

Discussions

- [http://marc.info/?l=php-internals&amp;m=138982990932300&amp;w=2](http://marc.info/?l=php-internals&m=138982990932300&w=2)
- [http://marc.info/?l=php-internals&amp;m=139069591127118&amp;w=2](http://marc.info/?l=php-internals&m=139069591127118&w=2)
- [http://marc.info/?l=php-internals&amp;m=139018752615166&amp;w=2](http://marc.info/?l=php-internals&m=139018752615166&w=2)

Related RFC

- [https://wiki.php.net/rfc/default\_encoding](https://externals.io/rfc/default_encoding)
- <https://externals.io/rfc/altmbstring>

CVE

- CVE is assigned for addslashes issue. CVE-2014-1239

## Rejected Features

Keep this updated with features that were discussed on the mail lists.
