Actual language characters, natural form, are loaded in the url regardless of whether the option is set to true or false.
The issue is with the mechanism of processing/loading natural form by core. I remember seeing something to do with encoding/decoding category slugs when I was trying to troubleshoot the language in URL bug (beta feature). Perhaps that function has a bug somewhere. I presume this new feature was developed to support natural lang form in links used in email templates so people don't see ugly (encoded) urls.
Personally, I view the encoded urls issue as being less important than the language url beta feature which enables proper static lang urls for multi-language platforms, including proper canonicalization, and that is the standard for SEO/search engines. Nevertheless, both are a step forward in the right direction. Add a recommended url structure ie. no commas, uppercase letters, underscores etc. (as advised by the head of search at Google as well as many SEO experts) and we've got a fully viable solution. Any other approach is considered a patch / workaround and a deviation from best practices.