Home
About
Search
🌐
English Română
  • PHP and Unicode using UTF-8

    Citește postarea în română

    Aug 11, 2009 PHP php6 unicode utf-8
    Share on:

    One of the biggest issues with the web is encoding.

    In the old days the formerly base standard was  ISO 8859-1, where there ware 191 latin characters defined, and 1 char = 1B. For different languages, different encodings ware used, but from here many portability issues appeared, the possibility to cover a greater number of languages etc.

    The problem occurs when a project should be available in several languages, and the number of the languages is not controlled. A big project like WordPress for example should be available with any language.

    Unicode is a better alternative for ISO 8859-1, having more then 100.000 characters defined. In other words it has about every character of about any existing language.

    As I was saying for MySQL, UTF-8 characters have a variable length between 1 and 4B.

    Displaying the UTF-8 content in PHP pages

    For browser to interpret the page content as UTF-8, it should receive the right headers:

    1<?php header("Content-type: text/html; charset=utf-8");?>
    

    Attention! The header should be the first thing that is send from the server! In other words it should be the first thing displayed on the page.

    The type of the document can be specified with the “Content-Type” meta tag. If there is a similar meta tag on the page it should be removed and replace with:

    1<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
    

    The .htaccess file and string processing

    Add to the .htaccess file (for Apache servers) the following lines:

    1# default charset used by PHP
    2php_value default_charset utf-8
    3# encoding for mbstring
    4php_value mbstring.internal_encoding utf-8
    5php_value mbstring.func_overload 7
    

    The first line sets the default charset for PHP, this setting can be made directly to php.ini.

    Second and third line sets the mbstring (multi byte string) functions.

    Using UTF-8, as I was saying earlier, 1 char != 1B, so errors may appear:

    1$var = 'aşadar';
    2
    3echo strlen($var).PHP_EOL; // 7
    4echo strtoupper($var).PHP_EOL; // AşADAR
    5
    6// using mbstring functions
    7echo mb_strlen($var).PHP_EOL; // 6
    8echo mb_strtoupper($var).PHP_EOL; // AŞADAR
    

    This is why we set the mbstring functions mode using the .htaccess file. Content entered through forms should be processed using mbstring functions, to avoid problems like in the earlier example.

    The available functions are in the manual.

    Coding old content

    There are many ways to encode ISO 8859-1 content to UTF-8. A couple of ways of doing that with PHP are:

    – iconv() function which converts from a format to another specified format:

    1echo iconv("ISO-8859-1", "UTF-8", "Test");
    

    – utf8_encode() function which converts from ISO 8859-1 to UTF-8:

    1echo utf8_encode("Test");
    

    What does the future bring?

    The long-expected PHP6 will have native support for Unicode, so all the above tricks will be unnecessary. At the moment of writing this blog PHP 6 is 70.70% done, and with a little luck it will be ready in less then an year.

  • MySQL and Unicode using UTF-8

    Citește postarea în română

    Aug 10, 2009 information_schema MySQL unicode utf-8
    Share on:

    Along with globalization, the old ASCII code is no longer suitable.  Consider that one day you have to develop a project in German, Russian or even Japanese, you could adapt the charset for each of these languages or you could simply develop using Unicode.

    To use Unicode with MySQL UTF-8 can be used.

    You must note that UTF-8 characters are variable in length and they are ASCII compatible. In ASCII 1 char = 1B, in UTF-8 1 char can be between 1 and 4 B.

    UTF-8 charset and collation on the server

    Character type in MySQL is dictated by charset.

    To check if UTF-8 in installed on the server:

    1SHOW CHARSET LIKE 'utf8';
    

    or with information_schema

    1SELECT * FROM `CHARACTER_SETS` WHERE CHARACTER_SET_NAME = 'utf8';
    

    If the charset was found then we can continue.

    Another element that appears with charset is collation, which it’s used for comparing strings at ordering.

    To see what collations are available on the server:

    1SHOW COLLATION WHERE CHARSET = 'utf8';
    

    or with information_schema

    1SELECT * FROM `COLLATIONS` WHERE CHARACTER_SET_NAME = 'utf8';
    

    Collation are usually by language, for comparing strings with or without diacritics for example, or “bin” can be used with orders strings in binary mode,  ie “A” is greater than “a” for example.

    If no collation is specified, then the default one will be used.

    UTF-8 and data bases

    When creating a database you can specify the default charset to be used with all new tables for which there isn’t any charset specified.

    For example:

    1CREATE DATABASE db_name CHARACTER SET utf8 COLLATE utf8_romanian_ci;
    

    Or for modifying the default charset for a data base which already exists:

    1ALTER DATABASE db_name CHARACTER SET utf8 COLLATE utf8_romanian_ci;
    

    UTF-8, tables and columns

    For modifying tables which already exist ALTER TABLE must be used is used.

    A table can have a default charset and collation, and each column can have it’s own charset and collation.

    For more information about the table:

    1SHOW CREATE TABLE tab;
    

    To set a charset for an existing table:

    1ALTER TABLE tab CHARSET = utf8 COLLATE = utf8_romanian_ci;
    

    For modifying the charset of a VARCHAR(200) column is used:

    1ALTER TABLE tab MODIFY c1 VARCHAR(200) CHARSET utf8 COLLATE utf8_romanian_ci;
    

    String size

    A “problem” that may arise is related to the size of the character, it’s size can be between 1 and 4B.  That is why for measuring a string column (like varchar) you must use CHAR_LENGTH(str) instead of LENGTH().

    A short exemple:

    1SET @var = 'aşadar';
    2SELECT CHAR_LENGTH(@var) AS 'Char', LENGTH(@var) AS 'Length';
    3
    4// The output is: Char = 6 and Length = 7 because ş is 2B
    

Claudiu Perșoiu

Programming, technology and more
Read More

Recent Posts

  • Adding a slider to Tasmota using BerryScript
  • The future proof project
  • Docker inside wsl2
  • Moving away from Wordpress
  • Custom path for Composer cache
  • Magento2 and the ugly truth
  • A bit of PHP, Go, FFI and holiday spirit
  • How to make use of the Xiaomi Air Conditioning Companion in Home Assistant in only 20 easy steps!

PHP 49 MISCELLANEOUS 46 JAVASCRIPT 14 MAGENTO 7 MYSQL 7 BROWSERS 6 DESIGN PATTERNS 5 HOME AUTOMATION 2 LINUX-UNIX 2 WEB STUFF 2 GO 1

PHP 35 JAVASCRIPT 15 PHP5.3 11 MAGENTO 7 PHP6 7 MYSQL 6 PHP5.4 6 ZCE 6 CERTIFICARE 5 CERTIFICATION 5 CLOSURES 4 DESIGN PATTERNS 4 HACK 4 ANDROID 3
All tags
3D1 ADOBE AIR2 ANDROID3 ANGULAR1 ANONYMOUS FUNCTIONS3 BERRYSCRIPT1 BOOK1 BROWSER2 CARTE1 CERTIFICARE5 CERTIFICATION5 CERTIFIED1 CERTIFIED DEVELOPER1 CHALLENGE1 CHM1 CLASS1 CLI2 CLOSURES4 CODE QUALITY1 CODEIGNITER3 COFFEESCRIPT1 COLLECTIONS1 COMPOSER1 CSS1 DEBUG1 DESIGN PATTERNS4 DEVELOPER1 DEVELOPMENT TIME1 DOCKER2 DOCKER-COMPOSE1 DOUGLAS CROCKFORD2 ELEPHPANT2 FACEBOOK2 FFI1 FINALLY1 FIREFOX3 GAMES1 GENERATOR1 GO1 GOOGLE1 GOOGLE CHROME1 GOOGLE MAPS1 HACK4 HOMEASSISTANT2 HTML2 HTML HELP WORKSHOP1 HTML51 HUG1 HUGO1 INFORMATION_SCHEMA1 INI1 INTERNET EXPLORER3 IPV41 IPV61 ITERATOR2 JAVASCRIPT15 JQUERY1 LAMBDA1 LINUX1 MAGENTO7 MAGENTO22 MAP1 MINESWEEPER1 MOTIVATION1 MYSQL6 NGINX1 NODE.JS2 NOSQL1 OBSERVER3 OBSERVER PATTERN1 OOP1 OPERA1 OPTIMIZATION1 ORACLE1 PAGESPEED1 PAIR1 PARSE_INI_FILE1 PHONEGAP2 PHP35 PHP ELEPHANT2 PHP FOR ANDROID1 PHP-GTK1 PHP42 PHP53 PHP5.311 PHP5.46 PHP5.53 PHP5.61 PHP67 PHP7.41 PROGRAMMING1 REVIEW1 ROMANIAN STEMMER2 SAFARY1 SCALAR TYPE HINTING1 SCHEME1 SET1 SHOPPING CART PRICE RULE1 SINGLETON1 SOAP1 SPL2 SQLITE1 SSH1 STACK TRACE1 STDERR1 STDIN1 STDOUT1 SUN1 SYMFONY2 TASMOTA1 TEST TO SPEECH1 TITANIUM2 TRAITS1 TTS1 UBUNTU1 UNICODE2 UTF-82 VECTOR1 WEBKIT1 WINBINDER1 WINDOWS2 WORDPRESS1 WSL21 YAHOO3 YAHOO MAPS1 YAHOO OPEN HACK1 YSLOW1 YUI1 ZCE6 ZCE5.31 ZEND3 ZEND FRAMEWORK3
[A~Z][0~9]

Copyright © 2008 - 2024 CLAUDIU PERȘOIU'S BLOG. All Rights Reserved