Home

Awesome

README file for PyICU

Welcome

Welcome to PyICU, a Python extension wrapping the ICU C++ libraries.

ICU stands for "International Components for Unicode". These are the i18n libraries of the Unicode Consortium. They implement much of the Unicode Standard, many of its companion Unicode Technical Standards, and much of Unicode CLDR.

The PyICU source code is hosted at https://gitlab.pyicu.org/main/pyicu.

The ICU homepage is http://site.icu-project.org/

See also the CLDR homepage at http://cldr.unicode.org/

Installing PyICU

PyICU is a python extension implemented in C++ that wraps the C/C++ ICU library. It is known to also work as a PyPy extension. Unless pkg-config and the ICU libraries and headers are already installed, building PyICU from the sources on PyPI involves more than just a pip call. Many operating systems distribute pre-built binary packages of ICU and PyICU, see below.

Building PyICU

Before building PyICU the ICU libraries must be built and installed. Refer to each system's instructions for more information.

PyICU is built with setuptools:

Running PyICU

What's available

See the CHANGES file for an up to date log of changes and additions.

API Documentation

There is no API documentation for PyICU. The API for ICU is documented at https://unicode-org.github.io/icu-docs/apidoc/released/icu4c/ and the following patterns can be used to translate from the C++ APIs to the corresponding Python APIs.

strings

The ICU string type, UnicodeString, is a type pointing at a mutable array of UChar Unicode 16-bit wide characters and is described here. The Python 3 str type is described here and here. The Python 2 unicode type is described here.

Because of their differences, ICU's and Python's string objects are not merged into the same type when crossing the C++ boundary but converted.

ICU APIs taking UnicodeString arguments have been overloaded to also accept arguments that are Python 3 str or Python 2 unicode objects. Python 2 str objects are auto-decoded into ICU strings using the utf-8 encoding.

To convert a Python 3 bytes or a Python 2 str object encoded in an encoding other than utf-8 to an ICU UnicodeString use the UnicodeString(str, encodingName) constructor.

ICU's C++ APIs accept and return UnicodeString arguments in several ways: by value, by pointer or by reference. When an ICU C++ API is documented to accept a UnicodeString reference parameter, it is safe to assume that there are several corresponding PyICU python APIs making it accessible in simpler ways:

For example, the 'UnicodeString &Locale::getDisplayName(UnicodeString &)' API, documented here, can be invoked from Python in several ways:

  1. The ICU way

     >>> from icu import UnicodeString, Locale
 >>> locale = Locale('pt_BR')
 >>> string = UnicodeString()
 >>> name = locale.getDisplayName(string)
 >>> name
 <UnicodeString: 'Portuguese (Brazil)'>
 >>> name is string
 True                  <-- string arg was returned, modified in place

  2. The Python way

     >>> from icu import Locale
 >>> locale = Locale('pt_BR')
 >>> name = locale.getDisplayName()
 >>> name
 'Portuguese (Brazil)'

    A UnicodeString object was allocated and converted to a Python str object.

A UnicodeString can be converted to a Python unicode string with Python 3's str() or Python 2's unicode() constructor. The usual len(), comparison, `[]and[:]operators are all available, with the additional twists that slicing is not read-only and that+=`` is also available since a UnicodeString is mutable. For example:

>>> name = locale.getDisplayName()
'Portuguese (Brazil)'
>>> name = UnicodeString(name)
>>> name
<UnicodeString: 'Portuguese (Brazil)'>
>>> str(name)
'Portuguese (Brazil)'
>>> len(name)
19
>>> str(name)
'Portuguese (Brazil)'
>>> name[3]
't'
>>> name[12:18]
<UnicodeString: 'Brazil'>
>>> name[12:18] = 'the country of Brasil'
>>> name
<UnicodeString: 'Portuguese (the country of Brasil)'>
>>> name += ' oh joy'
>>> name
<UnicodeString: 'Portuguese (the country of Brasil) oh joy'>

error reporting

The C++ ICU library does not use C++ exceptions to report errors. ICU C++ APIs return errors via a UErrorCode reference argument. All such APIs are wrapped by Python APIs that omit this argument and throw an ICUError Python exception instead. The same is true for ICU APIs taking both a ParseError and a UErrorCode, they are both to be omitted.

For example, the 'UnicodeString &DateFormat::format(const Formattable &, UnicodeString &, FieldPosition &, UErrorCode &)' API, documented here is invoked from Python with:

>>> from icu import DateFormat, Formattable
>>> df = DateFormat.createInstance()
>>> df
<SimpleDateFormat: M/d/yy h:mm a>
>>> f = Formattable(940284258.0, Formattable.kIsDate)
>>> df.format(f)
'10/18/99 3:04 PM'

Of course, the simpler 'UnicodeString &DateFormat::format(UDate, UnicodeString &)' documented here can be used too:

>>> from icu import DateFormat
>>> df = DateFormat.createInstance()
>>> df
<SimpleDateFormat: M/d/yy h:mm a>
>>> df.format(940284258.0)
'10/18/99 3:04 PM'

dates

ICU uses a double floating point type called UDate that represents the number of milliseconds elapsed since 1970-jan-01 UTC for dates.

In Python, the value returned by the time module's time() function is the number of seconds since 1970-jan-01 UTC. Because of this difference, floating point values are multiplied by 1000 when passed to APIs taking UDate and divided by 1000 when returned as UDate.

Python's datetime objects, with or without timezone information, can also be used with APIs taking UDate arguments. The datetime objects get converted to UDate when crossing into the C++ layer.

arrays

Many ICU API take array arguments. A list of elements of the array element types is to be passed from Python.

StringEnumeration

An ICU StringEnumeration has three next methods: next() which returns str objects, unext() which returns str objects in Python 3 or unicode objects in Python 2 and snext() which returns UnicodeString objects. Any of these methods can be used as an iterator, using the Python built-in iter function.

For example, let e be a StringEnumeration instance:

e = TimeZone.createEnumeration()
[s for s in e] # a list of 'str' objects
[s for s in iter(e.unext, '')] # a list of 'str' or 'unicode' objects
[s for s in iter(e.snext, '')] # a list of 'UnicodeString' objects

timezones

The ICU TimeZone type may be wrapped with an ICUtzinfo type for usage with Python's datetime type. For example:

from datetime import datetime
tz = ICUtzinfo(TimeZone.createTimeZone('US/Mountain'))
datetime.now(tz)

or, even simpler:

tz = ICUtzinfo.getInstance('Pacific/Fiji')
datetime.now(tz)

To get the default time zone use:

defaultTZ = ICUtzinfo.getDefault()

To get the time zone's id, use the tzid attribute or coerce the time zone to a string:

ICUtzinfo.getInstance('Pacific/Fiji').tzid -> 'Pacific/Fiji'
str(ICUtzinfo.getInstance('Pacific/Fiji')) -> 'Pacific/Fiji'