Page 1 of 2

How can I save Unicode characters in a .txt file?

Posted: April 9th, 2015, 10:24 am
by Stephen Hughes
The .txt file editor changes this:
Polybius 5.1.1
τὸ μὲν οὖν κατὰ τὴν Ἀράτου τοῦ νεωτέρου στρατηγίαν ἔτος ἐτύγχανε διεληλυθὸς περὶ τὴν τῆς Πλειάδος ἐπιτολήν: οὕτως γὰρ ἦγε τοὺς χρόνους τότε τὸ τῶν Ἀχαιῶν ἔθνος.

διέρχεσθαι pass, elapse
ἄγειν reckon
to
Polybius 5.1.1
¦Ó? ¦Ì?¦Í ¦Ï?¦Í ¦Ê¦Á¦Ó? ¦Ó?¦Í ?¦Ñ?¦Ó¦Ï¦Ô ¦Ó¦Ï? ¦Í¦Å¦Ø¦Ó?¦Ñ¦Ï¦Ô ¦Ò¦Ó¦Ñ¦Á¦Ó¦Ç¦Ã?¦Á¦Í ?¦Ó¦Ï? ?¦Ó?¦Ã¦Ö¦Á¦Í¦Å ¦Ä¦É¦Å¦Ë¦Ç¦Ë¦Ô¦È?? ¦Ð¦Å¦Ñ? ¦Ó?¦Í ¦Ó?? ¦°¦Ë¦Å¦É?¦Ä¦Ï? ?¦Ð¦É¦Ó¦Ï¦Ë?¦Í: ¦Ï?¦Ó¦Ø? ¦Ã?¦Ñ ?¦Ã¦Å ¦Ó¦Ï?? ¦Ö¦Ñ?¦Í¦Ï¦Ô? ¦Ó?¦Ó¦Å ¦Ó? ¦Ó?¦Í ?¦Ö¦Á¦É?¦Í ?¦È¦Í¦Ï?.
when it is saved, uploaded, downloaded and displayed.

Is there something simple that I am not doing?

Re: How can I save Unicode characters in a .txt file?

Posted: April 9th, 2015, 11:28 am
by Eeli Kaikkonen
Most modern text editors have a way to choose the text encoding. The default may be some other than unicode, for example ISO-8859-something. You have to change it to unicode (usually UTF-8). UTF-8 and UTF-16 are actual low level encodings which implement the unicode standard, which itself is not an encoding.

Re: How can I save Unicode characters in a .txt file?

Posted: April 9th, 2015, 11:34 pm
by Thomas Dolhanty
Stephen Hughes wrote:when it is saved, uploaded, downloaded and displayed.

Is there something simple that I am not doing?
Uploaded to ... downloaded from ... and displayed ... by what? That is, does it look right in your editor before you upload it? It is a bit nebulous, the way you describe it, where the text is getting jumbled.

Re: How can I save Unicode characters in a .txt file?

Posted: April 10th, 2015, 12:34 am
by Stephen Hughes
Thomas Dolhanty wrote:
Stephen Hughes wrote:when it is saved, uploaded, downloaded and displayed.

Is there something simple that I am not doing?
Uploaded to ... downloaded from ... and displayed ... by what? That is, does it look right in your editor before you upload it? It is a bit nebulous, the way you describe it, where the text is getting jumbled.
In the folder, accompanying the sound files.

I'm not sure at which of the steps it turns to gibberish.

Re: How can I save Unicode characters in a .txt file?

Posted: April 10th, 2015, 12:47 am
by Louis L Sorenson
When you choose to save a file, say in an application like Windows Notepad. You will see three different fields:

(1) File Name
(2) Save as Type: txt (text); all files
(3) Encoding: ANSI; Unicode; Unicode Big Endian; UTF-8 (I'm using a MS Windows box)

When you save a file, you will need to set the encoding to UTF-8,. The reason for this is that texts which contain characters which are not Latin characters cannot be represented side by side with Latin characters if only 255 bits are allotted for any given character. (Google ANSI to understand this further). Unicode is a computer memory allocation that allows thousands of characters to be represented on screen. Older non-Unicode fonts used the ANSI (255 bits) set of characters to represent Greek/Hebrew and other character sets on a computer screen. Some of those fonts may have names such as GreekKeys. There are utilities which have been created to convert non-unicode fonts (which were frequently used by Biblical scholars in the 1990's to digitize Greek or Hebrew texts) into Unicode compliant fonts. Since the early 2000's, Unicode fonts have taken over, and almost no one now uses non-unicode fonts for Greek or Hebrew.

But a person who types Unicode text into a notepad file, textfile, or other text (html file), still has to specify if the font they have typed, they need to specify whether the file has to be saved into a 255 bit format or a unicode format (?255x255). If you save a unicode format into an ANSI format, you will get the following:

UNICODE TEXT:
Polybius 5.1.1.1
Τὸ μὲν οὖν κατὰ τὴν Ἀράτου τοῦ νεωτέρου στρατηγίαν ἔτος ἐτύγχανε διεληλυθὸς περὶ τὴν τῆς Πλειάδος ἐπιτολήν· οὕτως γὰρ ἦγε τοὺς χρόνους τότε (τὸ) τῶν Ἀχαιῶν ἔθνος.

If you do not save the file in UTF-8, but choose ANSI, you may get something like
Polybius 5.1.1
¦Ó? ¦Ì?¦Í ¦Ï?¦Í ¦Ê¦Á¦Ó? ¦Ó?¦Í ?¦Ñ?¦Ó¦Ï¦Ô ¦Ó¦Ï? ¦Í¦Å¦Ø¦Ó?¦Ñ¦Ï¦Ô ¦Ò¦Ó¦Ñ¦Á¦Ó¦Ç¦Ã?¦Á¦Í ?¦Ó¦Ï? ?¦Ó?¦Ã¦Ö¦Á¦Í¦Å ¦Ä¦É¦Å¦Ë¦Ç¦Ë¦Ô¦È?? ¦Ð¦Å¦Ñ? ¦Ó?¦Í ¦Ó?? ¦°¦Ë¦Å¦É?¦Ä¦Ï? ?¦Ð¦É¦Ó¦Ï¦Ë?¦Í: ¦Ï?¦Ó¦Ø? ¦Ã?¦Ñ ?¦Ã¦Å ¦Ó¦Ï?? ¦Ö¦Ñ?¦Í¦Ï¦Ô? ¦Ó?¦Ó¦Å ¦Ó? ¦Ó?¦Í ?¦Ö¦Á¦É?¦Í ?¦È¦Í¦Ï?.

So the bottom line: Specify the encoding when you save a text file.

Re: How can I save Unicode characters in a .txt file?

Posted: April 10th, 2015, 1:37 am
by George F Somsel
If you really do mean a text file, it is not possible to save Unicode in a text file since it allows only 128 differing characters (ASCII). You will need to choose some other format than a text file.

Re: How can I save Unicode characters in a .txt file?

Posted: April 10th, 2015, 3:03 am
by Jonathan Robie
Stephen Hughes wrote:I'm not sure at which of the steps it turns to gibberish.
It happens when you save the file. It can apparently display Unicode fine.

What operating system are you on? When you run the text editor, can you choose "help" from the menu and tell us what the name of the text editor program is?

Re: How can I save Unicode characters in a .txt file?

Posted: April 10th, 2015, 3:12 am
by Jonathan Robie
George F Somsel wrote:If you really do mean a text file, it is not possible to save Unicode in a text file since it allows only 128 differing characters (ASCII). You will need to choose some other format than a text file.
A text file can contain Unicode in UTF8, but not all software supports this. I put Unicode in text files all the time. You do want an editor that knows what a byte order mark is ... Stephen, does your editor work correctly for Chinese?

But if you want to make sure everyone can read it, you're better off with a format that declares an encoding, such as HTML or XML. Proprietary word processor formats aren't so good for this.

Re: How can I save Unicode characters in a .txt file?

Posted: April 10th, 2015, 9:09 am
by Thomas Dolhanty
Jonathan Robie wrote:A text file can contain Unicode in UTF8, but not all software supports this. I put Unicode in text files all the time. You do want an editor that knows what a byte order mark is ... Stephen, does your editor work correctly for Chinese?
The simple, but quite elegant - and free - editor that I've used for these types of things for many years is Notepad++. It was designed by programmers for programmers and doesn't replace something like MS Word, but it is great for handling a large range of formatted files.

Re: How can I save Unicode characters in a .txt file?

Posted: April 10th, 2015, 8:50 pm
by Stephen Hughes
Jonathan Robie wrote:What operating system are you on? When you run the text editor, can you choose "help" from the menu and tell us what the name of the text editor program is? ... Stephen, does your editor work correctly for Chinese?
Windows 7 Simplified Chinese. The name of the text editor is 记事本. My Chinese is not good enough for the help to be helpful. It has no problem handling Chinese characters.