Welcome to the VanDyke Software Forums

Join the discussion today!


Go Back   VanDyke Software Forums > Secure Shell

Notices

Reply
 
Thread Tools Display Modes
  #1  
Old 08-16-2007, 11:31 PM
Motley Fool Motley Fool is offline
Registered User
 
Join Date: Aug 2007
Posts: 4
UTF-8: problems with characters and boxes

I'm still new to Unicode and all the various bits that need to be set in order for it to work right, so I may have missed something. I can't figure out what it is though, so any help is appreciated. I'm trying to get UTF-8 working in SecureCRT 5.5.1 which connects to an Ubuntu 6.06 box. Some UTF-8 characters are displayed fine, while others are not. I tested SecureCRT alongside PuTTY - PuTTY seemed to get the display "right" more of the time.

Here's the setup. First, on the Linux side of things:

Code:
> echo $TERM
xterm

> locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
and over in SecureCRT 5.5.1 (Windows XP):
Code:
Options -> Session Options -> Terminal -> Emulation:
  Terminal = Xterm
  ANSI Color = (checked)
  Use color scheme = (not checked)

Options -> Session Options -> Terminal -> Appearance:
  Normal Font = Lucida Console 9pt
    (after clicking "Font" button)  Script = Western
  Character Encoding = UTF (tried OEM as well - it was even worse)
  Use Unicode line-drawing characters = (checked)
and in PuTTY 0.60 (Windows XP):
Code:
Window -> Appearance
  Font = Lucida Console, 10-point
Window -> Translation
  Received data assumed to be... = UTF-8
  Handling of line drawing characters = Use Unicode line drawing code points

Window -> Colours
  Allow terminal to specify ANSI colours = (checked)
I looked at the good ol' UTF-8-demo.txt file in both SecureCRT and PuTTY. I took screenshots in 3 different parts of the file - the Greek anthem, the Russian conference invitation, and the first four box drawing tests at the bottom of the file.

Here's what I hoped to see: screenshot-actual.gif
Here's what I saw in PuTTY: screenshot-putty.gif
Here's what I saw in SecureCRT: screenshot-securecrt.gif

Now, getting a tip from this thread, I tried going into SecureCRT's Options -> Session Options -> Terminal -> Appearance window, clicking the "Font" button, and changing "Script" from "Western" to something else. When I set it to "Greek", the Greek anthem looked fine. When I set it to "Cyrillic", the Russian invitation came out fine. However, when the "Script" was set to "Greek" or "Cyrillic", it shot the box rendering all to pieces. Look here: bad-boxes.gif

So it seems like there are two problems:
  1. Incorrect UTF-8 character rendering (which I assume Sinpi accurately described in the thread I mentioned...?)
  2. Incorrect box rendering in any Script (whether Western, Greek or Cyrillic) that doesn't appear to be problem in PuTTY, even using the same Lucida Console font. Even in its best mode (i.e. in Western Script), SecureCRT seems to be using the "poor man's" box rendering method of pluses, hyphens and pipes rather than actual Unicode box characters. (I should note that it made no difference whether the "Use Unicode line-drawing characters" SecureCRT setting was checked or not - the boxes are drawn the same way.)

So, is there a problem with my setup, or is there a problem with SecureCRT 5.5.1?

Thanks to anyone who can help!
Reply With Quote
  #2  
Old 08-17-2007, 09:31 AM
Maureen's Avatar
Maureen Maureen is offline
VanDyke Product Director
 
Join Date: Feb 2004
Location: Albuquerque, NM
Posts: 1,612
Quote:
Originally Posted by Motley Fool
So, is there a problem with my setup, or is there a problem with SecureCRT 5.5.1?
Thanks for posting such a detailed description of what you are trying to do. Currently, SecureCRT doesn't support multiple international character sets at the same time. This is something we hope to address in a future version. I will add your request to our development database. When we have something for you to try, I will post a note here. If you would also like to be notified by e-mail, please send e-mail to me at Maureen.Jett@vandyke.com.

Maureen
Reply With Quote
  #3  
Old 08-17-2007, 10:18 AM
Motley Fool Motley Fool is offline
Registered User
 
Join Date: Aug 2007
Posts: 4
Quote:
Originally Posted by Maureen
Currently, SecureCRT doesn't support multiple international character sets at the same time.
Maureen, thanks for the reply. What you mentioned answers my questions about issue #1.

How about issue #2?
Quote:
2. Incorrect box rendering in any Script (whether Western, Greek or Cyrillic) that doesn't appear to be problem in PuTTY, even using the same Lucida Console font. Even in its best mode (i.e. in Western Script), SecureCRT seems to be using the "poor man's" box rendering method of pluses, hyphens and pipes rather than actual Unicode box characters. (I should note that it made no difference whether the "Use Unicode line-drawing characters" SecureCRT setting was checked or not - the boxes are drawn the same way.)
Can that be solved?

I found a similar box-drawing thread over here. The final post by troybtj said that he had resolved it by setting his LANG variable to "en_US.UTF-8". Mine was already set that way, however (see the "locale" section of my Linux box in my first posting), and yet the problem remains.

Thanks again.

Last edited by Motley Fool; 08-17-2007 at 11:19 AM.
Reply With Quote
  #4  
Old 08-17-2007, 03:23 PM
Maureen's Avatar
Maureen Maureen is offline
VanDyke Product Director
 
Join Date: Feb 2004
Location: Albuquerque, NM
Posts: 1,612
Quote:
Originally Posted by Motley Fool
How about issue #2?
Can that be solved?

I found a similar box-drawing thread over here. The final post by troybtj said that he had resolved it by setting his LANG variable to "en_US.UTF-8". Mine was already set that way, however (see the "locale" section of my Linux box in my first posting), and yet the problem remains.
Thanks for following up and sorry I only answered one of your questions. The line drawing problem is related to the multiple character set problem. The sample file contains pure Unicode characters and currently, SecureCRT doesn't have a transformer that can handle these characters.

Maureen
Reply With Quote
  #5  
Old 08-18-2007, 12:52 AM
Motley Fool Motley Fool is offline
Registered User
 
Join Date: Aug 2007
Posts: 4
Quote:
Originally Posted by Maureen
The sample file contains pure Unicode characters and currently, SecureCRT doesn't have a transformer that can handle these characters.
Maureen, I may just not be catching on, but let me ask one more question. To get rid of all the multiple character sets, I tried creating a file with just spaces and Unicode box characters. (i.e. using U+2500 through U+255D to make a simple single-line box and double-line box). I set both PuTTY and SecureCRT to use Lucida Console, which contains the characters needed for box drawing. I tried setting SecureCRT's "Use Unicode Line-drawing characters" checkbox to both on and off.

Here's what I got: two-boxes.gif

So, in this situation where the font has the necessary characters and everything is set to UTF-8, what's preventing SecureCRT from drawing the boxes correctly?

Thanks again.
Reply With Quote
  #6  
Old 08-20-2007, 06:56 PM
Maureen's Avatar
Maureen Maureen is offline
VanDyke Product Director
 
Join Date: Feb 2004
Location: Albuquerque, NM
Posts: 1,612
Quote:
Originally Posted by Motley Fool
So, in this situation where the font has the necessary characters and everything is set to UTF-8, what's preventing SecureCRT from drawing the boxes correctly?
You're correct that the font has the necessary characters. The problem is that the file is Unicode and currently, SecureCRT can only display line-drawing characters for single-byte character sets.

It seems like the samples you've posted are for testing purposes. I was wondering how close your "real" files are to what you've posted. For example, do you need to be able to display multiple language character sets in the same file? What are your line-drawing needs?

Maureen
Reply With Quote
  #7  
Old 08-22-2007, 12:28 PM
Motley Fool Motley Fool is offline
Registered User
 
Join Date: Aug 2007
Posts: 4
Well, for example, here are a couple of tools I use. This is "aptitude", the Ubuntu console-based package management system. aptitude.gif

And here's dpkg-config system for configuring Ubuntu's various packages. configuration.gif

The box-drawing quality is acceptable... just not as good as it could be.
Reply With Quote
  #8  
Old 09-27-2007, 01:28 PM
mjr mjr is offline
Registered User
 
Join Date: Sep 2007
Posts: 4
Angry

I'd like to chip into this thread. This is the only reason I have not yet purchased SecureCRT. If a simple program like mIRC or PuTTY can handle UTF-8 near flawlessly, why can't a program purchased by international defense contractors, banks, and manufacturing companies? Surely clients such as Microsoft and Hitachi need to work with both Japanese and Western script at the same time, and clients such as Verizon, a Vodafone subsidiary, need to work with other subsidiaries that use Eastern European, Arab, and Japanese scripts? I really love the software, but I often have to work with multiple scripts at the same time (such as editing a file in Japanese while using a Western editor).
Reply With Quote
  #9  
Old 10-08-2007, 12:03 PM
mdella's Avatar
mdella mdella is offline
Registered User
 
Join Date: Mar 2004
Location: Scotts Valley, CA
Posts: 44
Unicode and UTF-8 or UTF-16...

...I'll chip in as well ;-)

For the last couple of years I've been working on several international projects that are targeted to Japan, Korea, India, and China. To some degree I've given up on actually using SecureCRT to work mostly right with the UTF-8 version of the US character set and supporting characters. Rather I have been taking files and using SecureFX to transfer them to my desktop, editing them with EditPlus (and SuperEdit) to get the correct character set encodings (Could never trust what I say being typed by SecureCRT was actually the correct encoding, it just looked correctly on my screen rarely elsewhere) then upload the files back to the box once done.

Granted this is time consuming and the like, but I use SecureCRT for a plethera of other things and getting the Unicode stuff was a low priority for me. But since others are having this issue, I thought I'd chime in.

Oh by the way, slowly but surely, other countries (esp Europe) are converting to UTF-8 or -16 and away from their "custom" character sets. The US is the slowest in the world to conform since everyone here uses ASCII (ok, so its dating me by 20+ years) and (cough cough) Color ANSI Color (invented back in the 80s ;-)

Course most of the US stuff I use falls into this category so its hasn't been that big a deal until I took on this last international job...

Marcos
__________________
Marcos Della
Data Center Cloud Architect
Nutanix

PGP Fingerprint: BDC7 AFFD E94F FA09 C839 9153 F5FF E128 3094 2B9E
Key ID: 0x30942B9E
Reply With Quote
  #10  
Old 10-26-2008, 12:18 AM
schickb schickb is offline
Registered User
 
Join Date: Oct 2008
Posts: 1
Wow, very old thread, but it appears this is still broken. I sure wish SecureCRT could correctly display Debian package configuration under Ubuntu or Debian.

Any plans to fix this?
Reply With Quote
  #11  
Old 10-27-2008, 06:24 PM
Maureen's Avatar
Maureen Maureen is offline
VanDyke Product Director
 
Join Date: Feb 2004
Location: Albuquerque, NM
Posts: 1,612
Thanks for following up on this. We have spent some time looking into the Unicode issue and have made some code changes that will make it easier to implement this. We don't have a timeline for implementing full Unicode support, but I'll definitely post here when we have something to try.

Maureen
Reply With Quote
  #12  
Old 08-17-2009, 09:55 AM
Maureen's Avatar
Maureen Maureen is offline
VanDyke Product Director
 
Join Date: Feb 2004
Location: Albuquerque, NM
Posts: 1,612
SecureCRT 6.5 has improved UTF-8 support in the following ways.
  • character sets from multiple languages can be viewed in a session
  • improved support for right-to-left reading order languages
  • UTF-8 characters can be mapped in the Keymap Editor
  • Send/Receive ASCII works with UTF-8 data
  • UTF-8 filenames can be used in the SFTP tab
  • pass-through printing supports UTF-8 data
  • better behavior with scripts that use combining marks to compose characters
This version is currently in pre-beta testing. If you would like to try it, please send e-mail to me at Maureen.Jett@vandyke.com.

Maureen
Reply With Quote
  #13  
Old 12-04-2009, 11:24 PM
solarein solarein is offline
Registered User
 
Join Date: Dec 2009
Posts: 2
I'm using SecureCRT 6.1.4 and I'm having the same problems with unicode line drawing characters with UTF-8 not displaying correctly. After some searching I was able to find this ancient thread, and it looks like this is the same problem remaining after all this time. I'm using Linux emulation mode with ANSI Color enabled, and Courier New 10pt font. Like those before, my line drawing characters in aptitude and other programs are displayed as dashes and plusses, i.e. "poor man's line drawing characters" rather than as unicode line drawing characters. I haven't tried the 6.5 beta yet because Maureen's message doesn't list a fix to unicode line drawing. Could someone tell me if this has been fixed with version 6.5 or is planned to be fixed in a later release? Thanks.
Reply With Quote
  #14  
Old 12-05-2009, 09:46 AM
miked's Avatar
miked miked is offline
Registered User
 
Join Date: Feb 2004
Posts: 2,040
The SecureCRT 6.5 History describes changes to Unicode support. If you you try 6.5 and it doesn't work better for you, we can try to help. In the worst case scenario you can easily revert to 6.1 again.
__________________
Mike
VanDyke Software
Technical Support
[http://www.vandyke.com/support]
Reply With Quote
  #15  
Old 12-10-2009, 09:48 AM
solarein solarein is offline
Registered User
 
Join Date: Dec 2009
Posts: 2
I just upgraded to SecureCRT 6.5 and unicode line drawing is now working perfectly for me. I also like the new interface. Thank you very much and keep up the great work!
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -6. The time now is 12:03 AM.