Terminal: support for underline color and styles

the text is now printed above the underline.

Change-Id: I1a3a7713bf514830106532e1534793e0fe158bc2
Reviewed-on: https://review.haiku-os.org/c/haiku/+/6706
Reviewed-by: waddlesplash <waddlesplash@gmail.com>

Terminal: implement ECMA-48 "REP"

fix #16724

Change-Id: Ie9f8252393a65a0101a6d78db2360f48bb52bc73
Reviewed-on: https://review.haiku-os.org/c/haiku/+/3607
Reviewed-by: Adrien Destugues <pulkomandy@gmail.com>

Terminal: implement index and next-line CSI sequences.

* also ignore 0xa, 0xb, 0xc control characters in CSI sequences.

Terminal: Implement a few cursor CSI commands.

* Cursor back tab, cursor forward tab.
* Cursor next line, cursor previous line.
* also add size 13 to the list of font sizes.

Fix GB18030 encoding support. And some cleanup ...

* Fix GB18030 Chinese encoding support for two and four bytes long
characters. This finally resolves issue described in #6227;
* Processing of multi-byte characters was slightly refactored too;
* Remove the multi-byte 94/96 graphsets designation support for
Japanese encodings. That looks like MuTerm rudiment, it had incomplete
implementation and looked like abandoned. On the other hand multi-byte
designation must be implemented in the same way as designation for
single-byte graphsets was done. Note that this multi-byte graphsets
designation has nothing to do with the normal encoding support for
usual data flow conversion - so you will be on the safe side when
use terminal encoding menu switch.
The removed feature is the ancient technique to achieve different charsets
support on 8-bit serial lines by assigning (designating) predefined
sets of characters to G0, G1, G2 and G3 and selecting them during
program life-time into GL (x20-x07E) or GR (xA0-xFF) areas by using LS
or SS functions.
For example xterm has no support for designation multi-byte graphsets
at all. Anyway if this feature is required and you can provide the
test environment - please let me know and I will be glad to implement
this feature in more easy and consistent way;
* Remove unreferenced gSmbcsTable and gScsTable parsing tables that
looks like is not used anymore;
* Remove gCS96GroundTable and gMbcsTable parsing tables that were used
by multi-byte 94/96 Japanese graphsets support and now obsoleted by
removing mentioned feature;
* Remove some obsoleted #defines, like HW statusline support for
example, from parse tables definition.

Code style fixes. No functional changes.

Fix for some code style issues pointed out by Axel. Thanks.

More standard handling G0,G1,G2,G3 graph.char sets

Handling of graphic character sets G0-G3 is fixed and implemented in
more consistent, extensible way.

Terminal:Add ACS symbols supported by xterm-256color

More ACS (also known line drawing) symbols should be supported by
xterm-256color terminal emulator.
Partially fixes #9293.

CodeStyle fixes, some refactoring and cleanup

* Clear some codestyle issues catched by checker script;
* Rename RestartDebugCapture to more consistent StartStopDebugCapture;
* Updated Copyrights and authors lists, some occurences of the raw MIT
licence text replaced with "under the terms of MIT licence" reference;
* Fixes for x86_64 build.

This is cumulative cleanup commit for series of Terminal refactoring
changes I have introduced last time. No significant functional changes.

Support for underline and ibeam terminal cursor styles

* Add support for underline and i-beam cursor shapes. No corresponding
UI configuration in preferences view ATM because preferences are waiting
for refactoring and we need some eggs for this Easters. ;-)
* Add handling of VT520/xterm specific DECSCUSR control sequences
allowing applications to modify the style and blinking state of the
cursor. May be utilized, for example, by console version of vim;
* Implement cursor blinking/hiding on DECSET/DECRST commands.

Refactoring of 8-bit encodings support.
* Fixed issue introduced in hrev38139: restoring from the line
drawing table was hard-coded to UTF8 Ground table. That is wrong:
the table for currently configured encoding must be set back.
Please look on using of _GuessGroundTable() for details;

* Fixed issue introduced in hrev34894: the semantic of convert_xx_utf8
functions requires the destination length to be set equal to the
target buffer size. Pre-hrev34894 usage of "homebrew" conversion
functions was a bit different - destination length was set to 0.
This made any converstions of input data useless and produce no
visual results;

* Private list of supported encodings (Encoding.cpp) was replaced by
using BPrivate::BCharacterSetRoster functionality. That allows to
use centralized info about encodings in unified with other
applications (Mail & StyledEdit for example) way. Most of currently
enumerated in UTF8.h encodings now available in Terminal.
Note that UCS-2 and UTF-16 are temporary (???) excluded from the
list of encodings supported by Terminal.

* The B_UTF16_CONVERSION was added in system-wide UTF8.h declarations.
This character set is available for enumerating by BCharacterSetRoster
but not listed in public API. Looks like it was just missed;

* Special note about "Text Encoding" entry in Preference File:
So known "shortname" of encoding was used in the preferences file.
For details look on the encodings list in previous version of
Encoding.cpp. As result of migrating to BCharacterSet-provided
resources this list was deleted and is not available anymore.
Instead of it the IANA name of the character encoding targeted
to be used for this purposes. Frankly speaking this part looks
like not working at the moment. The value of text encoding is
hardcoded to "UTF-8" now and is not affected by any operations
in Terminal menu. Note that "shortname" for default encoding
was "UTF8" but the saved value is "UTF-8" - and they are looking
not dependent at all. So this change should not introduce any
kind of backward incompatibility.

Handle more pseudoÃ-graphic characters and ways of using them. This gets aptitude (from debianÃ) displaying properly and should also fix midnight commander.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38139 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fix #2673 : allow use of \E(0 to switch to line drawing character set.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38088 a95241bf-73f2-0310-859d-f6bbb57e9c96

Patch by Joshua R. Elsasser:
Implements DA, reporting that the terminal is a VT102.
Implements DEXREQTPARM, reporting the same parameters that xterm does.
Fixes DSR 6 to actually report the position of the cursor.

Resolves ticket #4659. Thanks!



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33327 a95241bf-73f2-0310-859d-f6bbb57e9c96

Patch by Joshua R. Elsasser:

Implement correctly handling tab stop escape sequences.

Resolves ticket #4657. Thanks!



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33326 a95241bf-73f2-0310-859d-f6bbb57e9c96

Ignore ^?. That's what xterm seems to do.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26137 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added support for \ESC[%dX (erase characters).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25988 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added support for \ESC[%dT (scroll screen down).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25984 a95241bf-73f2-0310-859d-f6bbb57e9c96

Support for \ESC[%dS (scroll screen up).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25982 a95241bf-73f2-0310-859d-f6bbb57e9c96

- some more ANSI attributes
- changed comments to pragma comments for parse tables as it's huge, but sadly Pe doesn't make them clickable in the function list...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25409 a95241bf-73f2-0310-859d-f6bbb57e9c96

added the g prefix to some global variable

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25401 a95241bf-73f2-0310-859d-f6bbb57e9c96

- SetPosX/Y were wrong, but not yet used.
- implemented HPA/VPA (CV/CH) sequences (set absolute v and h position). It's optional but because the beterm termcap advertises it we must implement it. Also note beterm increments args unlike others. This fixes all obvious display issues with the rhapsody IRC client (a small ncurses client I ported).
- maybe we should switch the CASE_* to an enum ?


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25392 a95241bf-73f2-0310-859d-f6bbb57e9c96

Moved the files out of the MYOB folder into the main folder.
There are some double entries (like TermApp.cpp and TerminalApp.cpp, so
it still needs a further cleanup).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13734 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fix GB18030 encoding support. And some cleanup ...

* Fix GB18030 Chinese encoding support for two and four bytes long
characters. This finally resolves issue described in #6227;
* Processing of multi-byte characters was slightly refactored too;
* Remove the multi-byte 94/96 graphsets designation support for
Japanese encodings. That looks like MuTerm rudiment, it had incomplete
implementation and looked like abandoned. On the other hand multi-byte
designation must be implemented in the same way as designation for
single-byte graphsets was done. Note that this multi-byte graphsets
designation has nothing to do with the normal encoding support for
usual data flow conversion - so you will be on the safe side when
use terminal encoding menu switch.
The removed feature is the ancient technique to achieve different charsets
support on 8-bit serial lines by assigning (designating) predefined
sets of characters to G0, G1, G2 and G3 and selecting them during
program life-time into GL (x20-x07E) or GR (xA0-xFF) areas by using LS
or SS functions.
For example xterm has no support for designation multi-byte graphsets
at all. Anyway if this feature is required and you can provide the
test environment - please let me know and I will be glad to implement
this feature in more easy and consistent way;
* Remove unreferenced gSmbcsTable and gScsTable parsing tables that
looks like is not used anymore;
* Remove gCS96GroundTable and gMbcsTable parsing tables that were used
by multi-byte 94/96 Japanese graphsets support and now obsoleted by
removing mentioned feature;
* Remove some obsoleted #defines, like HW statusline support for
example, from parse tables definition.

Code style fixes. No functional changes.

Fix for some code style issues pointed out by Axel. Thanks.

More standard handling G0,G1,G2,G3 graph.char sets

Handling of graphic character sets G0-G3 is fixed and implemented in
more consistent, extensible way.

Terminal:Add ACS symbols supported by xterm-256color

More ACS (also known line drawing) symbols should be supported by
xterm-256color terminal emulator.
Partially fixes #9293.

CodeStyle fixes, some refactoring and cleanup

* Clear some codestyle issues catched by checker script;
* Rename RestartDebugCapture to more consistent StartStopDebugCapture;
* Updated Copyrights and authors lists, some occurences of the raw MIT
licence text replaced with "under the terms of MIT licence" reference;
* Fixes for x86_64 build.

This is cumulative cleanup commit for series of Terminal refactoring
changes I have introduced last time. No significant functional changes.

Support for underline and ibeam terminal cursor styles

* Add support for underline and i-beam cursor shapes. No corresponding
UI configuration in preferences view ATM because preferences are waiting
for refactoring and we need some eggs for this Easters. ;-)
* Add handling of VT520/xterm specific DECSCUSR control sequences
allowing applications to modify the style and blinking state of the
cursor. May be utilized, for example, by console version of vim;
* Implement cursor blinking/hiding on DECSET/DECRST commands.

Refactoring of 8-bit encodings support.
* Fixed issue introduced in hrev38139: restoring from the line
drawing table was hard-coded to UTF8 Ground table. That is wrong:
the table for currently configured encoding must be set back.
Please look on using of _GuessGroundTable() for details;

* Fixed issue introduced in hrev34894: the semantic of convert_xx_utf8
functions requires the destination length to be set equal to the
target buffer size. Pre-hrev34894 usage of "homebrew" conversion
functions was a bit different - destination length was set to 0.
This made any converstions of input data useless and produce no
visual results;

* Private list of supported encodings (Encoding.cpp) was replaced by
using BPrivate::BCharacterSetRoster functionality. That allows to
use centralized info about encodings in unified with other
applications (Mail & StyledEdit for example) way. Most of currently
enumerated in UTF8.h encodings now available in Terminal.
Note that UCS-2 and UTF-16 are temporary (???) excluded from the
list of encodings supported by Terminal.

* The B_UTF16_CONVERSION was added in system-wide UTF8.h declarations.
This character set is available for enumerating by BCharacterSetRoster
but not listed in public API. Looks like it was just missed;

* Special note about "Text Encoding" entry in Preference File:
So known "shortname" of encoding was used in the preferences file.
For details look on the encodings list in previous version of
Encoding.cpp. As result of migrating to BCharacterSet-provided
resources this list was deleted and is not available anymore.
Instead of it the IANA name of the character encoding targeted
to be used for this purposes. Frankly speaking this part looks
like not working at the moment. The value of text encoding is
hardcoded to "UTF-8" now and is not affected by any operations
in Terminal menu. Note that "shortname" for default encoding
was "UTF8" but the saved value is "UTF-8" - and they are looking
not dependent at all. So this change should not introduce any
kind of backward incompatibility.

Handle more pseudoÃ-graphic characters and ways of using them. This gets aptitude (from debianÃ) displaying properly and should also fix midnight commander.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38139 a95241bf-73f2-0310-859d-f6bbb57e9c96

Fix #2673 : allow use of \E(0 to switch to line drawing character set.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@38088 a95241bf-73f2-0310-859d-f6bbb57e9c96

Patch by Joshua R. Elsasser:
Implements DA, reporting that the terminal is a VT102.
Implements DEXREQTPARM, reporting the same parameters that xterm does.
Fixes DSR 6 to actually report the position of the cursor.

Resolves ticket #4659. Thanks!



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33327 a95241bf-73f2-0310-859d-f6bbb57e9c96

Patch by Joshua R. Elsasser:

Implement correctly handling tab stop escape sequences.

Resolves ticket #4657. Thanks!



git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@33326 a95241bf-73f2-0310-859d-f6bbb57e9c96

Ignore ^?. That's what xterm seems to do.


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@26137 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added support for \ESC[%dX (erase characters).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25988 a95241bf-73f2-0310-859d-f6bbb57e9c96

Added support for \ESC[%dT (scroll screen down).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25984 a95241bf-73f2-0310-859d-f6bbb57e9c96

Support for \ESC[%dS (scroll screen up).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25982 a95241bf-73f2-0310-859d-f6bbb57e9c96

- some more ANSI attributes
- changed comments to pragma comments for parse tables as it's huge, but sadly Pe doesn't make them clickable in the function list...


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25409 a95241bf-73f2-0310-859d-f6bbb57e9c96

added the g prefix to some global variable

git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25401 a95241bf-73f2-0310-859d-f6bbb57e9c96

- SetPosX/Y were wrong, but not yet used.
- implemented HPA/VPA (CV/CH) sequences (set absolute v and h position). It's optional but because the beterm termcap advertises it we must implement it. Also note beterm increments args unlike others. This fixes all obvious display issues with the rhapsody IRC client (a small ncurses client I ported).
- maybe we should switch the CASE_* to an enum ?


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@25392 a95241bf-73f2-0310-859d-f6bbb57e9c96

Moved the files out of the MYOB folder into the main folder.
There are some double entries (like TermApp.cpp and TerminalApp.cpp, so
it still needs a further cleanup).


git-svn-id: file:///srv/svn/repos/haiku/haiku/trunk@13734 a95241bf-73f2-0310-859d-f6bbb57e9c96