strings.xml revision 1.5
1<chapter xmlns="http://docbook.org/ns/docbook" version="5.0" 2 xml:id="std.strings" xreflabel="Strings"> 3<?dbhtml filename="strings.html"?> 4 5<info><title> 6 Strings 7 <indexterm><primary>Strings</primary></indexterm> 8</title> 9 <keywordset> 10 <keyword>ISO C++</keyword> 11 <keyword>library</keyword> 12 </keywordset> 13</info> 14 15<!-- Sect1 01 : Character Traits --> 16 17<!-- Sect1 02 : String Classes --> 18<section xml:id="std.strings.string" xreflabel="string"><info><title>String Classes</title></info> 19 20 21 <section xml:id="strings.string.simple" xreflabel="Simple Transformations"><info><title>Simple Transformations</title></info> 22 23 <para> 24 Here are Standard, simple, and portable ways to perform common 25 transformations on a <code>string</code> instance, such as 26 "convert to all upper case." The word transformations 27 is especially apt, because the standard template function 28 <code>transform<></code> is used. 29 </para> 30 <para> 31 This code will go through some iterations. Here's a simple 32 version: 33 </para> 34 <programlisting> 35 #include <string> 36 #include <algorithm> 37 #include <cctype> // old <ctype.h> 38 39 struct ToLower 40 { 41 char operator() (char c) const { return std::tolower(c); } 42 }; 43 44 struct ToUpper 45 { 46 char operator() (char c) const { return std::toupper(c); } 47 }; 48 49 int main() 50 { 51 std::string s ("Some Kind Of Initial Input Goes Here"); 52 53 // Change everything into upper case 54 std::transform (s.begin(), s.end(), s.begin(), ToUpper()); 55 56 // Change everything into lower case 57 std::transform (s.begin(), s.end(), s.begin(), ToLower()); 58 59 // Change everything back into upper case, but store the 60 // result in a different string 61 std::string capital_s; 62 capital_s.resize(s.size()); 63 std::transform (s.begin(), s.end(), capital_s.begin(), ToUpper()); 64 } 65 </programlisting> 66 <para> 67 <emphasis>Note</emphasis> that these calls all 68 involve the global C locale through the use of the C functions 69 <code>toupper/tolower</code>. This is absolutely guaranteed to work -- 70 but <emphasis>only</emphasis> if the string contains <emphasis>only</emphasis> characters 71 from the basic source character set, and there are <emphasis>only</emphasis> 72 96 of those. Which means that not even all English text can be 73 represented (certain British spellings, proper names, and so forth). 74 So, if all your input forevermore consists of only those 96 75 characters (hahahahahaha), then you're done. 76 </para> 77 <para><emphasis>Note</emphasis> that the 78 <code>ToUpper</code> and <code>ToLower</code> function objects 79 are needed because <code>toupper</code> and <code>tolower</code> 80 are overloaded names (declared in <code><cctype></code> and 81 <code><locale></code>) so the template-arguments for 82 <code>transform<></code> cannot be deduced, as explained in 83 <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://gcc.gnu.org/ml/libstdc++/2002-11/msg00180.html">this 84 message</link>. 85 <!-- section 14.8.2.4 clause 16 in ISO 14882:1998 --> 86 At minimum, you can write short wrappers like 87 </para> 88 <programlisting> 89 char toLower (char c) 90 { 91 return std::tolower(c); 92 } </programlisting> 93 <para>(Thanks to James Kanze for assistance and suggestions on all of this.) 94 </para> 95 <para>Another common operation is trimming off excess whitespace. Much 96 like transformations, this task is trivial with the use of string's 97 <code>find</code> family. These examples are broken into multiple 98 statements for readability: 99 </para> 100 <programlisting> 101 std::string str (" \t blah blah blah \n "); 102 103 // trim leading whitespace 104 string::size_type notwhite = str.find_first_not_of(" \t\n"); 105 str.erase(0,notwhite); 106 107 // trim trailing whitespace 108 notwhite = str.find_last_not_of(" \t\n"); 109 str.erase(notwhite+1); </programlisting> 110 <para>Obviously, the calls to <code>find</code> could be inserted directly 111 into the calls to <code>erase</code>, in case your compiler does not 112 optimize named temporaries out of existence. 113 </para> 114 115 </section> 116 <section xml:id="strings.string.case" xreflabel="Case Sensitivity"><info><title>Case Sensitivity</title></info> 117 118 <para> 119 </para> 120 121 <para>The well-known-and-if-it-isn't-well-known-it-ought-to-be 122 <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://www.gotw.ca/gotw/">Guru of the Week</link> 123 discussions held on Usenet covered this topic in January of 1998. 124 Briefly, the challenge was, <quote>write a 'ci_string' class which 125 is identical to the standard 'string' class, but is 126 case-insensitive in the same way as the (common but nonstandard) 127 C function stricmp()</quote>. 128 </para> 129 <programlisting> 130 ci_string s( "AbCdE" ); 131 132 // case insensitive 133 assert( s == "abcde" ); 134 assert( s == "ABCDE" ); 135 136 // still case-preserving, of course 137 assert( strcmp( s.c_str(), "AbCdE" ) == 0 ); 138 assert( strcmp( s.c_str(), "abcde" ) != 0 ); </programlisting> 139 140 <para>The solution is surprisingly easy. The original answer was 141 posted on Usenet, and a revised version appears in Herb Sutter's 142 book <emphasis>Exceptional C++</emphasis> and on his website as <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://www.gotw.ca/gotw/029.htm">GotW 29</link>. 143 </para> 144 <para>See? Told you it was easy!</para> 145 <para> 146 <emphasis>Added June 2000:</emphasis> The May 2000 issue of C++ 147 Report contains a fascinating <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://lafstern.org/matt/col2_new.pdf"> article</link> by 148 Matt Austern (yes, <emphasis>the</emphasis> Matt Austern) on why 149 case-insensitive comparisons are not as easy as they seem, and 150 why creating a class is the <emphasis>wrong</emphasis> way to go 151 about it in production code. (The GotW answer mentions one of 152 the principle difficulties; his article mentions more.) 153 </para> 154 <para>Basically, this is "easy" only if you ignore some things, 155 things which may be too important to your program to ignore. (I chose 156 to ignore them when originally writing this entry, and am surprised 157 that nobody ever called me on it...) The GotW question and answer 158 remain useful instructional tools, however. 159 </para> 160 <para><emphasis>Added September 2000:</emphasis> James Kanze provided a link to a 161 <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://www.unicode.org/reports/tr21/tr21-5.html">Unicode 162 Technical Report discussing case handling</link>, which provides some 163 very good information. 164 </para> 165 166 </section> 167 <section xml:id="strings.string.character_types" xreflabel="Arbitrary Characters"><info><title>Arbitrary Character Types</title></info> 168 169 <para> 170 </para> 171 172 <para>The <code>std::basic_string</code> is tantalizingly general, in that 173 it is parameterized on the type of the characters which it holds. 174 In theory, you could whip up a Unicode character class and instantiate 175 <code>std::basic_string<my_unicode_char></code>, or assuming 176 that integers are wider than characters on your platform, maybe just 177 declare variables of type <code>std::basic_string<int></code>. 178 </para> 179 <para>That's the theory. Remember however that basic_string has additional 180 type parameters, which take default arguments based on the character 181 type (called <code>CharT</code> here): 182 </para> 183 <programlisting> 184 template <typename CharT, 185 typename Traits = char_traits<CharT>, 186 typename Alloc = allocator<CharT> > 187 class basic_string { .... };</programlisting> 188 <para>Now, <code>allocator<CharT></code> will probably Do The Right 189 Thing by default, unless you need to implement your own allocator 190 for your characters. 191 </para> 192 <para>But <code>char_traits</code> takes more work. The char_traits 193 template is <emphasis>declared</emphasis> but not <emphasis>defined</emphasis>. 194 That means there is only 195 </para> 196 <programlisting> 197 template <typename CharT> 198 struct char_traits 199 { 200 static void foo (type1 x, type2 y); 201 ... 202 };</programlisting> 203 <para>and functions such as char_traits<CharT>::foo() are not 204 actually defined anywhere for the general case. The C++ standard 205 permits this, because writing such a definition to fit all possible 206 CharT's cannot be done. 207 </para> 208 <para>The C++ standard also requires that char_traits be specialized for 209 instantiations of <code>char</code> and <code>wchar_t</code>, and it 210 is these template specializations that permit entities like 211 <code>basic_string<char,char_traits<char>></code> to work. 212 </para> 213 <para>If you want to use character types other than char and wchar_t, 214 such as <code>unsigned char</code> and <code>int</code>, you will 215 need suitable specializations for them. For a time, in earlier 216 versions of GCC, there was a mostly-correct implementation that 217 let programmers be lazy but it broke under many situations, so it 218 was removed. GCC 3.4 introduced a new implementation that mostly 219 works and can be specialized even for <code>int</code> and other 220 built-in types. 221 </para> 222 <para>If you want to use your own special character class, then you have 223 <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://gcc.gnu.org/ml/libstdc++/2002-08/msg00163.html">a lot 224 of work to do</link>, especially if you with to use i18n features 225 (facets require traits information but don't have a traits argument). 226 </para> 227 <para>Another example of how to specialize char_traits was given <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://gcc.gnu.org/ml/libstdc++/2002-08/msg00260.html">on the 228 mailing list</link> and at a later date was put into the file <code> 229 include/ext/pod_char_traits.h</code>. We agree 230 that the way it's used with basic_string (scroll down to main()) 231 doesn't look nice, but that's because <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://gcc.gnu.org/ml/libstdc++/2002-08/msg00236.html">the 232 nice-looking first attempt</link> turned out to <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://gcc.gnu.org/ml/libstdc++/2002-08/msg00242.html">not 233 be conforming C++</link>, due to the rule that CharT must be a POD. 234 (See how tricky this is?) 235 </para> 236 237 </section> 238 239 <section xml:id="strings.string.token" xreflabel="Tokenizing"><info><title>Tokenizing</title></info> 240 241 <para> 242 </para> 243 <para>The Standard C (and C++) function <code>strtok()</code> leaves a lot to 244 be desired in terms of user-friendliness. It's unintuitive, it 245 destroys the character string on which it operates, and it requires 246 you to handle all the memory problems. But it does let the client 247 code decide what to use to break the string into pieces; it allows 248 you to choose the "whitespace," so to speak. 249 </para> 250 <para>A C++ implementation lets us keep the good things and fix those 251 annoyances. The implementation here is more intuitive (you only 252 call it once, not in a loop with varying argument), it does not 253 affect the original string at all, and all the memory allocation 254 is handled for you. 255 </para> 256 <para>It's called stringtok, and it's a template function. Sources are 257 as below, in a less-portable form than it could be, to keep this 258 example simple (for example, see the comments on what kind of 259 string it will accept). 260 </para> 261 262<programlisting> 263#include <string> 264template <typename Container> 265void 266stringtok(Container &container, string const &in, 267 const char * const delimiters = " \t\n") 268{ 269 const string::size_type len = in.length(); 270 string::size_type i = 0; 271 272 while (i < len) 273 { 274 // Eat leading whitespace 275 i = in.find_first_not_of(delimiters, i); 276 if (i == string::npos) 277 return; // Nothing left but white space 278 279 // Find the end of the token 280 string::size_type j = in.find_first_of(delimiters, i); 281 282 // Push token 283 if (j == string::npos) 284 { 285 container.push_back(in.substr(i)); 286 return; 287 } 288 else 289 container.push_back(in.substr(i, j-i)); 290 291 // Set up for next loop 292 i = j + 1; 293 } 294} 295</programlisting> 296 297 298 <para> 299 The author uses a more general (but less readable) form of it for 300 parsing command strings and the like. If you compiled and ran this 301 code using it: 302 </para> 303 304 305 <programlisting> 306 std::list<string> ls; 307 stringtok (ls, " this \t is\t\n a test "); 308 for (std::list<string>const_iterator i = ls.begin(); 309 i != ls.end(); ++i) 310 { 311 std::cerr << ':' << (*i) << ":\n"; 312 } </programlisting> 313 <para>You would see this as output: 314 </para> 315 <programlisting> 316 :this: 317 :is: 318 :a: 319 :test: </programlisting> 320 <para>with all the whitespace removed. The original <code>s</code> is still 321 available for use, <code>ls</code> will clean up after itself, and 322 <code>ls.size()</code> will return how many tokens there were. 323 </para> 324 <para>As always, there is a price paid here, in that stringtok is not 325 as fast as strtok. The other benefits usually outweigh that, however. 326 </para> 327 328 <para><emphasis>Added February 2001:</emphasis> Mark Wilden pointed out that the 329 standard <code>std::getline()</code> function can be used with standard 330 <code>istringstreams</code> to perform 331 tokenizing as well. Build an istringstream from the input text, 332 and then use std::getline with varying delimiters (the three-argument 333 signature) to extract tokens into a string. 334 </para> 335 336 337 </section> 338 <section xml:id="strings.string.shrink" xreflabel="Shrink to Fit"><info><title>Shrink to Fit</title></info> 339 340 <para> 341 </para> 342 <para>From GCC 3.4 calling <code>s.reserve(res)</code> on a 343 <code>string s</code> with <code>res < s.capacity()</code> will 344 reduce the string's capacity to <code>std::max(s.size(), res)</code>. 345 </para> 346 <para>This behaviour is suggested, but not required by the standard. Prior 347 to GCC 3.4 the following alternative can be used instead 348 </para> 349 <programlisting> 350 std::string(str.data(), str.size()).swap(str); 351 </programlisting> 352 <para>This is similar to the idiom for reducing 353 a <code>vector</code>'s memory usage 354 (see <link linkend="faq.size_equals_capacity">this FAQ 355 entry</link>) but the regular copy constructor cannot be used 356 because libstdc++'s <code>string</code> is Copy-On-Write in GCC 3. 357 </para> 358 <para>In <link linkend="status.iso.2011">C++11</link> mode you can call 359 <code>s.shrink_to_fit()</code> to achieve the same effect as 360 <code>s.reserve(s.size())</code>. 361 </para> 362 363 364 </section> 365 366 <section xml:id="strings.string.Cstring" xreflabel="CString (MFC)"><info><title>CString (MFC)</title></info> 367 368 <para> 369 </para> 370 371 <para>A common lament seen in various newsgroups deals with the Standard 372 string class as opposed to the Microsoft Foundation Class called 373 CString. Often programmers realize that a standard portable 374 answer is better than a proprietary nonportable one, but in porting 375 their application from a Win32 platform, they discover that they 376 are relying on special functions offered by the CString class. 377 </para> 378 <para>Things are not as bad as they seem. In 379 <link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://gcc.gnu.org/ml/gcc/1999-04n/msg00236.html">this 380 message</link>, Joe Buck points out a few very important things: 381 </para> 382 <itemizedlist> 383 <listitem><para>The Standard <code>string</code> supports all the operations 384 that CString does, with three exceptions. 385 </para></listitem> 386 <listitem><para>Two of those exceptions (whitespace trimming and case 387 conversion) are trivial to implement. In fact, we do so 388 on this page. 389 </para></listitem> 390 <listitem><para>The third is <code>CString::Format</code>, which allows formatting 391 in the style of <code>sprintf</code>. This deserves some mention: 392 </para></listitem> 393 </itemizedlist> 394 <para> 395 The old libg++ library had a function called form(), which did much 396 the same thing. But for a Standard solution, you should use the 397 stringstream classes. These are the bridge between the iostream 398 hierarchy and the string class, and they operate with regular 399 streams seamlessly because they inherit from the iostream 400 hierarchy. An quick example: 401 </para> 402 <programlisting> 403 #include <iostream> 404 #include <string> 405 #include <sstream> 406 407 string f (string& incoming) // incoming is "foo N" 408 { 409 istringstream incoming_stream(incoming); 410 string the_word; 411 int the_number; 412 413 incoming_stream >> the_word // extract "foo" 414 >> the_number; // extract N 415 416 ostringstream output_stream; 417 output_stream << "The word was " << the_word 418 << " and 3*N was " << (3*the_number); 419 420 return output_stream.str(); 421 } </programlisting> 422 <para>A serious problem with CString is a design bug in its memory 423 allocation. Specifically, quoting from that same message: 424 </para> 425 <programlisting> 426 CString suffers from a common programming error that results in 427 poor performance. Consider the following code: 428 429 CString n_copies_of (const CString& foo, unsigned n) 430 { 431 CString tmp; 432 for (unsigned i = 0; i < n; i++) 433 tmp += foo; 434 return tmp; 435 } 436 437 This function is O(n^2), not O(n). The reason is that each += 438 causes a reallocation and copy of the existing string. Microsoft 439 applications are full of this kind of thing (quadratic performance 440 on tasks that can be done in linear time) -- on the other hand, 441 we should be thankful, as it's created such a big market for high-end 442 ix86 hardware. :-) 443 444 If you replace CString with string in the above function, the 445 performance is O(n). 446 </programlisting> 447 <para>Joe Buck also pointed out some other things to keep in mind when 448 comparing CString and the Standard string class: 449 </para> 450 <itemizedlist> 451 <listitem><para>CString permits access to its internal representation; coders 452 who exploited that may have problems moving to <code>string</code>. 453 </para></listitem> 454 <listitem><para>Microsoft ships the source to CString (in the files 455 MFC\SRC\Str{core,ex}.cpp), so you could fix the allocation 456 bug and rebuild your MFC libraries. 457 <emphasis><emphasis>Note:</emphasis> It looks like the CString shipped 458 with VC++6.0 has fixed this, although it may in fact have been 459 one of the VC++ SPs that did it.</emphasis> 460 </para></listitem> 461 <listitem><para><code>string</code> operations like this have O(n) complexity 462 <emphasis>if the implementors do it correctly</emphasis>. The libstdc++ 463 implementors did it correctly. Other vendors might not. 464 </para></listitem> 465 <listitem><para>While parts of the SGI STL are used in libstdc++, their 466 string class is not. The SGI <code>string</code> is essentially 467 <code>vector<char></code> and does not do any reference 468 counting like libstdc++'s does. (It is O(n), though.) 469 So if you're thinking about SGI's string or rope classes, 470 you're now looking at four possibilities: CString, the 471 libstdc++ string, the SGI string, and the SGI rope, and this 472 is all before any allocator or traits customizations! (More 473 choices than you can shake a stick at -- want fries with that?) 474 </para></listitem> 475 </itemizedlist> 476 477 </section> 478</section> 479 480<!-- Sect1 03 : Interacting with C --> 481 482</chapter> 483