1What's New In Libevent 2.0 so far:
2
31. Meta-issues
4
51.1. About this document
6
7  This document describes the key differences between Libevent 1.4 and
8  Libevent 2.0, from a user's point of view.  It was most recently
9  updated based on features in git master as of August 2010.
10
11  NOTE: I am very sure that I missed some thing on this list.  Caveat
12  haxxor.
13
141.2. Better documentation
15
16  There is now a book-in-progress that explains how to use Libevent and its
17  growing pile of APIs.  As of this writing, it covers everything except the
18  http and rpc code.  Check out the latest draft at
19  http://www.wangafu.net/~nickm/libevent-book/ .
20
212. New and Improved Event APIs
22
23  Many APIs are improved, refactored, or deprecated in Libevent 2.0.
24
25  COMPATIBILITY:
26
27  Nearly all existing code that worked with Libevent 1.4 should still
28  work correctly with Libevent 2.0.  However, if you are writing new code,
29  or if you want to port old code, we strongly recommend using the new APIs
30  and avoiding deprecated APIs as much as possible.
31
32  Binaries linked against Libevent 1.4 will need to be recompiled to link
33  against Libevent 2.0.  This is nothing new; we have never been good at
34  preserving binary compatibility between releases.  We'll try harder in the
35  future, though: see 2.1 below.
36
372.1. New header layout for improved forward-compatibility
38
39  Libevent 2.0 has a new header layout to make it easier for programmers to
40  write good, well-supported libevent code.  The new headers are divided
41  into three types.
42
43  There are *regular headers*, like event2/event.h.  These headers contain
44  the functions that most programmers will want to use.
45
46  There are *backward compatibility headers*, like event2/event_compat.h.
47  These headers contain declarations for deprecated functions from older
48  versions of Libevent.  Documentation in these headers should suggest what's
49  wrong with the old functions, and what functions you want to start using
50  instead of the old ones.  Some of these functions might be removed in a
51  future release.  New programs should generally not include these headers.
52
53  Finally, there are *structure headers*, like event2/event_struct.h.
54  These headers contain definitions of some structures that Libevent has
55  historically exposed.  Exposing them caused problems in the past,
56  since programs that were compiled to work with one version of Libevent
57  would often stop working with another version that changed the size or
58  layout of some object.  We've moving them into separate headers so
59  that programmers can know that their code is not depending on any
60  unstable aspect of the Libvent ABI.  New programs should generally not
61  include these headers unless they really know what they are doing, are
62  willing to rebuild their software whenever they want to link it
63  against a new version of Libevent, and are willing to risk their code
64  breaking if and when data structures change.
65
66  Functionality that once was located in event.h is now more subdivided.
67  The core event logic is now in event2/event.h.  The "evbuffer" functions
68  for low-level buffer manipulation are in event2/buffer.h.  The
69  "bufferevent" functions for higher-level buffered IO are in
70  event2/bufferevent.h.
71
72  COMPATIBILITY:
73
74  All of the old headers (event.h, evdns.h, evhttp.h, evrpc.h, and
75  evutil.h) will continue to work by including the corresponding new
76  headers.  Old code should not be broken by this change.
77
782.2. New thread-safe, binary-compatible, harder-to-mess-up APIs
79
80  Some aspects of the historical Libevent API have encouraged
81  non-threadsafe code, or forced code built against one version of Libevent
82  to no longer build with another.  The problems with now-deprecated APIs
83  fell into two categories:
84
85     1) Dependence on the "current" event_base.  In an application with
86        multiple event_bases, Libevent previously had a notion of the
87        "current" event_base.  New events were linked to this base, and
88        the caller needed to explicitly reattach them to another base.
89        This was horribly error-prone.
90
91        Functions like "event_set" that worked with the "current" event_base
92        are now deprecated but still available (see 2.1).  There are new
93        functions like "event_assign" that take an explicit event_base
94        argument when setting up a structure.  Using these functions will help
95        prevent errors in your applications, and to be more threadsafe.
96
97     2) Structure dependence.  Applications needed to allocate 'struct
98        event' themselves, since there was no function in Libevent to do it
99        for them.  But since the size and contents of struct event can
100        change between libevent versions, this created binary-compatibility
101        nightmares.  All structures of this kind are now isolated in
102        _struct.h header (see 2.1), and there are new allocate-and-
103        initialize functions you can use instead of the old initialize-only
104        functions.  For example, instead of malloc and event_set, you
105        can use event_new().
106
107        (For people who do really want to allocate a struct event on the
108        stack, or put one inside another structure, you can still use
109        event2/event_compat.h.)
110
111   So in the case where old code would look like this:
112
113      #include <event.h>
114      ...
115      struct event *ev = malloc(sizeof(struct event));
116      /* This call will cause a buffer overrun if you compile with one version
117         of Libevent and link dynamically against another. */
118      event_set(ev, fd, EV_READ, cb, NULL);
119      /* If you forget this call, your code will break in hard-to-diagnose
120         ways in the presence of multiple event bases. */
121      event_set_base(ev, base);
122
123   New code will look more like this:
124
125     #include <event2/event.h>
126     ...
127     struct event *ev;
128     ev = event_new(base, fd, EV_READ, cb, NULL);
129
1302.3. Overrideable allocation functions
131
132  If you want to override the allocation functions used by libevent
133  (for example, to use a specialized allocator, or debug memory
134  issues, or so on), you can replace them by calling
135  event_set_mem_functions.  It takes replacements for malloc(),
136  free(), and realloc().
137
138  If you're going to use this facility, you need to call it _before_
139  Libevent does any memory allocation; otherwise, Libevent may allocate some
140  memory with malloc(), and free it with the free() function you provide.
141
142  You can disable this feature when you are building Libevent by passing
143  the --disable-malloc-replacement argument to configure.
144
1452.4. Configurable event_base creation
146
147  Older versions of Libevent would always got the fastest backend
148  available, unless you reconfigured their behavior with the environment
149  variables EVENT_NOSELECT, EVENT_NOPOLL, and so forth.  This was annoying
150  to programmers who wanted to pick a backend explicitly without messing
151  with the environment.
152
153  Also, despite our best efforts, not every backend supports every
154  operation we might like.  Some features (like edge-triggered events, or
155  working with non-socket file descriptors) only work with some operating
156  systems' fast backends.  Previously, programmers who cared about this
157  needed to know which backends supported what.  This tended to get quite
158  ungainly.
159
160  There is now an API to choose backends, either by name or by feature.
161  Here is an example:
162
163      struct event_config_t *config;
164      struct event_base *base;
165
166      /* Create a new configuration object. */
167      config = event_config_new();
168      /* We don't want to use the "select" method. */
169      event_config_avoid_method(config, "select");
170      /* We want a method that can work with non-socket file descriptors */
171      event_config_require_features(config, EV_FEATURE_FDS);
172
173      base = event_base_new_with_config(config);
174      if (!base) {
175         /* There is no backend method that does what we want. */
176         exit(1);
177      }
178      event_config_free(config);
179
180  Supported features are documented in event2/event.h
181
1822.5. Socket is now an abstract type
183
184  All APIs that formerly accepted int as a socket type now accept
185  "evutil_socket_t".  On Unix, this is just an alias for "int" as
186  before.  On Windows, however, it's an alias for SOCKET, which can
187  be wider than int on 64-bit platforms.
188
1892.6. Timeouts and persistent events work together.
190
191  Previously, it wasn't useful to set a timeout on a persistent event:
192  the timeout would trigger once, and never again.  This is not what
193  applications tend to want.  Instead, applications tend to want every
194  triggering of the event to re-set the timeout.  So now, if you set
195  up an event like this:
196       struct event *ev;
197       struct timeval tv;
198       ev = event_new(base, fd, EV_READ|EV_PERSIST, cb, NULL);
199       tv.tv_sec = 1;
200       tv.tv_usec = 0;
201       event_add(ev, &tv);
202
203  The callback 'cb' will be invoked whenever fd is ready to read, OR whenever
204  a second has passed since the last invocation of cb.
205
2062.7. Multiple events allowed per fd
207
208  Older versions of Libevent allowed at most one EV_READ event and at most
209  one EV_WRITE event per socket, per event base.  This restriction is no
210  longer present.
211
2122.8. evthread_* functions for thread-safe structures.
213
214  Libevent structures can now be built with locking support.  This code
215  makes it safe to add, remove, and activate events on an event base from a
216  different thread.  (Previously, if you wanted to write multithreaded code
217  with Libevent, you could only an event_base or its events in one thread at
218  a time.)
219
220  If you want threading support and you're using pthreads, you can just
221  call evthread_use_pthreads().  (You'll need to link against the
222  libevent_pthreads library in addition to libevent_core.  These functions are
223  not in libevent_core.)
224
225  If you want threading support and you're using Windows, you can just
226  call evthread_use_windows_threads().
227
228  If you are using some locking system besides Windows and pthreads, You
229  can enable this on a per-event-base level by writing functions to
230  implement mutexes, conditions, and thread IDs, and passing them to
231  evthread_set_lock_callbacks and related functions in event2/thread.h.
232
233  Once locking functions are enabled, every new event_base is created with a
234  lock.  You can prevent a single event_base from being built with a lock
235  disabled by using the EVENT_BASE_FLAG_NOLOCK flag in its
236  event_config.  If an event_base is created with a lock, it is safe to call
237  event_del, event_add, and event_active on its events from any thread.  The
238  event callbacks themselves are still all executed from the thread running
239  the event loop.
240
241  To make an evbuffer or a bufferevent object threadsafe, call its
242  *_enable_locking() function.
243
244  The HTTP api is not currently threadsafe.
245
246  To build Libevent with threading support disabled, pass
247  --disable-thread-support to the configure script.
248
2492.9. Edge-triggered events on some backends.
250
251  With some backends, it's now possible to add the EV_ET flag to an event
252  in order to request that the event's semantics be edge-triggered.  Right
253  now, epoll and kqueue support this.
254
255  The corresponding event_config feature is EV_FEATURE_ET; see 2.4 for more
256  information.
257
2582.10. Better support for huge numbers of timeouts
259
260  The heap-based priority queue timer implementation for Libevent 1.4 is good
261  for randomly distributed timeouts, but suboptimal if you have huge numbers
262  of timeouts that all expire in the same amount of time after their
263  creation.  The new event_base_init_common_timeout() logic lets you signal
264  that a given timeout interval will be very common, and should use a linked
265  list implementation instead of a priority queue.
266
2672.11. Improved debugging support
268
269  It's been pretty easy to forget to delete all your events before you
270  re-initialize them, or otherwise put Libevent in an internally inconsistent
271  state.  You can tell libevent to catch these and other common errors with
272  the new event_enable_debug_mode() call.  Just invoke it before you do
273  any calls to other libevent functions, and it'll catch many common
274  event-level errors in your code.
275
2762.12. Functions to access all event fields
277
278  So that you don't have to access the struct event fields directly, Libevent
279  now provides accessor functions to retrieve everything from an event that
280  you set during event_new() or event_assign().
281
2823. Backend-specific and performance improvements.
283
2843.1. Change-minimization on O(1) backends
285
286  With previous versions of Libevent, if you called event_del() and
287  event_add() repeatedly on a single event between trips to the backend's
288  dispatch function, the backend might wind up making unnecessary calls or
289  passing unnecessary data to the kernel.  The new backend logic batches up
290  redundant adds and deletes, and performs no more operations than necessary
291  at the kernel level.
292
293  This logic is on for the kqueue backend, and available (but off by
294  default) for the epoll backend.  To turn it on for the epoll backend,
295  set the EVENT_BASE_FLAG_EPOLL_USE_CHANGELIST flag in the
296  event_base_cofig, or set the EVENT_EPOLL_USE_CHANGELIST environment
297  variable.  Doing this with epoll may result in weird bugs if you give
298  any fds closed by dup() or its variants.
299
3003.2. Improved notification on Linux
301
302  When we need to wake the event loop up from another thread, we use
303  an epollfd to do so, instead of a socketpair.  This is supposed to be
304  faster.
305
3063.3. Windows: better support for everything
307
308  Bufferevents on Windows can use a new mechanism (off-by-default; see below)
309  to send their data via Windows overlapped IO and get their notifications
310  via the IOCP API.  This should be much faster than using event-based
311  notification.
312
313  Other functions throughout the code have been fixed to work more
314  consistently with Windows.  Libevent now builds on Windows using either
315  mingw, or using MSVC (with nmake).  Libevent works fine with UNICODE
316  defined, or not.
317
318  Data structures are a little smarter: our lookups from socket to pending
319  event are now done with O(1) hash tables rather than O(lg n) red-black
320  trees.
321
322  Unfortunately, the main Windows backend is still select()-based: from
323  testing the IOCP backends on the mailing list, it seems that there isn't
324  actually a way to tell for certain whether a socket is writable with IOCP.
325  Libevent 2.1 may add a multithreaded WaitForMultipleEvents-based
326  backend for better performance with many inactive sockets and better
327  integration with Windows events.
328
3294. Improvements to evbuffers
330
331  Libevent has long had an "evbuffer" implementation to wrap access to an
332  input or output memory buffer.  In previous versions, the implementation
333  was very inefficient and lacked some desirable features.  We've made many
334  improvements in Libevent 2.0.
335
3364.1. Chunked-memory internal representation
337
338  Previously, each evbuffer was a huge chunk of memory.  When we ran out of
339  space in an evbuffer, we used realloc() to grow the chunk of memory.  When
340  data was misaligned, we used memmove to move the data back to the front
341  of the buffer.
342
343  Needless to say, this is a terrible interface for networked IO.
344
345  Now, evbuffers are implemented as a linked list of memory chunks, like
346  most Unix kernels use for network IO.  (See Linux's skbuf interfaces,
347  or *BSD's mbufs).  Data is added at the end of the linked list and
348  removed from the front, so that we don't ever need realloc huge chunks
349  or memmove the whole buffer contents.
350
351  To avoid excessive calls to read and write, we use the readv/writev
352  interfaces (or WSASend/WSARecv on Windows) to do IO on multiple chunks at
353  once with a single system call.
354
355  COMPATIBILITY NOTE:
356  The evbuffer struct is no longer exposed in a header.  The code here is
357  too volatile to expose an official evbuffer structure, and there was never
358  any means provided to create an evbuffer except via evbuffer_new which
359  heap-allocated the buffer.
360
361  If you need access to the whole bufer as a linear chunk of memory, the
362  EVBUFFER_DATA() function still works.  Watch out, though: it needs to copy
363  the buffer's contents in a linear chunk before you can use it.
364
3654.2. More flexible readline support
366
367  The old evbuffer_readline() function (which accepted any sequence of
368  CR and LF characters as a newline, and which couldn't handle lines
369  containing NUL characters), is now deprecated.  The preferred
370  function is evbuffer_readln(), which supports a variety of
371  line-ending styles, and which can return the number of characters in
372  the line returned.
373
374  You can also call evbuffer_search_eol() to find the end of a line
375  in an evbuffer without ever extracting the line.
376
3774.3. Support for file-based IO in evbuffers.
378
379  You can now add chunks of a file into a evbuffer, and Libevent will have
380  your OS use mapped-memory functionality, sendfile, or splice to transfer
381  the data without ever copying it to userspace.  On OSs where this is not
382  supported, Libevent just loads the data.
383
384  There are probably some bugs remaining in this code.  On some platforms
385  (like Windows), it just reads the relevant parts of the file into RAM.
386
3874.4. Support for zero-copy ("scatter/gather") writes in evbuffers.
388
389  You can add a piece of memory to an evbuffer without copying it.
390  Instead, Libevent adds a new element to the evbuffer's linked list of
391  chunks with a pointer to the memory you supplied.  You can do this
392  either with a reference-counted chunk (via evbuffer_add_reference), or
393  by asking Libevent for a pointer to its internal vectors (via
394  evbuffer_reserve_space or evbuffer_peek()).
395
3964.5. Multiple callbacks per evbuffer
397
398  Previously, you could only have one callback active on an evbuffer at a
399  time.  In practice, this meant that if one part of Libevent was using an
400  evbuffer callback to notice when an internal evbuffer was reading or
401  writing data, you couldn't have your own callback on that evbuffer.
402
403  Now, you can now use the evbuffer_add_cb() function to add a callback that
404  does not interfere with any other callbacks.
405
406  The evbuffer_setcb() function is now deprecated.
407
4084.6. New callback interface
409
410  Previously, evbuffer callbacks were invoked with the old size of the
411  buffer and the new size of the buffer.  This interface could not capture
412  operations that simultaneously filled _and_ drained a buffer, or handle
413  cases where we needed to postpone callbacks until multiple operations were
414  complete.
415
416  Callbacks that are set with evbuffer_setcb still use the old API.
417  Callbacks added with evbuffer_add_cb() use a new interface that takes a
418  pointer to a struct holding the total number of bytes drained read and the
419  total number of bytes written.  See event2/buffer.h for full details.
420
4214.7. Misc new evbuffer features
422
423   You can use evbuffer_remove() to move a given number of bytes from one
424   buffer to another.
425
426   The evbuffer_search() function lets you search for repeated instances of
427   a pattern inside an evbuffer.
428
429   You can use evbuffer_freeze() to temporarily suspend drains from or adds
430   to a given evbuffer.  This is useful for code that exposes an evbuffer as
431   part of its public API, but wants users to treat it as a pure source or
432   sink.
433
434   There's an evbuffer_copyout() that looks at the data at the start of an
435   evbuffer without doing a drain.
436
437   You can have an evbuffer defer all of its callbacks, so that rather than
438   being invoked immediately when the evbuffer's length changes, they are
439   invoked from within the event_loop.  This is useful when you have a
440   complex set of callbacks that can change the length of other evbuffers,
441   and you want to avoid having them recurse and overflow your stack.
442
4435. Bufferevents improvements
444
445   Libevent has long included a "bufferevents" structure and related
446   functions that were useful for generic buffered IO on a TCP connection.
447   This is what Libevent uses for its HTTP implementation.  In addition to
448   the improvements that they get for free from the underlying evbuffer
449   implementation above, there are many new features in Libevent 2.0's
450   evbuffers.
451
4525.1. New OO implementations
453
454   The "bufferevent" structure is now an abstract base type with multiple
455   implementations.  This should not break existing code, which always
456   allocated bufferevents with bufferevent_new().
457
458   Current implementations of the bufferevent interface are described below.
459
4605.2. bufferevent_socket_new() replaces bufferevent_new()
461
462   Since bufferevents that use a socket are not the only kind,
463   bufferevent_new() is now deprecated.  Use bufferevent_socket_new()
464   instead.
465
4665.3. Filtered bufferevent IO
467
468   You can use bufferevent_filter_new() to create a bufferevent that wraps
469   around another bufferevent and transforms data it is sending and
470   receiving.  See test/regress_zlib.c for a toy example that uses zlib to
471   compress data before sending it over a bufferevent.
472
4735.3. Linked pairs of bufferevents
474
475   You can use bufferevent_pair_new() to produce two linked
476   bufferevents.  This is like using socketpair, but doesn't require
477   system-calls.
478
4795.4. SSL support for bufferevents with OpenSSL
480
481   There is now a bufferevent type that supports SSL/TLS using the
482   OpenSSL library.  The code for this is build in a separate
483   library, libevent_openssl, so that your programs don't need to
484   link against OpenSSL unless they actually want SSL support.
485
486   There are two ways to construct one of these bufferevents, both
487   declared in <event2/bufferevent_ssl.h>.  If you want to wrap an
488   SSL layer around an existing bufferevent, you would call the
489   bufferevent_openssl_filter_new() function.  If you want to do SSL
490   on a socket directly, call bufferevent_openssl_socket_new().
491
4925.5. IOCP support for bufferevents on Windows
493
494   There is now a bufferevents backend that supports IOCP on Windows.
495   Supposedly, this will eventually make Windows IO much faster for
496   programs using bufferevents.  We'll have to see; the code is not
497   currently optimized at all.  To try it out, call the
498   event_base_start_iocp() method on an event_base before contructing
499   bufferevents.
500
501   This is tricky code; there are probably some bugs hiding here.
502
5035.6. Improved connect support for bufferevents.
504
505   You can now create a bufferevent that is not yet connected to any
506   host, and tell it to connect, either by address or by hostname.
507
508   The functions to do this are bufferevent_socket_connect and
509   bufferevent_socket_connect_hostname.
510
5115.7. Rate-limiting for bufferevents
512
513   If you need to limit the number of bytes read/written by a single
514   bufferevent, or by a group of them, you can do this with a new set of
515   bufferevent rate-limiting calls.
516
5176. Other improvements
518
5196.1. DNS improvements
520
5216.1.1. DNS: IPv6 nameservers
522
523   The evdns code now lets you have nameservers whose addresses are IPv6.
524
5256.1.2. DNS: Better security
526
527   Libevent 2.0 tries harder to resist DNS answer-sniping attacks than
528   earlier versions of evdns.  See comments in the code for full details.
529
530   Notably, evdns now supports the "0x20 hack" to make it harder to
531   impersonate a DNS server.  Additionally, Libevent now uses a strong
532   internal RNG to generate DNS transaction IDs, so you don't need to supply
533   your own.
534
5356.1.3. DNS: Getaddrinfo support
536
537   There's now an asynchronous getaddrinfo clone, evdns_getaddrinfo(),
538   to make the results of the evdns functions more usable.  It doesn't
539   support every feature of a typical platform getaddrinfo() yet, but it
540   is quite close.
541
542   There is also a blocking evutil_getaddrinfo() declared in
543   event2/util.h, to provide a getaddrinfo() implementation for
544   platforms that don't have one, and smooth over the differences in
545   various platforms implementations of RFC3493.
546
547   Bufferevents provide bufferevent_connect_hostname(), which combines
548   the name lookup and connect operations.
549
5506.1.4. DNS: No more evdns globals
551
552   Like an event base, evdns operations are now supposed to use an evdns_base
553   argument.  This makes them easier to wrap for other (more OO) languages,
554   and easier to control the lifetime of.  The old evdns functions will
555   still, of course, continue working.
556
5576.2. Listener support
558
559   You can now more easily automate setting up a bound socket to listen for
560   TCP connections.  Just use the evconnlistener_*() functions in the
561   event2/listener.h header.
562
563   The listener code supports IOCP on Windows if available.
564
5656.3. Secure RNG support
566
567   Network code very frequently needs a secure, hard-to-predict random number
568   generator.  Some operating systems provide a good C implementation of one;
569   others do not.  Libevent 2.0 now provides a consistent implementation
570   based on the arc4random code originally from OpenBSD.  Libevent (and you)
571   can use the evutil_secure_rng_*() functions to access a fairly secure
572   random stream of bytes.
573
5746.4. HTTP
575
576   The evhttp uriencoding and uridecoding APIs have updated versions
577   that behave more correctly, and can handle strings with internal NULs.
578
579   The evhttp query parsing and URI parsing logic can now detect errors
580   more usefully.  Moreover, we include an actual URI parsing function
581   (evhttp_uri_parse()) to correctly parse URIs, so as to discourage
582   people from rolling their own ad-hoc parsing functions.
583
584   There are now accessor functions for the useful fields of struct http
585   and friends; it shouldn't be necessary to access them directly any
586   more.
587
588   Libevent now lets you declare support for all specified HTTP methods,
589   including OPTIONS, PATCH, and so on.  The default list is unchanged.
590
591   Numerous evhttp bugs also got fixed.
592
5937. Infrastructure improvements
594
5957.1. Better unit test framework
596
597   We now use a unit test framework that Nick wrote called "tinytest".
598   The main benefit from Libevent's point of view is that tests which
599   might mess with global state can all run each in their own
600   subprocess.  This way, when there's a bug that makes one unit test
601   crash or mess up global state, it doesn't affect any others.
602
6037.2. Better unit tests
604
605   Despite all the code we've added, our unit tests are much better than
606   before.  Right now, iterating over the different backends on various
607   platforms, I'm getting between 78% and 81% test coverage, compared
608   with less than 45% test coverage in Libevent 1.4.
609
610