- Copy stable/10 (r259064) to releng/10.0 as part of the
10.0-RELEASE cycle.
- Update __FreeBSD_version [1]
- Set branch name to -RC1

[1] 10.0-CURRENT __FreeBSD_version value ended at '55', so
start releng/10.0 at '100' so the branch is started with
a value ending in zero.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Import jemalloc 9ef7f5dc34ff02f50d401e41c8d9a4a928e7c2aa (dev branch,
prior to 3.0.0 release) as contrib/jemalloc, and integrate it into libc.
The code being imported by this commit diverged from
lib/libc/stdlib/malloc.c in March 2010, which means that a portion of
the jemalloc 1.0.0 ChangeLog entries are relevant, as are the entries
for all subsequent releases.

Correct a typo in the malloc(3) manpage. Malloc options are set in the
MALLOC_OPTIONS environment variable, not JEMALLOC_OPTIONS.

Reviewed by: jasone
Approved by: emaste (mentor)

mdoc: drop redundant .Pp and .LP calls

They have no effect when coming in pairs, or before .Bl/.Bd

Revert changes of 'assure' to 'ensure' made in r211936.

Approved by: rrs (mentor)

Fix incorrect usage of 'assure' and 'insure'.

Approved by: rrs (mentor)

Fix typos and spelling mistakes.

Fix bugs:

* Fix a race in chunk_dealloc_dss().

* Check for allocation failure before zeroing memory in base_calloc().

Merge enhancements from a divergent version of jemalloc:

* Convert thread-specific caching from magazines to an algorithm that is
more tunable, and implement incremental GC.

* Add support for medium size classes, [4KiB..32KiB], 2KiB apart by
default.

* Add dirty page tracking for pages within active small/medium object
runs. This allows malloc to track precisely which pages are in active
use, which makes dirty page purging more effective.

* Base maximum dirty page count on proportion of active memory.

* Use optional zeroing in arena_chunk_alloc() to avoid needless zeroing
of chunks. This is useful in the context of DSS allocation, since a
long-lived application may commonly recycle chunks.

* Increase the default chunk size from 1MiB to 4MiB.

Remove feature:

* Remove the dynamic rebalancing code, since thread caching reduces its
utility.

Make malloc(3) superpage aware. Specifically, if getpagesizes(3) returns
a large page size that is greater than malloc(3)'s default chunk size but
less than or equal to 4 MB, then increase the chunk size to match the large
page size.

Most often, using a chunk size that is less than the large page size is not
a problem. However, consider a long-running application that allocates and
frees significant amounts of memory. In particular, it frees enough memory
at times that some of that memory is munmap()ed. Up until the first
munmap(), a 1MB chunk size is just fine; it's not a problem for the virtual
memory system. Two adjacent 1MB chunks that are aligned on a 2MB boundary
will be promoted automatically to a superpage even though they were
allocated at different times. The trouble begins with the munmap(),
releasing a 1MB chunk will trigger the demotion of the containing superpage,
leaving behind a half-used 2MB reservation. Now comes the real problem.
Unfortunately, when the application needs to allocate more memory, and it
recycles the previously munmap()ed address range, the implementation of
mmap() won't be able to reuse the reservation. Basically, the coalescing
rules in the virtual memory system don't allow this new range to combine
with its neighbor. The effect being that superpage promotion will not
reoccur for this range of addresses until both 1MB chunks are freed at some
point in the future.

Reviewed by: jasone
MFC after: 3 weeks

Revert to preferring mmap(2) over sbrk(2) when mapping memory, due to
potential extreme contention in the kernel for multi-threaded applications
on SMP systems.

Reported by: kris

Add thread-specific caching for small size classes, based on magazines.
This caching allows for completely lock-free allocation/deallocation in the
steady state, at the expense of likely increased memory use and
fragmentation.

Reduce the default number of arenas to 2*ncpus, since thread-specific
caching typically reduces arena contention.

Modify size class spacing to include ranges of 2^n-spaced, quantum-spaced,
cacheline-spaced, and subpage-spaced size classes. The advantages are:
fewer size classes, reduced false cacheline sharing, and reduced internal
fragmentation for allocations that are slightly over 512, 1024, etc.

Increase RUN_MAX_SMALL, in order to limit fragmentation for the
subpage-spaced size classes.

Add a size-->bin lookup table for small sizes to simplify translating sizes
to size classes. Include a hard-coded constant table that is used unless
custom size class spacing is specified at run time.

Add the ability to disable tiny size classes at compile time via
MALLOC_TINY.

Remove support for lazy deallocation. Benchmarks across a wide range of
allocation patterns, number of CPUs, and MALLOC_OPTIONS settings indicate
that lazy deallocation has the potential to worsen throughput dramatically.
Performance degradation occurs when multiple threads try to clear the lazy
free cache simultaneously. Various experiments to avoid this bottleneck
failed to completely solve this problem, while adding yet more complexity.

Track dirty unused pages so that they can be purged if they exceed a
threshold, according to the 'F' MALLOC_OPTIONS flag. This obsoletes the
'H' flag.

Try to realloc() large objects in place. This substantially speeds up
incremental large reallocations in the common case.

Fix a bug in arena_ralloc() that caused relocation of sub-page objects
even if the old and new sizes were in the same size class.

Maintain trees of runs and simplify the per-chunk page map. This allows
logarithmic-time searching for sufficiently large runs in
arena_run_alloc(), whereas the previous algorithm required linear time
in the worst case.

Break various large functions into smaller sub-functions, and inline
only the functions that are in the fast path for small object
allocation/deallocation.

Remove an unnecessary check in base_pages_alloc_mmap().

Avoid integer division in choose_arena() for the NO_TLS case on
single-CPU systems.

Enable both sbrk(2)- and mmap(2)-based memory acquisition methods by
default. This has the disadvantage of rendering the datasize resource
limit irrelevant, but without this change, legitimate uses of more
memory than will fit in the data segment are thwarted by default.

Fix chunk_alloc_mmap() to work correctly if initial mapping is not
chunk-aligned and mapping extension fails.

Add the 'D' and 'M' run time options, and use them to control whether
memory is acquired from the system via sbrk(2) and/or mmap(2). By default,
use sbrk(2) only, in order to support traditional use of resource limits.
Additionally, when both options are enabled, prefer the data segment to
anonymous mappings, in order to coexist better with large file mappings
in applications on 32-bit platforms. This change has the potential to
increase memory fragmentation due to the linear nature of the data
segment, but from a performance perspective this is mitigated by the use
of madvise(2). [1]

Add the ability to interpret integer prefixes in MALLOC_OPTIONS
processing. For example, MALLOC_OPTIONS=lllllllll can now be specified as
MALLOC_OPTIONS=9l.

Reported by: [1] rwatson
Design review: [1] alc, peter, rwatson

Document the B and L MALLOC_OPTIONS.

Add information about the implications of using mmap(2) instead of sbrk(2).

Submitted by: bmah, jhb

Update the IMPLEMENTATION NOTES section to reflect recent malloc
enhancements.

Markup fixes.

s/soley/solely

Add malloc_usable_size() to the RETURN VALUES section.

Add malloc_usable_size(3).

Discussed with: arch@

Modify allocation policy, in order to avoid excessive fragmentation for
allocation patterns that involve a relatively even mixture of many
different size classes.

Reduce the chunk size from 16 MB to 2 MB. Since chunks are now carved up
using an address-ordered first best fit policy, VM map fragmentation is
much less likely, which makes smaller chunks not as much of a risk. This
reduces the virtual memory size of most applications.

Remove redzones, since program buffer overruns are no longer as likely to
corrupt malloc data structures.

Remove the C MALLOC_OPTIONS flag, and add H and S.

Expand contractions.

Replace malloc(), calloc(), posix_memalign(), realloc(), and free() with
a scalable concurrent allocator implementation.

Reviewed by: current@
Approved by: phk, markm (mentor)

Fix prototype.

Sort sections.

Reword recent addition about memory moving.

Requested by: keramida

Bump .Dd

Requested by: ru

Clarify that realloc and reallocf may move the memory allocation.

mdoc(7) police: Deal with self-xrefs.

Document what really occurs when we obtain an error.

PR: 43357
Submitted by: David Schultz <dschultz@uclink.Berkeley.EDU>

mdoc(7) police: Fixed abuses of the .Ar and .Em macros.

mdoc(7) police: "The .Fn function".

Uniformly refer to a file system as "file system".

Approved by: re

Consistently mark std(in|out|err) with .Dv, because that's how they
are marked up in stdio(3), and because they are defined expressions
of type "FILE *".

Approved by: re

Clarify the bit about realloc() and its `ptr' argument a bit.
Hopefully, now it is more clear that the memory referenced by the
ptr argument of realloc(ptr,size) is freed and only the return value
of realloc() points to a valid memory area upon successful completion.

Submitted by: Martin Faxer <gmh003532@brfmasthugget.se>

mdoc(7) police: nit.

Constify _malloc_options.

mdoc(7) police: kill HSBs, add missing comma.

Be more explicit about the fact that realloc() might return a
different pointer than the one passed to it.

PR: docs/31925
Submitted by: Andrew <andrew@ugh.net.au>

malloc and calloc do not free memory.

PR: 31365
Submitted by: SUZUKI Koichi <koich@cac.co.jp>

Move to using .In instead of .Fd #include <> for include mark-up.

Inspired by comment from: dd

Markup nits: use diagnostic type lists for error and warning messages.

Backout previous revision. We should not expand plain text xrefs if
they appear in the literal text, e.g. in the error or warning message
of the library function. (Submitted by: bde)

Moved "out of memory" from warning to errors section.

mdoc(7) police: expand plain text xrefs.

Remove whitespace at EOL.

mdoc(7) police: removed HISTORY info from the .Os call.

mdoc(7) police: remove extraneous .Pp before and/or after .Sh.

mdoc(7) police: mark NULL with .Dv.

mdoc(7) police: the BUGS section should go after HISTORY and AUTHORS.

The fact that phk wrote this is not a bug!

Reword a sentence to make it proper English.

.St -ansiC -> .St -isoC

mmap(3) -> mmap(2).

.Xr to mmap.

mdoc(7) police: Change -filled displays (which just happen
to be the same as -ragged in the current implementation) to
-ragged. With mdocNG, -filled displays produce the correct
output, formatted and justified to both margins.

mdoc(7) police: split punctuation characters + misc fixes.

Prepare for mdoc(7)NG.

Prepare for mdoc(7)NG.

mdoc(7) police: removed history info from the .Os FreeBSD call.

Make it possible to override the function which writes messages to
stderr in case of warnings and errors.

Rename malloc_options to have a leading underscore, I belive I have been
told that is more correct namespace wise.

Use Fx macro wherever possible.

Minor mdoc cleanup.

PR: docs/13218

Introduce ".Lb" macro to libc manpages.

More libraries manpages updates following.

Add a missing THREAD_UNLOCK() found missing by Valentin Nechayev
<netch@segfault.kiev.ua>

Remove allocation failure check from 'A' option, the 'X' option does
this as a standalone check now.

Add history: The reallocf() function first appeared in FreeBSD-3.0.

See imp's 199808201619.KAA20970@harmony.village.org in freebsd-hackers (the
reallocf.c cvs history mistakenly refers to freebsd-current).

Document the current behaviour with respect to the handling of errno.

Approved by: phk

Go to a bit more trouble to make it absolutely clear that malloc(3)
does not zero the allocated memory.

Change .Fn to .Xr on a couple of lines where abort(3) is referenced.

$Id$ -> $FreeBSD$

Various man page cleanup:

- Sort xrefs
- FreeBSD.ORG -> FreeBSD.org
- Be consistent with section names as outlines in mdoc(7)
- Other misc mdoc cleanup.

PR: doc/13144
Submitted by: Alexy M. Zelkin <phantom@cris.net>

Fix .Xr line for `getpagesize'

Disable the "Hint" option in phkmalloc as default. Recent VM system
changes have made this too expensive. This gains about 1.25% on
worldstone on my SMP machine.

Swap-less machines, for instance PicoBSDs, and machines which experience
page-out trafic, check with top(1), will probably want to reenable this
with:
ln -s H /etc/malloc.conf

Suggested by: alc (&dyson ?)

o Don't reference non-existant function freealloc.
o Properly order things

Pointed out by: bde

Add reallocf to the library. This function is simliar to realloc, but
when it returns NULL to indicate failure, it will also free the memory
that was passed to it, if that was non-null.

This does not change the semantics of realloc.

A second commit will be done to commit the conversion of those places in
the code that can safely use this to avoid memory leaks when confronted
with low memory situations.

Beaten-to-death-but-finally-approved-in: -current

environmental -> environment.

Malloc option H is now default.

malloc_sysv used before initialized, reported in PR4002 by
Dmitrij Tejblum <dima@tejblum.dnttm.rssi.ru>

Various cleanup from Keith Bostic

Reinstate calloc() as a separate funtion, in its own source/object file.
leave the manpage integrated with malloc.3 and friends. Too many things
were broken in this respect.

PR: 4002
Reviewed by: phk
Submitted by: Dmitrij Tejblum <dima@tejblum.dnttm.rssi.ru>
Submitted by: Keith Bostic <bostic@bostic.com>

Integrate calloc with the rest of the gang.
Various portability and stylistic cleanups.
Kill MALLOC_STATS & the 'D' option.
Fix the 'V' option.
Major overhaul of the man-page.
You milage should not vary.

Reviewed by: Keith Bostic <bostic@bostic.com>
Submitted by: Keith Bostic <bostic@bostic.com>

Add yet an option, this time on how to deal with malloc(0) and realloc(ptr.0)

Prompted by: X11 & XFree86

Malloc flag X makes malloc behave like the canonical xmalloc() wrapper.

Untested support for Solaris from John-Mark Gurney

Reviewed by: phk
Submitted by: (partially) John-Mark Gurney <gurney_j@resnet.uoregon.edu>

Remove an extra comma.

Revert $FreeBSD$ to $Id$

Make the long-awaited change from $Id$ to $FreeBSD$

This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.

Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.

phkmalloc/3
Various neat features added. More documentation in the manpage.
If your machine has very little RAM, I guess that would be < 16M
these days :-(, you may want to try this:
ln -fs 'H<' /etc/malloc.conf
check the manpage.

Correct a bunch of man page cross references and generally
try and silence "manck".

ncurses, rpc, and some of the gnu stuff are still a big mess, however.

Fix even more spelling errors in some more man pages.

phkmalloc/2
"zero' and 'junk' options to help find and diagnose malloc abuse.
EXTRA_SANITY defaults "junk" to on.
Don't dump the internal state, unless EXTRA_SANITY.
General code cleanup.
Error messages cleaned up a bit, more checking earlier.
EXTRA_SANITY is default at this time (just in case).
Performance (without EXTRA_SANITY) is better, beats gnumalloc in
both time & space most of the time:

# In-memory test.
./malloc 50000000 2000 8192
159.2u 1.5s 2:41.85 99.3% 5+7742k 0+0io 0pf+0w
./gnumalloc 50000000 2000 8192
272.6u 0.4s 4:35.01 99.3% 5+8533k 0+0io 0pf+0w

# Swap-space test.
./malloc 500000 14000 8192
6.5u 4.1s 4:08.87 4.3% 5+49209k 0+0io 9772pf+0w
./gnumalloc 500000 14000 8192
16.2u 14.5s 15:36.14 3.2% 5+54100k 0+0io 47651pf+0w

# Small items test.
./malloc 20000000 20000 2048
67.0u 0.3s 1:07.83 99.2% 5+18199k 0+0io 4pf+0w
./gnumalloc 20000000 20000 2048
66.2u 0.3s 1:07.03 99.3% 5+18107k 0+0io 0pf+0w

SANITY is not an option anymore. (!!)

``phkmalloc''
Performance is comparable to gnumalloc if you have sufficient RAM, and
it screams around it if you don't.
Compiled with "EXTRA_SANITY" until further notice.
see malloc.3 for more details.

This commit was generated by cvs2svn to compensate for changes in r1573,
which included commits to RCS files with non-trunk default branches.

BSD 4.4 Lite Lib Sources