#
95ee2897 |
|
16-Aug-2023 |
Warner Losh <imp@FreeBSD.org> |
sys: Remove $FreeBSD$: two-line .h pattern Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/
|
#
208fcb55 |
|
13-Jul-2023 |
Kristof Provost <kp@FreeBSD.org> |
Fix MINIMAL build on amd64 amd64/include/counter.h uses KASSERT, but failed to include the kassert.h header.
|
#
4d846d26 |
|
10-May-2023 |
Warner Losh <imp@FreeBSD.org> |
spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch up to that fact and revert to their recommended match of BSD-2-Clause. Discussed with: pfg MFC After: 3 days Sponsored by: Netflix
|
#
8dc3fdfe |
|
13-Jun-2022 |
Brooks Davis <brooks@FreeBSD.org> |
amd64: -m32 support for machine/counter.h Install the i386 counter.h under /usr/include/i386 on amd64 and include when targeting i386. This is a kernel-only header required by procstat's ZFS support. Reviewed by: jhb, imp
|
#
2318ed25 |
|
12-Feb-2020 |
Mateusz Guzik <mjg@FreeBSD.org> |
amd64: provide custom zpcpu set/add/sub routines Note that clobbers are highly overzealous, can be cleaned up later.
|
#
fb886947 |
|
12-Feb-2020 |
Mateusz Guzik <mjg@FreeBSD.org> |
amd64: store per-cpu allocations subtracted by __pcpu This eliminates a runtime subtraction from counter_u64_add. before: mov 0x4f00ed(%rip),%rax # 0xffffffff80c01788 <numfullpathfail4> sub 0x808ff6(%rip),%rax # 0xffffffff80f1a698 <__pcpu> addq $0x1,%gs:(%rax) after: mov 0x4f02fd(%rip),%rax # 0xffffffff80c01788 <numfullpathfail4> addq $0x1,%gs:(%rax) Reviewed by: jeff Differential Revision: https://reviews.freebsd.org/D23570
|
#
e2b81f51 |
|
07-Feb-2020 |
Mateusz Guzik <mjg@FreeBSD.org> |
amd64: clean up counter(9) - stop open-coding access to per-cpu data, use common macros instead - consistently use counter_t type where appropriate
|
#
13189065 |
|
24-Nov-2019 |
Konstantin Belousov <kib@FreeBSD.org> |
amd64: assert that EARLY_COUNTER does not corrupt memory. Reviewed by: imp Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential revision: https://reviews.freebsd.org/D22514
|
#
a2a0f906 |
|
29-Aug-2019 |
Konstantin Belousov <kib@FreeBSD.org> |
Centralize __pcpu definitions. Many extern struct pcpu <something>__pcpu declarations were copied/pasted in sources. The issue is that the definition is MD, but it cannot be provided by machine/pcpu.h due to actual struct pcpu defined in sys/pcpu.h later than the inclusion of machine/pcpu.h. This forced the copying when other code needed direct access to __pcpu. There is no way around it, due to machine/pcpu.h supplying part of struct pcpu fields. To work around the problem, add a new machine/pcpu_aux.h header, which should fill any needed MD definitions after struct pcpu definition is completed. This allows to remove copies of __pcpu spread around the source. Also on x86 it makes it possible to remove work arounds like OFFSETOF_CURTHREAD or clang specific warnings supressions. Reported and tested by: lwhsu, bcran Reviewed by: imp, markj (previous version) Discussed with: jhb Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D21418
|
#
10ff5eeb |
|
24-Aug-2019 |
Konstantin Belousov <kib@FreeBSD.org> |
amd64: rework PCPU allocation Move pcpu KVA out of .bss into dynamically allocated VA at pmap_bootstrap(). This avoids demoting superpage mapping .data/.bss. Also it makes possible to use pmap_qenter() for installation of domain-local pcpu page on NUMA configs. Refactor pcpu and IST initialization by moving it to helper functions. Reviewed by: markj Tested by: pho Discussed with: jeff Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D21320
|
#
53dec71d |
|
06-Jul-2018 |
Konstantin Belousov <kib@FreeBSD.org> |
Expand x86 struct pcpus to UMA_PCPU_ALLOC_SIZE AKA PAGE_SIZE. This restores counters(9) operation. Revert r336024. Improve assert of pcpu size on x86. Reviewed by: mmacy Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D16163
|
#
fb0a2811 |
|
06-Jul-2018 |
Konstantin Belousov <kib@FreeBSD.org> |
Revert to recommit with the proper message.
|
#
16147166 |
|
06-Jul-2018 |
Konstantin Belousov <kib@FreeBSD.org> |
Save a call to pmap_remove() if entry cannot have any pages mapped. Due to the way rtld creates mappings for the shared objects, each dso causes unmap of at least three guard map entries. For instance, in the buildworld load, this change reduces the amount of pmap_remove() calls by 1/5. Profiled by: alc Reviewed by: alc, markj Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential revision: https://reviews.freebsd.org/D16148
|
#
428194fe |
|
06-Jul-2018 |
Matt Macy <mmacy@FreeBSD.org> |
counter(9): unbreak amd64 following r336020 Apply temporary fix to counter until daylight hours. The fact that the assembly for counter_u64_add relied on the sizeof(struct pcpu) was the basis for the otherwise arbitrary offset never came up in D15933. critical_{enter,exit} is now inline so the only real added overhead is the added (mostly false) conditional branch in exit.
|
#
ab3059a8 |
|
05-Jul-2018 |
Matt Macy <mmacy@FreeBSD.org> |
Back pcpu zone with domain correct pages - Change pcpu zone consumers to use a stride size of PAGE_SIZE. (defined as UMA_PCPU_ALLOC_SIZE to make future identification easier) - Allocate page from the correct domain for a given cpu. - Don't initialize pc_domain to non-zero value if NUMA is not defined There are some misconceptions surrounding this field. It is the _VM_ NUMA domain and should only ever correspond to valid domain values as understood by the VM. The former slab size of sizeof(struct pcpu) was somewhat arbitrary. The new value is PAGE_SIZE because that's the smallest granularity which the VM can allocate a slab for a given domain. If you have fewer than PAGE_SIZE/8 counters on your system there will be some memory wasted, but this is obviously something where you want the cache line to be coming from the correct domain. Reviewed by: jeff Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15933
|
#
c49761dd |
|
27-Nov-2017 |
Pedro F. Giffuni <pfg@FreeBSD.org> |
sys/amd64: further adoption of SPDX licensing ID tags. Mainly focus on files that use BSD 2-Clause license, however the tool I was using misidentified many licenses so this was mostly a manual - error prone - task. The Software Package Data Exchange (SPDX) group provides a specification to make it easier for automated tools to detect and summarize well known opensource licenses. We are gradually adopting the specification, noting that the tags are considered only advisory and do not, in any way, superceed or replace the license texts.
|
#
83c9dea1 |
|
17-Apr-2017 |
Gleb Smirnoff <glebius@FreeBSD.org> |
- Remove 'struct vmmeter' from 'struct pcpu', leaving only global vmmeter in place. To do per-cpu stats, convert all fields that previously were maintained in the vmmeters that sit in pcpus to counter(9). - Since some vmmeter stats may be touched at very early stages of boot, before we have set up UMA and we can do counter_u64_alloc(), provide an early counter mechanism: o Leave one spare uint64_t in struct pcpu, named pc_early_dummy_counter. o Point counter(9) fields of vmmeter to pcpu[0].pc_early_dummy_counter, so that at early stages of boot, before counters are allocated we already point to a counter that can be safely written to. o For sparc64 that required a whole dummy pcpu[MAXCPU] array. Further related changes: - Don't include vmmeter.h into pcpu.h. - vm.stats.vm.v_swappgsout and vm.stats.vm.v_swappgsin changed to 64-bit, to match kernel representation. - struct vmmeter hidden under _KERNEL, and only vmstat(1) is an exclusion. This is based on benno@'s 4-year old patch: https://lists.freebsd.org/pipermail/freebsd-arch/2013-July/014471.html Reviewed by: kib, gallatin, marius, lidl Differential Revision: https://reviews.freebsd.org/D10156
|
#
67d955aa |
|
08-Apr-2017 |
Patrick Kelsey <pkelsey@FreeBSD.org> |
Corrected misspelled versions of rendezvous. The MFC will include a compat definition of smp_no_rendevous_barrier() that calls smp_no_rendezvous_barrier(). Reviewed by: gnn, kib MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D10313
|
#
96c85efb |
|
06-Jul-2016 |
Nathan Whitehorn <nwhitehorn@FreeBSD.org> |
Replace a number of conflations of mp_ncpus and mp_maxid with either mp_maxid or CPU_FOREACH() as appropriate. This fixes a number of places in the kernel that assumed CPU IDs are dense in [0, mp_ncpus) and would try, for example, to run tasks on CPUs that did not exist or to allocate too few buffers on systems with sparse CPU IDs in which there are holes in the range and mp_maxid > mp_ncpus. Such circumstances generally occur on systems with SMT, but on which SMT is disabled. This patch restores system operation at least on POWER8 systems configured in this way. There are a number of other places in the kernel with potential problems in these situations, but where sparse CPU IDs are not currently known to occur, mostly in the ARM machine-dependent code. These will be fixed in a follow-up commit after the stable/11 branch. PR: kern/210106 Reviewed by: jhb Approved by: re (glebius)
|
#
70a7dd5d |
|
30-Jun-2013 |
Konstantin Belousov <kib@FreeBSD.org> |
Fix issues with zeroing and fetching the counters, on x86 and ppc64. Issues were noted by Bruce Evans and are present on all architectures. On i386, a counter fetch should use atomic read of 64bit value, otherwise carry from the increment on other CPU could be lost for the given fetch, making error of 2^32. If 64bit read (cmpxchg8b) is not available on the machine, it cannot be SMP and it is enough to disable preemption around read to avoid the split read. On x86 the counter increment is not atomic on purpose, which makes it possible for the store of the incremented result to override just zeroed per-cpu slot. The effect would be a counter going off by arbitrary value after zeroing. Perform the counter zeroing on the same processor which does the increments, making the operations mutually exclusive. On i386, same as for the fetching, if the cmpxchg8b is not available, machine is not SMP and we disable preemption for zeroing. PowerPC64 is treated the same as amd64. For other architectures, the changes made to allow the compilation to succeed, without fixing the issues with zeroing or fetching. It should be possible to handle them by using the 64bit loads and stores atomic WRT preemption (assuming the architectures also converted from using critical sections to proper asm). If architecture does not provide the facility, using global (spin) mutex would be non-optimal but working solution. Noted by: bde Sponsored by: The FreeBSD Foundation
|
#
9dbb63fe |
|
20-Jun-2013 |
Konstantin Belousov <kib@FreeBSD.org> |
Allow immediate operand. Sponsored by: The FreeBSD Foundation
|
#
4e76af6a |
|
08-Apr-2013 |
Gleb Smirnoff <glebius@FreeBSD.org> |
Merge from projects/counters: counter(9). Introduce counter(9) API, that implements fast and raceless counters, provided (but not limited to) for gathering of statistical data. See http://lists.freebsd.org/pipermail/freebsd-arch/2013-April/014204.html for more details. In collaboration with: kib Reviewed by: luigi Tested by: ae, ray Sponsored by: Nginx, Inc.
|