#
22f6e3e1 |
|
14-Jan-2024 |
YiPeng Chai <YiPeng.Chai@amd.com> |
drm/amdgpu: Add log info for umc_v12_0 Add log info for umc_v12_0. v2: Delete redundant logs. Signed-off-by: YiPeng Chai <YiPeng.Chai@amd.com> Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
a2d3c692 |
|
16-Nov-2023 |
David Yat Sin <David.YatSin@amd.com> |
drm/amdkfd: Copy HW exception data to user event Fixes issue where user events of type KFD_EVENT_TYPE_HW_EXCEPTION do not have valid data Signed-off-by: David Yat Sin <David.YatSin@amd.com> Reviewed-by: Lijo Lazar <lijo.lazar@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
c99a2e7a |
|
27-Jul-2023 |
Alex Deucher <alexander.deucher@amd.com> |
drm/amdkfd: drop IOMMUv2 support Now that we use the dGPU path for all APUs, drop the IOMMUv2 support. v2: drop the now unused queue manager functions for gfx7/8 APUs Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Tested-by: Mike Lothian <mike@fireburn.co.uk> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
973fddea |
|
08-Jun-2023 |
James Zhu <James.Zhu@amd.com> |
drm/amdkfd: update user space last_event_age Update user space last_event_age when event age is enabled. It is only for KFD_EVENT_TYPE_SIGNAL which is checked by user space. Signed-off-by: James Zhu <James.Zhu@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
96cdb538 |
|
17-May-2023 |
James Zhu <James.Zhu@amd.com> |
drm/amdkfd: set activated flag true when event age unmatchs Set waiter's activated flag true when event age unmatchs with last_event_age. Signed-off-by: James Zhu <James.Zhu@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
4057e6ce |
|
17-May-2023 |
James Zhu <James.Zhu@amd.com> |
drm/amdkfd: add event_age tracking when receiving interrupt Add event_age tracking when receiving interrupt. Signed-off-by: James Zhu <James.Zhu@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
7386f88a |
|
07-Jun-2023 |
Jonathan Kim <jonathan.kim@amd.com> |
drm/amdkfd: fix vmfault signalling with additional data. Exception handling for vmfaults should be raised with additional data. Reported-by: Mukul Joshi <mukul.joshi@amd.com> Signed-off-by: Jonathan Kim <jonathan.kim@amd.com> Reviewed-by: Mukul Joshi <mukul.joshi@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
c2d2588c |
|
07-Apr-2022 |
Jonathan Kim <jonathan.kim@amd.com> |
drm/amdkfd: add send exception operation Add a debug operation that allows the debugger to send an exception directly to runtime through a payload address. For memory violations, normal vmfault signals will be applied to notify runtime instead after passing in the saved exception data when a memory violation was raised to the debugger. For runtime exceptions, this will unblock the runtime enable function which will be explained and implemented in a follow up patch. Signed-off-by: Jonathan Kim <jonathan.kim@amd.com> Reviewed-by: Felix Kuehling <felix.kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
8dc1db31 |
|
14-Sep-2022 |
Mukul Joshi <mukul.joshi@amd.com> |
drm/amdkfd: Introduce kfd_node struct (v5) Introduce a new structure, kfd_node, which will now represent a compute node. kfd_node is carved out of kfd_dev structure. kfd_dev struct now will become the parent of kfd_node, and will store common resources such as doorbells, GTT sub-alloctor etc. kfd_node struct will store all resources specific to a compute node, such as device queue manager, interrupt handling etc. This is the first step in adding compute partition support in KFD. v2: introduce kfd_node struct to gc v11 (Hawking) v3: make reference to kfd_dev struct through kfd_node (Morris) v4: use kfd_node instead for kfd isr/mqd functions (Morris) v5: rebase (Alex) Signed-off-by: Mukul Joshi <mukul.joshi@amd.com> Tested-by: Amber Lin <Amber.Lin@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com> Signed-off-by: Morris Zhang <Shiwu.Zhang@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
4fc8fff3 |
|
21-Feb-2023 |
Qu Huang <qu.huang@linux.dev> |
drm/amdkfd: Fix an illegal memory access In the kfd_wait_on_events() function, the kfd_event_waiter structure is allocated by alloc_event_waiters(), but the event field of the waiter structure is not initialized; When copy_from_user() fails in the kfd_wait_on_events() function, it will enter exception handling to release the previously allocated memory of the waiter structure; Due to the event field of the waiters structure being accessed in the free_waiters() function, this results in illegal memory access and system crash, here is the crash log: localhost kernel: RIP: 0010:native_queued_spin_lock_slowpath+0x185/0x1e0 localhost kernel: RSP: 0018:ffffaa53c362bd60 EFLAGS: 00010082 localhost kernel: RAX: ff3d3d6bff4007cb RBX: 0000000000000282 RCX: 00000000002c0000 localhost kernel: RDX: ffff9e855eeacb80 RSI: 000000000000279c RDI: ffffe7088f6a21d0 localhost kernel: RBP: ffffe7088f6a21d0 R08: 00000000002c0000 R09: ffffaa53c362be64 localhost kernel: R10: ffffaa53c362bbd8 R11: 0000000000000001 R12: 0000000000000002 localhost kernel: R13: ffff9e7ead15d600 R14: 0000000000000000 R15: ffff9e7ead15d698 localhost kernel: FS: 0000152a3d111700(0000) GS:ffff9e855ee80000(0000) knlGS:0000000000000000 localhost kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 localhost kernel: CR2: 0000152938000010 CR3: 000000044d7a4000 CR4: 00000000003506e0 localhost kernel: Call Trace: localhost kernel: _raw_spin_lock_irqsave+0x30/0x40 localhost kernel: remove_wait_queue+0x12/0x50 localhost kernel: kfd_wait_on_events+0x1b6/0x490 [hydcu] localhost kernel: ? ftrace_graph_caller+0xa0/0xa0 localhost kernel: kfd_ioctl+0x38c/0x4a0 [hydcu] localhost kernel: ? kfd_ioctl_set_trap_handler+0x70/0x70 [hydcu] localhost kernel: ? kfd_ioctl_create_queue+0x5a0/0x5a0 [hydcu] localhost kernel: ? ftrace_graph_caller+0xa0/0xa0 localhost kernel: __x64_sys_ioctl+0x8e/0xd0 localhost kernel: ? syscall_trace_enter.isra.18+0x143/0x1b0 localhost kernel: do_syscall_64+0x33/0x80 localhost kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9 localhost kernel: RIP: 0033:0x152a4dff68d7 Allocate the structure with kcalloc, and remove redundant 0-initialization and a redundant loop condition check. Signed-off-by: Qu Huang <qu.huang@linux.dev> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
1c71222e |
|
26-Jan-2023 |
Suren Baghdasaryan <surenb@google.com> |
mm: replace vma->vm_flags direct modifications with modifier calls Replace direct modifications to vma->vm_flags with calls to modifier functions to be able to track flag changes and to keep vma locking correctness. [akpm@linux-foundation.org: fix drivers/misc/open-dice.c, per Hyeonggon Yoo] Link: https://lkml.kernel.org/r/20230126193752.297968-5-surenb@google.com Signed-off-by: Suren Baghdasaryan <surenb@google.com> Acked-by: Michal Hocko <mhocko@suse.com> Acked-by: Mel Gorman <mgorman@techsingularity.net> Acked-by: Mike Rapoport (IBM) <rppt@kernel.org> Acked-by: Sebastian Reichel <sebastian.reichel@collabora.com> Reviewed-by: Liam R. Howlett <Liam.Howlett@Oracle.com> Reviewed-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: Andy Lutomirski <luto@kernel.org> Cc: Arjun Roy <arjunroy@google.com> Cc: Axel Rasmussen <axelrasmussen@google.com> Cc: David Hildenbrand <david@redhat.com> Cc: David Howells <dhowells@redhat.com> Cc: Davidlohr Bueso <dave@stgolabs.net> Cc: David Rientjes <rientjes@google.com> Cc: Eric Dumazet <edumazet@google.com> Cc: Greg Thelen <gthelen@google.com> Cc: Hugh Dickins <hughd@google.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jann Horn <jannh@google.com> Cc: Joel Fernandes <joelaf@google.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Kent Overstreet <kent.overstreet@linux.dev> Cc: Laurent Dufour <ldufour@linux.ibm.com> Cc: Lorenzo Stoakes <lstoakes@gmail.com> Cc: Matthew Wilcox <willy@infradead.org> Cc: Minchan Kim <minchan@google.com> Cc: Paul E. McKenney <paulmck@kernel.org> Cc: Peter Oskolkov <posk@google.com> Cc: Peter Xu <peterx@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Punit Agrawal <punit.agrawal@bytedance.com> Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Cc: Shakeel Butt <shakeelb@google.com> Cc: Soheil Hassas Yeganeh <soheil@google.com> Cc: Song Liu <songliubraving@fb.com> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: Will Deacon <will@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
#
2d2ccc4c |
|
03-Nov-2022 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: Fix error handling in kfd_criu_restore_events mutex_unlock before the exit label because all the error code paths that jump there didn't take that lock. This fixes unbalanced locking errors in case of restore errors. Fixes: 40e8a766a761 ("drm/amdkfd: CRIU checkpoint and restore events") Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Rajneesh Bhardwaj <rajneesh.bhardwaj@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
66f79037 |
|
03-Nov-2022 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: Fix error handling in kfd_criu_restore_events mutex_unlock before the exit label because all the error code paths that jump there didn't take that lock. This fixes unbalanced locking errors in case of restore errors. Fixes: 40e8a766a761 ("drm/amdkfd: CRIU checkpoint and restore events") Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Rajneesh Bhardwaj <rajneesh.bhardwaj@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> Cc: stable@vger.kernel.org
|
#
bea9a56a |
|
04-Aug-2022 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: Handle restart of kfd_ioctl_wait_events When kfd_ioctl_wait_events needs to restart due to a signal, we need to update the timeout to account for the time already elapsed. We also need to undo auto_reset of events that have signaled already, so that the restarted ioctl will be able to count those signals again. This fixes infinite hangs when kfd_ioctl_wait_events is interrupted by a signal. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-and-tested-by: Xiaogang Chen <Xiaogang.Chen@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
4e2d1044 |
|
31-May-2022 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: Document and fix GTT BO kmap API Removed an unused parameter from two functions and added kernel-doc comments. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Philip Yang <Philip.Yang@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
c3eb12df |
|
07-Apr-2022 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: Ignore bogus signals from MEC efficiently MEC firmware sometimes sends signal interrupts without a valid context ID on end of pipe events that don't intend to signal any HSA signals. This triggers the slow path in kfd_signal_event_interrupt that scans the entire event page for signaled events. Detect these signals in the top half interrupt handler to stop processing them as early as possible. Because we now always treat event ID 0 as invalid, reserve that ID during process initialization. v2: Update firmware version checks to support more GPUs Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Philip Yang <Philip.Yang@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
250e64a3 |
|
12-Apr-2022 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: fix race condition in kfd_wait_on_events Add the waiters to the wait queue during initialization, while holding the event spinlock. Otherwise the waiter will not get activated if the event signals before being added to the wait queue. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Philip Yang<Philip.Yang@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
abb5bc59 |
|
13-Apr-2022 |
Dan Carpenter <dan.carpenter@oracle.com> |
drm/amdkfd: potential NULL dereference in kfd_set/reset_event() If lookup_event_by_id() returns a NULL "ev" pointer then the spin_lock(&ev->lock) will crash. This was detected by Smatch: drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_events.c:644 kfd_set_event() error: we previously assumed 'ev' could be null (see line 639) Fixes: 5273e82c5f47 ("drm/amdkfd: Improve concurrency of event handling") Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
34d292d5 |
|
07-Apr-2022 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: Asynchronously free events The synchronize_rcu call in destroy_events can take several ms, which noticeably slows down applications destroying many events. Use kfree_rcu to free the event structure asynchronously and eliminate the synchronize_rcu call in the user thread. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Philip Yang <Philip.Yang@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
5273e82c |
|
01-Mar-2022 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: Improve concurrency of event handling Use rcu_read_lock to read p->event_idr concurrently with other readers and writers. Use p->event_mutex only for creating and destroying events and in kfd_wait_on_events. Protect the contents of the kfd_event structure with a per-event spinlock that can be taken inside the rcu_read_lock critical section. This eliminates contention of p->event_mutex in set_event, which tends to be on the critical path for dispatch latency even when busy waiting is used. It also eliminates lock contention in event interrupt handlers. Since the p->event_mutex is now used much less, the impact of requiring it in kfd_wait_on_events should also be much smaller. This should improve event handling latency for processes using multiple GPUs concurrently. v2: Reschedule the worker periodically to avoid soft lockup warnings Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Sean Keely <Sean.Keely@amd.com> # v1 Tested-by: Sanjay Tripathi <sanjay.tripathi@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
ebbb7bb9 |
|
24-Mar-2022 |
QintaoShen <unSimple1993@163.com> |
drm/amdkfd: Check for potential null return of kmalloc_array() As the kmalloc_array() may return null, the 'event_waiters[i].wait' would lead to null-pointer dereference. Therefore, it is better to check the return value of kmalloc_array() to avoid this confusion. Signed-off-by: QintaoShen <unSimple1993@163.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
d87f36a0 |
|
10-Feb-2022 |
Rajneesh Bhardwaj <rajneesh.bhardwaj@amd.com> |
drm/amdkfd: update SPDX license header Update the SPDX License header for all the KFD files. Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Rajneesh Bhardwaj <rajneesh.bhardwaj@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
bef153b7 |
|
08-Apr-2021 |
David Yat Sin <david.yatsin@amd.com> |
drm/amdkfd: CRIU implement gpu_id remapping When doing a restore on a different node, the gpu_id's on the restore node may be different. But the user space application will still refer use the original gpu_id's in the ioctl calls. Adding code to create a gpu id mapping so that kfd can determine actual gpu_id during the user ioctl's. Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: David Yat Sin <david.yatsin@amd.com> Signed-off-by: Rajneesh Bhardwaj <rajneesh.bhardwaj@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
40e8a766 |
|
05-Mar-2021 |
David Yat Sin <david.yatsin@amd.com> |
drm/amdkfd: CRIU checkpoint and restore events Add support to existing CRIU ioctl's to save and restore events during criu checkpoint and restore. Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: David Yat Sin <david.yatsin@amd.com> Signed-off-by: Rajneesh Bhardwaj <rajneesh.bhardwaj@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
046e674b |
|
09-Nov-2021 |
Graham Sider <Graham.Sider@amd.com> |
drm/amdkfd: convert misc checks to IP version checking Switch to IP version checking instead of asic_type on various KFD version checks. Signed-off-by: Graham Sider <Graham.Sider@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
96b62c8a |
|
18-May-2021 |
Dennis Li <Dennis.Li@amd.com> |
drm/amdkfd: fix a resource leakage issue The function kfd_lookup_process_by_pasid will increase the reference count of kfd_process object, its caller should call kfd_unref_process to decrease the reference count. Otherwise resource leakage will happen. Signed-off-by: Dennis Li <Dennis.Li@amd.com> Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
e2b1f9f5 |
|
11-May-2021 |
Dennis Li <Dennis.Li@amd.com> |
drm/amdkfd: refine the poison data consumption handling The user applications maybe register the KFD_EVENT_TYPE_HW_EXCEPTION and KFD_EVENT_TYPE_MEMORY events, driver could notify them when poison data consumed. Beside that, some applications maybe register SIGBUS signal hander. These applications will handle poison data by themselves, exit or re-create context to re-dispatch works. Signed-off-by: Dennis Li <Dennis.Li@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
c7b6bac9 |
|
15-Sep-2020 |
Fenghua Yu <fenghua.yu@intel.com> |
drm, iommu: Change type of pasid to u32 PASID is defined as a few different types in iommu including "int", "u32", and "unsigned int". To be consistent and to match with uapi definitions, define PASID and its variations (e.g. max PASID) as "u32". "u32" is also shorter and a little more explicit than "unsigned int". No PASID type change in uapi although it defines PASID as __u64 in some places. Suggested-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Fenghua Yu <fenghua.yu@intel.com> Signed-off-by: Borislav Petkov <bp@suse.de> Reviewed-by: Tony Luck <tony.luck@intel.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Acked-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Joerg Roedel <jroedel@suse.de> Link: https://lkml.kernel.org/r/1600187413-163670-2-git-send-email-fenghua.yu@intel.com
|
#
d8ed45c5 |
|
08-Jun-2020 |
Michel Lespinasse <walken@google.com> |
mmap locking API: use coccinelle to convert mmap_sem rwsem call sites This change converts the existing mmap_sem rwsem calls to use the new mmap locking API instead. The change is generated using coccinelle with the following rule: // spatch --sp-file mmap_lock_api.cocci --in-place --include-headers --dir . @@ expression mm; @@ ( -init_rwsem +mmap_init_lock | -down_write +mmap_write_lock | -down_write_killable +mmap_write_lock_killable | -down_write_trylock +mmap_write_trylock | -up_write +mmap_write_unlock | -downgrade_write +mmap_write_downgrade | -down_read +mmap_read_lock | -down_read_killable +mmap_read_lock_killable | -down_read_trylock +mmap_read_trylock | -up_read +mmap_read_unlock ) -(&mm->mmap_sem) +(mm) Signed-off-by: Michel Lespinasse <walken@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: Daniel Jordan <daniel.m.jordan@oracle.com> Reviewed-by: Laurent Dufour <ldufour@linux.ibm.com> Reviewed-by: Vlastimil Babka <vbabka@suse.cz> Cc: Davidlohr Bueso <dbueso@suse.de> Cc: David Rientjes <rientjes@google.com> Cc: Hugh Dickins <hughd@google.com> Cc: Jason Gunthorpe <jgg@ziepe.ca> Cc: Jerome Glisse <jglisse@redhat.com> Cc: John Hubbard <jhubbard@nvidia.com> Cc: Liam Howlett <Liam.Howlett@oracle.com> Cc: Matthew Wilcox <willy@infradead.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Ying Han <yinghan@google.com> Link: http://lkml.kernel.org/r/20200520052908.204642-5-walken@google.com Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
#
8f2e0c03 |
|
09-Mar-2020 |
Yong Zhao <Yong.Zhao@amd.com> |
drm/amdkfd: Use pr_debug to print the message of reaching event limit People are inclined to think of the previous pr_warn message as an error, so use pre_debug instead. Signed-off-by: Yong Zhao <Yong.Zhao@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
29453755 |
|
15-Jan-2019 |
Yong Zhao <Yong.Zhao@amd.com> |
drm/amdkfd: Simplify the mmap offset related bit operations The new code uses straightforward bit shifts and thus has better readability. Signed-off-by: Yong Zhao <Yong.Zhao@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
6027b1bf |
|
25-Sep-2019 |
Yong Zhao <Yong.Zhao@amd.com> |
drm/amdkfd: Use hex print format for pasid Since KFD pasid starts from 0x8000 (32768 in decimal), it is better perceived as a hex number. Meanwhile, change the pasid type from unsigned int to uint16_t to be consistent throughout the code. Signed-off-by: Yong Zhao <Yong.Zhao@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
514e5e7e |
|
02-Sep-2019 |
Huang Rui <ray.huang@amd.com> |
drm/amdkfd: add renoir type for the workaround of iommu v2 (v2) Renoir is the same with Raven, will enable iommu event in future. v2: fix the checking (Thong) Signed-off-by: Huang Rui <ray.huang@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
0d87c9cf |
|
01-May-2019 |
Kent Russell <kent.russell@amd.com> |
drm/amdkfd: Cosmetic cleanup Fix some spacing issues, log output, uses of !=NULL/==NULL, unneeded extra lines and clean up a declaration from =1 to =true for clarity Signed-off-by: Kent Russell <kent.russell@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
9b54d201 |
|
11-Jan-2019 |
Eric Huang <JinhuiEric.Huang@amd.com> |
drm/amdkfd: add RAS ECC event support (v3) RAS ECC event will combine with GPU reset event, due to ECC interrupts are caused by uncorrectable error that triggers GPU reset. v2: Fix misleading-indentation warning v3: fix build with CONFIG_HSA_AMD disabled Signed-off-by: Eric Huang <JinhuiEric.Huang@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
#
359cecdd |
|
13-Jul-2018 |
Yong Zhao <yong.zhao@amd.com> |
drm/amdkfd: Optimize out some duplicated code in kfd_signal_iommu_event() memory_exception_data is already initialized for not-present faults. It only needs to be overridden for permission faults. Signed-off-by: Yong Zhao <yong.zhao@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
8725aeca |
|
13-Jul-2018 |
Yong Zhao <yong.zhao@amd.com> |
drm/amdkfd: Workaround to accommodate Raven too many PPR issue On Raven multiple PPRs can be queued up by the hardware. When the first of those requests is handled by the IOMMU driver, the memory access succeeds. After that the application may be done with the memory and unmap it. At that point the page table entries are invalidated, but there are still outstanding duplicate PPRs for those addresses. When the IOMMU driver processes those duplicate requests, it finds invalid page table entries and triggers an invalid PPR fault. As a workaround, don't signal invalid PPR faults on Raven to avoid segfaulting applications that haven't done anything wrong. As a side effect, real GPU memory access faults may go unnoticed by the application. Signed-off-by: Yong Zhao <yong.zhao@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
e42051d2 |
|
11-Jul-2018 |
Shaoyun Liu <Shaoyun.Liu@amd.com> |
drm/amdkfd: Implement GPU reset handlers in KFD Lock KFD and evict existing queues on reset. Notify user mode by signaling hw_exception events. Signed-off-by: Shaoyun Liu <Shaoyun.Liu@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
2640c3fa |
|
11-Jul-2018 |
shaoyunl <Shaoyun.Liu@amd.com> |
drm/amdkfd: Handle VM faults in KFD 1. Pre-GFX9 the amdgpu ISR saves the vm-fault status and address per per-vmid. amdkfd needs to get the information from amdgpu through the new get_vm_fault_info interface. On GFX9 and later, all the required information is in the IH ring 2. amdkfd unmaps all queues from the faulting process and create new run-list without the guilty process 3. amdkfd notifies the runtime of the vm fault trap via EVENT_TYPE_MEMORY Signed-off-by: shaoyun liu <shaoyun.liu@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
101fee63 |
|
11-Jul-2018 |
Moses Reuben <moses.reuben@amd.com> |
drm/amdkfd: send SIGSEGV to process upon KFD_EVENT_TYPE_MEMORY Signed-off-by: Moses Reuben <moses.reuben@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
eeb27b7e |
|
01-May-2018 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: Fix signal handling performance again It turns out that idr_for_each_entry is really slow compared to just iterating over the slots. Based on measurements the difference is estimated to be about a factor 64. That means using idr_for_each_entry is only worth it with very few allocated events. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
df03ef93 |
|
10-Apr-2018 |
Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com> |
drm/amdkfd: Clean up KFD_MMAP_ offset handling Use bit-rotate for better clarity and remove _MASK from the #defines as these represent mmap types. Centralize all the parsing of the mmap offset in kfd_mmap and add device parameter to doorbell and reserved_mem map functions. Encode gpu_id into upper bits of vm_pgoff. This frees up the lower bits for encoding the the doorbell ID on Vega10. Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
0fc8011f |
|
15-Mar-2018 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: Kmap event page for dGPUs The events page must be accessible in user mode by the GPU and CPU as well as in kernel mode by the CPU. On dGPUs user mode virtual addresses are managed by the Thunk's GPU memory allocation code. Therefore we can't allocate the memory in kernel mode like we do on APUs. But KFD still needs to map the memory for kernel access. To facilitate this, the Thunk provides the buffer handle of the events page to KFD when creating the first event. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
64d1c3a4 |
|
08-Dec-2017 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: Centralize IOMMUv2 code and make it conditional dGPUs work without IOMMUv2. Make IOMMUv2 initialization dependent on ASIC information. Also allow building KFD without IOMMUv2 support. This is still useful for dGPUs and prepares for enabling KFD on architectures that don't support AMD IOMMUv2. v2: * Centralize IOMMUv2 code to avoid #ifdefs in too many places v3: * Imply AMD_IOMMU_V2 in Kconfig Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian Konig <christian.koenig@amd.com> Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
abb208a8 |
|
27-Nov-2017 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: Use ref count to prevent kfd_process destruction Use a reference counter instead of a lock to prevent process destruction while functions running out of process context are using the kfd_process structure. In many cases these functions don't need the structure to be locked. In the few cases that really do need the process lock, take it explicitly. This helps simplify lock dependencies between the process lock and other locks, particularly amdgpu and mm_struct locks. This will be important when amdgpu calls back to amdkfd for memory evictions. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Christian König <christian.koenig@amd.com> Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
b9a5d0a5 |
|
27-Oct-2017 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: Make event limit dependent on user mode mapping size This allows increasing the KFD_SIGNAL_EVENT_LIMIT in kfd_ioctl.h without breaking processes built with older kfd_ioctl.h versions. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
3f04f961 |
|
27-Oct-2017 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: Use IH context ID for signal lookup This speeds up signal lookup when the IH ring entry includes a valid context ID or partial context ID. Only if the context ID is found to be invalid, fall back to an exhaustive search of all signaled events. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
482f0777 |
|
27-Oct-2017 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: Simplify event ID and signal slot management Signal slots are identical to event IDs. Replace the used_slot_bitmap and events hash table with an IDR to allocate and lookup event IDs and signal slots more efficiently. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
50cb7dd9 |
|
27-Oct-2017 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: Simplify events page allocator The first event page is always big enough to handle all events. Handling of multiple events pages is not supported by user mode, and not necessary. Signed-off-by: Yong Zhao <yong.zhao@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
74e40716 |
|
27-Oct-2017 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: Use wait_queue_t to implement event waiting Use standard wait queues for waiting and waking up waiting threads instead of inventing our own. We still have our own wait loop because the HSA event semantics require the ability to have one thread waiting on multiple wait queues (events) at the same time. Signed-off-by: Kent Russell <kent.russell@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
ebf947fe |
|
27-Oct-2017 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: remove redundant kfd_event_waiter.input_index This always identical with the index of the event_waiter in the array. No need to store it in the waiter record. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
fe528c13 |
|
27-Oct-2017 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: Fix event destruction with pending waiters When an event with pending waiters is destroyed, those waiters may end up sleeping forever unless they are notified and woken up. Implement the notification by clearing the waiter->event pointer, which becomes invalid anyway, when the event is freed, and waking up the waiting tasks. Waiters on an event that's destroyed return failure. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
fdf0c833 |
|
27-Oct-2017 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: Clean up kfd_wait_on_events Cleaned up the code while resolving some potential bugs and inconsistencies in the process. Clean-ups: * Remove enum kfd_event_wait_result, which duplicates KFD_IOC_EVENT_RESULT definitions * alloc_event_waiters can be called without holding p->event_mutex * Return an error code from copy_signaled_event_data instead of bool * Clean up error handling code paths to minimize duplication in kfd_wait_on_events Fixes: * Consistently return an error code from kfd_wait_on_events and set wait_result to KFD_IOC_WAIT_RESULT_FAIL in all failure cases. * Always call free_waiters while holding p->event_mutex * copy_signaled_event_data might sleep. Don't call it while the task state is TASK_INTERRUPTIBLE. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
d9aeec4c |
|
27-Oct-2017 |
Sean Keely <sean.keely@amd.com> |
drm/amdkfd: Fix scheduler race in kfd_wait_on_events sleep loop Signed-off-by: Sean Keely <sean.keely@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
1f9d09be |
|
27-Oct-2017 |
Sean Keely <sean.keely@amd.com> |
drm/amdkfd: Short cut for kfd_wait_on_events without waiting If kfd_wait_on_events can return immediately, we don't need to populate the wait list and don't need to enter the sleep-loop. Signed-off-by: Sean Keely <sean.keely@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Acked-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
9b56bb11 |
|
27-Oct-2017 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: Don't dereference kfd_process.mm The kfd_process doesn't own a reference to the mm_struct, so it can disappear without warning even while the kfd_process still exists. Therefore, avoid dereferencing the kfd_process.mm pointer and make it opaque. Use get_task_mm to get a temporary reference to the mm when it's needed. v2: removed unnecessary WARN_ON Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
c986169f |
|
20-Sep-2017 |
Felix Kuehling <Felix.Kuehling@amd.com> |
drm/amdkfd: Print event limit messages only once per process To avoid spamming the log. Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
4eacc26b |
|
15-Aug-2017 |
Kent Russell <kent.russell@amd.com> |
drm/amdkfd: Change x==NULL/false references to !x Upstream prefers the !x notation to x==NULL or x==false. Along those lines change the ==true or !=NULL references as well. Also make the references to !x the same, excluding () for readability. Signed-off-by: Kent Russell <kent.russell@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
79775b62 |
|
15-Aug-2017 |
Kent Russell <kent.russell@amd.com> |
drm/amdkfd: Consolidate and clean up log commands Consolidate log commands so that dev_info(NULL, "Error...") uses the more accurate pr_err, remove the module name from the log (can be seen via dynamic debugging with +m), and the function name (can be seen via dynamic debugging with +f). We also don't need debug messages saying what function we're in. Those can be added by devs when needed Don't print vendor and device ID in error messages. They are typically the same for all GPUs in a multi-GPU system. So this doesn't add any value to the message. Lastly, remove parentheses around %d, %i and 0x%llX. According to kernel.org: "Printing numbers in parentheses (%d) adds no value and should be avoided." Signed-off-by: Kent Russell <kent.russell@amd.com> Signed-off-by: Yong Zhao <Yong.Zhao@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
8eabaf54 |
|
15-Aug-2017 |
Kent Russell <kent.russell@amd.com> |
drm/amdkfd: Clean up KFD style errors and warnings v2 Using checkpatch.pl -f <file> showed a number of style issues. This patch addresses as many of them as possible. Some long lines have been left for readability, but attempts to minimize them have been made. v2: Broke long lines in gfx_v7 get_fw_version Signed-off-by: Kent Russell <kent.russell@amd.com> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
3f07c014 |
|
08-Feb-2017 |
Ingo Molnar <mingo@kernel.org> |
sched/headers: Prepare for new header dependencies before moving code to <linux/sched/signal.h> We are going to split <linux/sched/signal.h> out of <linux/sched.h>, which will have to be picked up from other headers and a couple of .c files. Create a trivial placeholder <linux/sched/signal.h> file that just maps to <linux/sched.h> to make this patch obviously correct and bisectable. Include the new header in the files that are going to need it. Acked-by: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mike Galbraith <efault@gmx.de> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-kernel@vger.kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org>
|
#
8bf79388 |
|
01-Dec-2016 |
Pan Bian <bianpan2016@163.com> |
drm/amdkfd: fix improper return value on error In function kfd_wait_on_events(), when the call to copy_from_user() fails, the value of return variable ret is 0. 0 indicates success, which is inconsistent with the execution status. This patch fixes the bug by assigning "-EFAULT" to ret when copy_from_user() returns an unexpected value. Signed-off-by: Pan Bian <bianpan2016@163.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
991ca8ee |
|
30-Apr-2016 |
Edward O'Callaghan <eocallaghan@alterapraxis.com> |
amdkfd: Use the canonical form in branch predicates Found-By: Coccinelle Signed-off-by: Edward O'Callaghan <eocallaghan@alterapraxis.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
6235e15e |
|
30-Apr-2015 |
Oded Gabbay <oded.gabbay@gmail.com> |
drm/amdkfd: add debug print to kfd_events.c Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
81663016 |
|
24-Dec-2014 |
Oded Gabbay <oded.gabbay@gmail.com> |
drm/amdkfd: Add module parameter of send_sigterm This patch adds a new kernel module parameter to amdkfd, called send_sigterm. This parameter specifies whether amdkfd should send the SIGTERM signal to an HSA process, when the following conditions occur: 1. The GPU triggers an exception regarding a kernel that was issued by this process. 2. The HSA process isn't waiting on an event that handles this exception. The default behavior is not to send a SIGTERM and suffice with a dmesg error print. Reviewed-by: Ben Goz <ben.goz@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
930c5ff4 |
|
25-Nov-2014 |
Alexey Skidanov <alexey.skidanov@gmail.com> |
drm/amdkfd: Add bad opcode exception handling Signed-off-by: Alexey Skidanov <alexey.skidanov@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
59d3e8be |
|
14-Apr-2015 |
Alexey Skidanov <alexey.skidanov@gmail.com> |
drm/amdkfd: Add memory exception handling This patch adds Peripheral Page Request (PPR) failure processing and reporting. Bad address or pointer to a system memory block with inappropriate read/write permission cause such PPR failure during a user queue processing. PPR request handling is done by IOMMU driver notifying AMDKFD module on PPR failure. The process triggering a PPR failure will be notified by appropriate event or SIGTERM signal will be sent to it. v3: - Change all bool fields in struct kfd_memory_exception_failure to uint32_t Signed-off-by: Alexey Skidanov <alexey.skidanov@gmail.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|
#
f3a39818 |
|
09-May-2015 |
Andrew Lewycky <Andrew.Lewycky@amd.com> |
drm/amdkfd: Add the events module This patch adds the events module (kfd_events.c) and the interrupt handle module for Kaveri (cik_event_interrupt.c). The patch updates the interrupt_is_wanted(), so that it now calls the interrupt isr function specific for the device that received the interrupt. That function(implemented in cik_event_interrupt.c) returns whether this interrupt is of interest to us or not. The patch also updates the interrupt_wq(), so that it now calls the device's specific wq function, which checks the interrupt source and tries to signal relevant events. v2: Increase limit of signal events to 4096 per process Remove bitfields from struct cik_ih_ring_entry Rename radeon_kfd_event_mmap to kfd_event_mmap Add debug prints to allocate_free_slot and allocate_signal_page Make allocate_event_notification_slot return a correct value Add warning prints to create_signal_event Remove error print from IOCTL path Reformatted debug prints in kfd_event_mmap Map correct size (as received from mmap) in kfd_event_mmap v3: Reduce limit of signal events back to 256 per process Fix allocation of kernel memory for signal events Signed-off-by: Andrew Lewycky <Andrew.Lewycky@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
|