Commit Graph

52 Commits

Author SHA1 Message Date
Jerry Snitselaar f7a70512fb dma-debug: avoid deadlock between dma debug vs printk and netconsole
JIRA: https://issues.redhat.com/browse/RHEL-55466
Upstream Status: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

commit bd44ca3de49cc1badcff7a96010fa2c64f04868c
Author: Rik van Riel <riel@surriel.com>
Date:   Tue Aug 6 11:56:45 2024 -0400

    dma-debug: avoid deadlock between dma debug vs printk and netconsole

    Currently the dma debugging code can end up indirectly calling printk
    under the radix_lock. This happens when a radix tree node allocation
    fails.

    This is a problem because the printk code, when used together with
    netconsole, can end up inside the dma debugging code while trying to
    transmit a message over netcons.

    This creates the possibility of either a circular deadlock on the same
    CPU, with that CPU trying to grab the radix_lock twice, or an ABBA
    deadlock between different CPUs, where one CPU grabs the console lock
    first and then waits for the radix_lock, while the other CPU is holding
    the radix_lock and is waiting for the console lock.

    The trace captured by lockdep is of the ABBA variant.

    -> #2 (&dma_entry_hash[i].lock){-.-.}-{2:2}:
		      _raw_spin_lock_irqsave+0x5a/0x90
		      debug_dma_map_page+0x79/0x180
		      dma_map_page_attrs+0x1d2/0x2f0
		      bnxt_start_xmit+0x8c6/0x1540
		      netpoll_start_xmit+0x13f/0x180
		      netpoll_send_skb+0x20d/0x320
		      netpoll_send_udp+0x453/0x4a0
		      write_ext_msg+0x1b9/0x460
		      console_flush_all+0x2ff/0x5a0
		      console_unlock+0x55/0x180
		      vprintk_emit+0x2e3/0x3c0
		      devkmsg_emit+0x5a/0x80
		      devkmsg_write+0xfd/0x180
		      do_iter_readv_writev+0x164/0x1b0
		      vfs_writev+0xf9/0x2b0
		      do_writev+0x6d/0x110
		      do_syscall_64+0x80/0x150
		      entry_SYSCALL_64_after_hwframe+0x4b/0x53

    -> #0 (console_owner){-.-.}-{0:0}:
		      __lock_acquire+0x15d1/0x31a0
		      lock_acquire+0xe8/0x290
		      console_flush_all+0x2ea/0x5a0
		      console_unlock+0x55/0x180
		      vprintk_emit+0x2e3/0x3c0
		      _printk+0x59/0x80
		      warn_alloc+0x122/0x1b0
		      __alloc_pages_slowpath+0x1101/0x1120
		      __alloc_pages+0x1eb/0x2c0
		      alloc_slab_page+0x5f/0x150
		      new_slab+0x2dc/0x4e0
		      ___slab_alloc+0xdcb/0x1390
		      kmem_cache_alloc+0x23d/0x360
		      radix_tree_node_alloc+0x3c/0xf0
		      radix_tree_insert+0xf5/0x230
		      add_dma_entry+0xe9/0x360
		      dma_map_page_attrs+0x1d2/0x2f0
		      __bnxt_alloc_rx_frag+0x147/0x180
		      bnxt_alloc_rx_data+0x79/0x160
		      bnxt_rx_skb+0x29/0xc0
		      bnxt_rx_pkt+0xe22/0x1570
		      __bnxt_poll_work+0x101/0x390
		      bnxt_poll+0x7e/0x320
		      __napi_poll+0x29/0x160
		      net_rx_action+0x1e0/0x3e0
		      handle_softirqs+0x190/0x510
		      run_ksoftirqd+0x4e/0x90
		      smpboot_thread_fn+0x1a8/0x270
		      kthread+0x102/0x120
		      ret_from_fork+0x2f/0x40
		      ret_from_fork_asm+0x11/0x20

    This bug is more likely than it seems, because when one CPU has run out
    of memory, chances are the other has too.

    The good news is, this bug is hidden behind the CONFIG_DMA_API_DEBUG, so
    not many users are likely to trigger it.

    Signed-off-by: Rik van Riel <riel@surriel.com>
    Reported-by: Konstantin Ovsepian <ovs@meta.com>
    Signed-off-by: Christoph Hellwig <hch@lst.de>

(cherry picked from commit bd44ca3de49cc1badcff7a96010fa2c64f04868c)
Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com>
2024-09-20 12:29:01 -07:00
Jerry Snitselaar 260fbbe3fa dma-debug: fix kernel-doc warnings
JIRA: https://issues.redhat.com/browse/RHEL-28780
Upstream Status: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

commit 7c65aa3cc072cee76f577262fbe381a111a98774
Author: Randy Dunlap <rdunlap@infradead.org>
Date:   Sat Jan 13 20:46:42 2024 -0800

    dma-debug: fix kernel-doc warnings

    Update the kernel-doc comments to catch up with the code changes and
    fix the kernel-doc warnings:

    debug.c:83: warning: Excess struct member 'stacktrace' description in 'dma_debug_entry'
    debug.c:83: warning: Function parameter or struct member 'stack_len' not described in 'dma_debug_entry'
    debug.c:83: warning: Function parameter or struct member 'stack_entries' not described in 'dma_debug_entry'

    Fixes: 746017ed8d ("dma/debug: Simplify stracktrace retrieval")
    Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
    Cc: Christoph Hellwig <hch@lst.de>
    Cc: Marek Szyprowski <m.szyprowski@samsung.com>
    Cc: Robin Murphy <robin.murphy@arm.com>
    Cc: iommu@lists.linux.dev
    Signed-off-by: Christoph Hellwig <hch@lst.de>

(cherry picked from commit 7c65aa3cc072cee76f577262fbe381a111a98774)
Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com>
2024-05-13 08:19:12 -07:00
Jerry Snitselaar 3ea76cb5c7 dma-debug: make dma_debug_add_bus take a const pointer
JIRA: https://issues.redhat.com/browse/RHEL-28780
Upstream Status: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

commit 86438841e48f6361f0a6a04805b7d7813738761f
Author: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Date:   Tue Dec 19 14:41:42 2023 +0100

    dma-debug: make dma_debug_add_bus take a const pointer

    The driver core now can handle a const struct bus_type pointer, and the
    dma_debug_add_bus() call just passes on the pointer give to it to the
    driver core, so make this pointer const as well to allow everyone to use
    read-only struct bus_type pointers going forward.

    Cc: Christoph Hellwig <hch@lst.de>
    Cc: Marek Szyprowski <m.szyprowski@samsung.com>
    Cc: Robin Murphy <robin.murphy@arm.com>
    Cc:  <iommu@lists.linux.dev>
    Reviewed-by: Robin Murphy <robin.murphy@arm.com>
    Link: https://lore.kernel.org/r/2023121941-dejected-nugget-681e@gregkh
    Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

(cherry picked from commit 86438841e48f6361f0a6a04805b7d7813738761f)
Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com>
2024-05-13 08:16:46 -07:00
Jerry Snitselaar 26f96c7be4 dma-debug: Fix a typo in a debugging eye-catcher
JIRA: https://issues.redhat.com/browse/RHEL-28780
Upstream Status: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

commit 36d91e851598a9ea523ad4681dd11fa661d59695
Author: Chuck Lever <chuck.lever@oracle.com>
Date:   Thu Oct 19 11:25:38 2023 -0400

    dma-debug: Fix a typo in a debugging eye-catcher

    Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
    Signed-off-by: Christoph Hellwig <hch@lst.de>

(cherry picked from commit 36d91e851598a9ea523ad4681dd11fa661d59695)
Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com>
2024-05-13 08:16:02 -07:00
Jerry Snitselaar 6047c64ac0 dma-debug: don't call __dma_entry_alloc_check_leak() under free_entries_lock
JIRA: https://issues.redhat.com/browse/RHEL-10094
Upstream Status: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

commit fb5a4315591dae307a65fc246ca80b5159d296e1
Author: Sergey Senozhatsky <senozhatsky@chromium.org>
Date:   Wed Aug 16 11:32:21 2023 +0900

    dma-debug: don't call __dma_entry_alloc_check_leak() under free_entries_lock

    __dma_entry_alloc_check_leak() calls into printk -> serial console
    output (qcom geni) and grabs port->lock under free_entries_lock
    spin lock, which is a reverse locking dependency chain as qcom_geni
    IRQ handler can call into dma-debug code and grab free_entries_lock
    under port->lock.

    Move __dma_entry_alloc_check_leak() call out of free_entries_lock
    scope so that we don't acquire serial console's port->lock under it.

    Trimmed-down lockdep splat:

     The existing dependency chain (in reverse order) is:

		   -> #2 (free_entries_lock){-.-.}-{2:2}:
	    _raw_spin_lock_irqsave+0x60/0x80
	    dma_entry_alloc+0x38/0x110
	    debug_dma_map_page+0x60/0xf8
	    dma_map_page_attrs+0x1e0/0x230
	    dma_map_single_attrs.constprop.0+0x6c/0xc8
	    geni_se_rx_dma_prep+0x40/0xcc
	    qcom_geni_serial_isr+0x310/0x510
	    __handle_irq_event_percpu+0x110/0x244
	    handle_irq_event_percpu+0x20/0x54
	    handle_irq_event+0x50/0x88
	    handle_fasteoi_irq+0xa4/0xcc
	    handle_irq_desc+0x28/0x40
	    generic_handle_domain_irq+0x24/0x30
	    gic_handle_irq+0xc4/0x148
	    do_interrupt_handler+0xa4/0xb0
	    el1_interrupt+0x34/0x64
	    el1h_64_irq_handler+0x18/0x24
	    el1h_64_irq+0x64/0x68
	    arch_local_irq_enable+0x4/0x8
	    ____do_softirq+0x18/0x24
	    ...

		   -> #1 (&port_lock_key){-.-.}-{2:2}:
	    _raw_spin_lock_irqsave+0x60/0x80
	    qcom_geni_serial_console_write+0x184/0x1dc
	    console_flush_all+0x344/0x454
	    console_unlock+0x94/0xf0
	    vprintk_emit+0x238/0x24c
	    vprintk_default+0x3c/0x48
	    vprintk+0xb4/0xbc
	    _printk+0x68/0x90
	    register_console+0x230/0x38c
	    uart_add_one_port+0x338/0x494
	    qcom_geni_serial_probe+0x390/0x424
	    platform_probe+0x70/0xc0
	    really_probe+0x148/0x280
	    __driver_probe_device+0xfc/0x114
	    driver_probe_device+0x44/0x100
	    __device_attach_driver+0x64/0xdc
	    bus_for_each_drv+0xb0/0xd8
	    __device_attach+0xe4/0x140
	    device_initial_probe+0x1c/0x28
	    bus_probe_device+0x44/0xb0
	    device_add+0x538/0x668
	    of_device_add+0x44/0x50
	    of_platform_device_create_pdata+0x94/0xc8
	    of_platform_bus_create+0x270/0x304
	    of_platform_populate+0xac/0xc4
	    devm_of_platform_populate+0x60/0xac
	    geni_se_probe+0x154/0x160
	    platform_probe+0x70/0xc0
	    ...

		   -> #0 (console_owner){-...}-{0:0}:
	    __lock_acquire+0xdf8/0x109c
	    lock_acquire+0x234/0x284
	    console_flush_all+0x330/0x454
	    console_unlock+0x94/0xf0
	    vprintk_emit+0x238/0x24c
	    vprintk_default+0x3c/0x48
	    vprintk+0xb4/0xbc
	    _printk+0x68/0x90
	    dma_entry_alloc+0xb4/0x110
	    debug_dma_map_sg+0xdc/0x2f8
	    __dma_map_sg_attrs+0xac/0xe4
	    dma_map_sgtable+0x30/0x4c
	    get_pages+0x1d4/0x1e4 [msm]
	    msm_gem_pin_pages_locked+0x38/0xac [msm]
	    msm_gem_pin_vma_locked+0x58/0x88 [msm]
	    msm_ioctl_gem_submit+0xde4/0x13ac [msm]
	    drm_ioctl_kernel+0xe0/0x15c
	    drm_ioctl+0x2e8/0x3f4
	    vfs_ioctl+0x30/0x50
	    ...

     Chain exists of:
       console_owner --> &port_lock_key --> free_entries_lock

      Possible unsafe locking scenario:

	    CPU0                    CPU1
	    ----                    ----
       lock(free_entries_lock);
				    lock(&port_lock_key);
				    lock(free_entries_lock);
       lock(console_owner);

		    *** DEADLOCK ***

     Call trace:
      dump_backtrace+0xb4/0xf0
      show_stack+0x20/0x30
      dump_stack_lvl+0x60/0x84
      dump_stack+0x18/0x24
      print_circular_bug+0x1cc/0x234
      check_noncircular+0x78/0xac
      __lock_acquire+0xdf8/0x109c
      lock_acquire+0x234/0x284
      console_flush_all+0x330/0x454
      console_unlock+0x94/0xf0
      vprintk_emit+0x238/0x24c
      vprintk_default+0x3c/0x48
      vprintk+0xb4/0xbc
      _printk+0x68/0x90
      dma_entry_alloc+0xb4/0x110
      debug_dma_map_sg+0xdc/0x2f8
      __dma_map_sg_attrs+0xac/0xe4
      dma_map_sgtable+0x30/0x4c
      get_pages+0x1d4/0x1e4 [msm]
      msm_gem_pin_pages_locked+0x38/0xac [msm]
      msm_gem_pin_vma_locked+0x58/0x88 [msm]
      msm_ioctl_gem_submit+0xde4/0x13ac [msm]
      drm_ioctl_kernel+0xe0/0x15c
      drm_ioctl+0x2e8/0x3f4
      vfs_ioctl+0x30/0x50
      ...

    Reported-by: Rob Clark <robdclark@chromium.org>
    Signed-off-by: Sergey Senozhatsky <senozhatsky@chromium.org>
    Acked-by: Robin Murphy <robin.murphy@arm.com>
    Signed-off-by: Christoph Hellwig <hch@lst.de>

(cherry picked from commit fb5a4315591dae307a65fc246ca80b5159d296e1)
Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com>
2023-10-27 01:26:58 -07:00
Jerry Snitselaar f3cfdfee95 dma-debug: Use %pa to format phys_addr_t
JIRA: https://issues.redhat.com/browse/RHEL-10094
Upstream Status: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

commit b31507dcaf3566d8ec3bdbf9922d8d5748b08a0f
Author: Geert Uytterhoeven <geert+renesas@glider.be>
Date:   Wed Mar 29 09:14:05 2023 +0200

    dma-debug: Use %pa to format phys_addr_t

    On 32-bit without LPAE:

	kernel/dma/debug.c: In function ‘debug_dma_dump_mappings’:
	kernel/dma/debug.c:537:7: warning: format ‘%llx’ expects argument of type ‘long long unsigned int’, but argument 9 has type ‘phys_addr_t’ {aka ‘unsigned int’} [-Wformat=]
	kernel/dma/debug.c: In function ‘dump_show’:
	kernel/dma/debug.c:568:59: warning: format ‘%llx’ expects argument of type ‘long long unsigned int’, but argument 11 has type ‘phys_addr_t’ {aka ‘unsigned int’} [-Wformat=]

    Fixes: bd89d69a529fbef3 ("dma-debug: add cacheline to user/kernel space dump messages")
    Reported-by: kernel test robot <lkp@intel.com>
    Link: https://lore.kernel.org/r/202303160548.ReyuTsGD-lkp@intel.com
    Reported-by: noreply@ellerman.id.au
    Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
    Signed-off-by: Christoph Hellwig <hch@lst.de>

(cherry picked from commit b31507dcaf3566d8ec3bdbf9922d8d5748b08a0f)
Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com>
2023-10-27 01:26:58 -07:00
Jerry Snitselaar ddc567597f dma-debug: add cacheline to user/kernel space dump messages
JIRA: https://issues.redhat.com/browse/RHEL-10094
Upstream Status: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

commit bd89d69a529fbef3559a16dbe4cee4b04225136a
Author: Desnes Nunes <desnesn@redhat.com>
Date:   Thu Mar 16 11:09:11 2023 -0300

    Having the cacheline also printed on the debug_dma_dump_mappings() and
    dump_show() is useful for debugging. Furthermore, this also standardizes
    the messages shown on both dump functions.

    Signed-off-by: Desnes Nunes <desnesn@redhat.com>
    Signed-off-by: Christoph Hellwig <hch@lst.de>

(cherry picked from commit bd89d69a529fbef3559a16dbe4cee4b04225136a)
Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com>
2023-10-27 01:26:58 -07:00
Jerry Snitselaar e135144ef7 dma-debug: small dma_debug_entry's comment and variable name updates
JIRA: https://issues.redhat.com/browse/RHEL-10094
Upstream Status: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

commit 479623fd0c5ad86a43ae11670f7da2a7c462acd4
Author: Desnes Nunes <desnesn@redhat.com>
Date:   Thu Mar 16 11:09:10 2023 -0300

    dma-debug: small dma_debug_entry's comment and variable name updates

    Small update on dma_debug_entry's struct commentary and also standardize
    the usage of 'dma_addr' variable name from debug_dma_map_page() on
    debug_dma_unmap_page(), and similarly on debug_dma_free_coherent()

    Signed-off-by: Desnes Nunes <desnesn@redhat.com>
    Signed-off-by: Christoph Hellwig <hch@lst.de>

(cherry picked from commit 479623fd0c5ad86a43ae11670f7da2a7c462acd4)
Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com>
2023-10-27 01:26:58 -07:00
Jerry Snitselaar 9529cb5882 dma-debug: improve search for partial syncs
Bugzilla: https://bugzilla.redhat.com/2112024
Upstream Status: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

commit 2995b8002cefc7b0b00b8f9a0ce36601a8c390c0
Author: Robin Murphy <robin.murphy@arm.com>
Date:   Mon Aug 15 20:28:40 2022 +0100

    dma-debug: improve search for partial syncs

    When bucket_find_contains() tries to find the original entry for a
    partial sync, it manages to constrain its search in a way that is both
    too restrictive and not restrictive enough. A driver which only uses
    single mappings rather than scatterlists might not set max_seg_size, but
    could still technically perform a partial sync at an offset of more than
    64KB into a sufficiently large mapping, so we could stop searching too
    early before reaching a legitimate entry. Conversely, if no valid entry
    is present and max_range is large enough, we can pointlessly search
    buckets that we've already searched, or that represent an impossible
    wrapping around the bottom of the address space. At worst, the
    (legitimate) case of max_seg_size == UINT_MAX can make the loop
    infinite.

    Replace the fragile and frankly hard-to-follow "range" logic with a
    simple counted loop for the number of possible hash buckets below the
    given address.

    Reported-by: Yunfei Wang <yf.wang@mediatek.com>
    Signed-off-by: Robin Murphy <robin.murphy@arm.com>
    Signed-off-by: Christoph Hellwig <hch@lst.de>

(cherry picked from commit 2995b8002cefc7b0b00b8f9a0ce36601a8c390c0)
Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com>
2022-10-27 16:31:40 -07:00
Jerry Snitselaar 44c27de71a dma-debug: make things less spammy under memory pressure
Bugzilla: https://bugzilla.redhat.com/2105609
Upstream Status: kernel/git/torvalds/linux.git

commit e19f8fa6ce1ca9b8b934ba7d2e8f34c95abc6e60
Author: Rob Clark <robdclark@chromium.org>
Date:   Wed Jun 1 07:51:16 2022 -0700

    dma-debug: make things less spammy under memory pressure

    Limit the error msg to avoid flooding the console.  If you have a lot of
    threads hitting this at once, they could have already gotten passed the
    dma_debug_disabled() check before they get to the point of allocation
    failure, resulting in quite a lot of this error message spamming the
    log.  Use pr_err_once() to limit that.

    Signed-off-by: Rob Clark <robdclark@chromium.org>
    Signed-off-by: Christoph Hellwig <hch@lst.de>

(cherry picked from commit e19f8fa6ce1ca9b8b934ba7d2e8f34c95abc6e60)
Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com>
2022-07-15 09:22:46 -07:00
Jerry Snitselaar 597870a92e dma-debug: change allocation mode from GFP_NOWAIT to GFP_ATIOMIC
Bugzilla: https://bugzilla.redhat.com/2072170
Upstream Status: kernel/git/torvalds/linux.git

commit 84bc4f1dbbbb5f8aa68706a96711dccb28b518e5
Author: Mikulas Patocka <mpatocka@redhat.com>
Date:   Tue May 10 13:17:32 2022 -0400

    dma-debug: change allocation mode from GFP_NOWAIT to GFP_ATIOMIC

    We observed the error "cacheline tracking ENOMEM, dma-debug disabled"
    during a light system load (copying some files). The reason for this error
    is that the dma_active_cacheline radix tree uses GFP_NOWAIT allocation -
    so it can't access the emergency memory reserves and it fails as soon as
    anybody reaches the watermark.

    This patch changes GFP_NOWAIT to GFP_ATOMIC, so that it can access the
    emergency memory reserves.

    Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
    Signed-off-by: Christoph Hellwig <hch@lst.de>

(cherry picked from commit 84bc4f1dbbbb5f8aa68706a96711dccb28b518e5)
Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com>
2022-05-31 21:50:32 -07:00
Jerry Snitselaar 739cbe0547 dma-debug: fix return value of __setup handlers
Bugzilla: https://bugzilla.redhat.com/2072170
Upstream Status: kernel/git/torvalds/linux.git

commit 80e4390981618e290616dbd06ea190d4576f219d
Author: Randy Dunlap <rdunlap@infradead.org>
Date:   Mon Feb 28 14:04:53 2022 -0800

    dma-debug: fix return value of __setup handlers

    When valid kernel command line parameters
      dma_debug=off dma_debug_entries=100
    are used, they are reported as Unknown parameters and added to init's
    environment strings, polluting it.

      Unknown kernel command line parameters "BOOT_IMAGE=/boot/bzImage-517rc5
	dma_debug=off dma_debug_entries=100", will be passed to user space.

    and

     Run /sbin/init as init process
       with arguments:
	 /sbin/init
       with environment:
	 HOME=/
	 TERM=linux
	 BOOT_IMAGE=/boot/bzImage-517rc5
	 dma_debug=off
	 dma_debug_entries=100

    Return 1 from these __setup handlers to indicate that the command line
    option has been handled.

    Fixes: 59d3daafa1 ("dma-debug: add kernel command line parameters")
    Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
    Reported-by: Igor Zhbanov <i.zhbanov@omprussia.ru>
    Link: lore.kernel.org/r/64644a2f-4a20-bab3-1e15-3b2cdd0defe3@omprussia.ru
    Cc: Joerg Roedel <joro@8bytes.org>
    Cc: Christoph Hellwig <hch@lst.de>
    Cc: Marek Szyprowski <m.szyprowski@samsung.com>
    Cc: iommu@lists.linux-foundation.org
    Cc: Robin Murphy <robin.murphy@arm.com>
    Signed-off-by: Christoph Hellwig <hch@lst.de>

(cherry picked from commit 80e4390981618e290616dbd06ea190d4576f219d)
Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com>
2022-05-19 10:58:10 -07:00
Jerry Snitselaar 58cf47e2ec dma-debug: teach add_dma_entry() about DMA_ATTR_SKIP_CPU_SYNC
Bugzilla: https://bugzilla.redhat.com/2004348
Upstream Status: kernel/git/torvalds/linux.git

commit c2bbf9d1e9ac7d4fdd503b190bc1ba8a6302bc49
Author: Hamza Mahfooz <someguy@effective-light.com>
Date:   Tue Oct 12 01:54:03 2021 -0400

    dma-debug: teach add_dma_entry() about DMA_ATTR_SKIP_CPU_SYNC

    Mapping something twice should be possible as long as,
    DMA_ATTR_SKIP_CPU_SYNC is passed to the strictly speaking second relevant
    mapping operation (that attempts to map the same thing). So, don't issue a
    warning if the specified condition is met in add_dma_entry().

    Signed-off-by: Hamza Mahfooz <someguy@effective-light.com>
    Signed-off-by: Christoph Hellwig <hch@lst.de>

(cherry picked from commit c2bbf9d1e9ac7d4fdd503b190bc1ba8a6302bc49)
Cc: Donald Dutile <ddutile@redhat.com>
Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com>
2021-10-27 16:42:13 -07:00
Jerry Snitselaar 62a0db7c8c dma-debug: fix sg checks in debug_dma_map_sg()
Bugzilla: https://bugzilla.redhat.com/2004348
Upstream Status: kernel/git/torvalds/linux.git

commit 293d92cbbd2418ca2ba43fed07f1b92e884d1c77
Author: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
Date:   Wed Oct 6 22:19:43 2021 +0200

    dma-debug: fix sg checks in debug_dma_map_sg()

    The following warning occurred sporadically on s390:
    DMA-API: nvme 0006:00:00.0: device driver maps memory from kernel text or rodata [addr=0000000048cc5e2f] [len=131072]
    WARNING: CPU: 4 PID: 825 at kernel/dma/debug.c:1083 check_for_illegal_area+0xa8/0x138

    It is a false-positive warning, due to broken logic in debug_dma_map_sg().
    check_for_illegal_area() checks for overlay of sg elements with kernel text
    or rodata. It is called with sg_dma_len(s) instead of s->length as
    parameter. After the call to ->map_sg(), sg_dma_len() will contain the
    length of possibly combined sg elements in the DMA address space, and not
    the individual sg element length, which would be s->length.

    The check will then use the physical start address of an sg element, and
    add the DMA length for the overlap check, which could result in the false
    warning, because the DMA length can be larger than the actual single sg
    element length.

    In addition, the call to check_for_illegal_area() happens in the iteration
    over mapped_ents, which will not include all individual sg elements if
    any of them were combined in ->map_sg().

    Fix this by using s->length instead of sg_dma_len(s). Also put the call to
    check_for_illegal_area() in a separate loop, iterating over all the
    individual sg elements ("nents" instead of "mapped_ents").

    While at it, as suggested by Robin Murphy, also move check_for_stack()
    inside the new loop, as it is similarly concerned with validating the
    individual sg elements.

    Link: https://lore.kernel.org/lkml/20210705185252.4074653-1-gerald.schaefer@linux.ibm.com
    Fixes: 884d05970b ("dma-debug: use sg_dma_len accessor")
    Signed-off-by: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
    Signed-off-by: Christoph Hellwig <hch@lst.de>

(cherry picked from commit 293d92cbbd2418ca2ba43fed07f1b92e884d1c77)
Cc: Donald Dutile <ddutile@redhat.com>
Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com>
2021-10-27 16:42:12 -07:00
Jerry Snitselaar d48c63f8d2 dma-debug: prevent an error message from causing runtime problems
Bugzilla: https://bugzilla.redhat.com/2004348
Upstream Status: kernel/git/torvalds/linux.git

commit 510e1a724ab1bf38150be2c1acabb303f98d0047
Author: Hamza Mahfooz <someguy@effective-light.com>
Date:   Fri Sep 10 19:53:37 2021 -0400

    dma-debug: prevent an error message from causing runtime problems

    For some drivers, that use the DMA API. This error message can be reached
    several millions of times per second, causing spam to the kernel's printk
    buffer and bringing the CPU usage up to 100% (so, it should be rate
    limited). However, since there is at least one driver that is in the
    mainline and suffers from the error condition, it is more useful to
    err_printk() here instead of just rate limiting the error message (in hopes
    that it will make it easier for other drivers that suffer from this issue
    to be spotted).

    Link: https://lkml.kernel.org/r/fd67fbac-64bf-f0ea-01e1-5938ccfab9d0@arm.com
    Reported-by: Jeremy Linton <jeremy.linton@arm.com>
    Signed-off-by: Hamza Mahfooz <someguy@effective-light.com>
    Signed-off-by: Christoph Hellwig <hch@lst.de>

(cherry picked from commit 510e1a724ab1bf38150be2c1acabb303f98d0047)
Cc: Donald Dutile <ddutile@redhat.com>
Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com>
2021-10-27 16:42:12 -07:00
Jerry Snitselaar 17f4dcbccb dma-debug: fix debugfs initialization order
Bugzilla: https://bugzilla.redhat.com/2004348
Upstream Status: kernel/git/torvalds/linux.git

commit 173735c346c412d9f084825ecb04f24ada0e2986
Author: Anthony Iliopoulos <ailiop@suse.com>
Date:   Thu Jul 22 16:10:55 2021 +0200

    dma-debug: fix debugfs initialization order

    Due to link order, dma_debug_init is called before debugfs has a chance
    to initialize (via debugfs_init which also happens in the core initcall
    stage), so the directories for dma-debug are never created.

    Decouple dma_debug_fs_init from dma_debug_init and defer its init until
    core_initcall_sync (after debugfs has been initialized) while letting
    dma-debug initialization occur as soon as possible to catch any early
    mappings, as suggested in [1].

    [1] https://lore.kernel.org/linux-iommu/YIgGa6yF%2Fadg8OSN@kroah.com/

    Fixes: 15b28bbcd5 ("dma-debug: move initialization to common code")
    Signed-off-by: Anthony Iliopoulos <ailiop@suse.com>
    Signed-off-by: Christoph Hellwig <hch@lst.de>

(cherry picked from commit 173735c346c412d9f084825ecb04f24ada0e2986)
Cc: Donald Dutile <ddutile@redhat.com>
Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com>
2021-10-27 16:25:28 -07:00
Jerry Snitselaar 65dd88a384 dma-debug: use memory_intersects() directly
Bugzilla: https://bugzilla.redhat.com/2004348
Upstream Status: kernel/git/torvalds/linux.git

commit 1d7db834a027ecfb5ac93be2e8bb45bfcbbf6eaa
Author: Kefeng Wang <wangkefeng.wang@huawei.com>
Date:   Thu Jul 22 18:55:12 2021 +0800

    dma-debug: use memory_intersects() directly

    There is already a memory_intersects() helper in sections.h,
    use memory_intersects() directly instead of private overlap().

    Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
    Signed-off-by: Christoph Hellwig <hch@lst.de>

(cherry picked from commit 1d7db834a027ecfb5ac93be2e8bb45bfcbbf6eaa)
Cc: Donald Dutile <ddutile@redhat.com>
Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com>
2021-10-27 16:25:25 -07:00
Hamza Mahfooz 2b4bbc6231 dma-debug: report -EEXIST errors in add_dma_entry
Since, overlapping mappings are not supported by the DMA API we should
report an error if active_cacheline_insert returns -EEXIST.

Suggested-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Hamza Mahfooz <someguy@effective-light.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
2021-06-22 08:15:47 +02:00
Christoph Hellwig a1fd09e8e6 dma-mapping: move dma-debug.h to kernel/dma/
Most of dma-debug.h is not required by anything outside of kernel/dma.
Move the four declarations needed by dma-mappin.h or dma-ops providers
into dma-mapping.h and dma-map-ops.h, and move the remainder of the
file to kernel/dma/debug.h.

Signed-off-by: Christoph Hellwig <hch@lst.de>
2020-10-06 07:07:05 +02:00
Xu Wang a97740f818 dma-debug: convert comma to semicolon
Replace a comma between expression statements by a semicolon.

Signed-off-by: Xu Wang <vulab@iscas.ac.cn>
Signed-off-by: Christoph Hellwig <hch@lst.de>
2020-09-21 16:48:38 +02:00
Christoph Hellwig ec91ccb274 dma-debug: remove most exports
Now that the main dma mapping entry points are out of line most of the
symbols in dma-debug.c can only be called from built-in code.  Remove
the unused exports.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
2020-09-11 08:13:12 +02:00
Linus Torvalds 5848dc5b1b dma-debug: remove debug_dma_assert_idle() function
This remoes the code from the COW path to call debug_dma_assert_idle(),
which was added many years ago.

Google shows that it hasn't caught anything in the 6+ years we've had it
apart from a false positive, and Hugh just noticed how it had a very
unfortunate spinlock serialization in the COW path.

He fixed that issue the previous commit (a85ffd59bd36: "dma-debug: fix
debug_dma_assert_idle(), use rcu_read_lock()"), but let's see if anybody
even notices when we remove this function entirely.

NOTE! We keep the dma tracking infrastructure that was added by the
commit that introduced it.  Partly to make it easier to resurrect this
debug code if we ever deside to, and partly because that tracking by pfn
and offset looks quite reasonable.

The problem with this debug code was simply that it was expensive and
didn't seem worth it, not that it was wrong per se.

Acked-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-08-14 15:22:43 -07:00
Hugh Dickins a85ffd59bd dma-debug: fix debug_dma_assert_idle(), use rcu_read_lock()
Since commit 2a9127fcf2 ("mm: rewrite wait_on_page_bit_common()
logic") improved unlock_page(), it has become more noticeable how
cow_user_page() in a kernel with CONFIG_DMA_API_DEBUG=y can create and
suffer from heavy contention on DMA debug's radix_lock in
debug_dma_assert_idle().

It is only doing a lookup: use rcu_read_lock() and rcu_read_unlock()
instead; though that does require the static ents[] to be moved
onstack...

...but, hold on, isn't that radix_tree_gang_lookup() and loop doing
quite the wrong thing: searching CACHELINES_PER_PAGE entries for an
exact match with the first cacheline of the page in question?
radix_tree_gang_lookup() is the right tool for the job, but we need
nothing more than to check the first entry it can find, reporting if
that falls anywhere within the page.

(Is RCU safe here? As safe as using the spinlock was. The entries are
never freed, so don't need to be freed by RCU. They may be reused, and
there is a faint chance of a race, with an offending entry reused while
printing its error info; but the spinlock did not prevent that either,
and I agree that it's not worth worrying about. ]

[ Side noe: this patch is a clear improvement to the status quo, but the
  next patch will be removing this debug function entirely.

  But just in case we decide we want to resurrect the debugging code
  some day, I'm first applying this improvement patch so that it doesn't
  get lost    - Linus ]

Fixes: 3b7a6418c7 ("dma debug: account for cachelines and read-only mappings in overlap tracking")
Signed-off-by: Hugh Dickins <hughd@google.com>
Acked-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-08-14 15:16:03 -07:00
Linus Torvalds 2324d50d05 It's been a busy cycle for documentation - hopefully the busiest for a
while to come.  Changes include:
 
  - Some new Chinese translations
 
  - Progress on the battle against double words words and non-HTTPS URLs
 
  - Some block-mq documentation
 
  - More RST conversions from Mauro.  At this point, that task is
    essentially complete, so we shouldn't see this kind of churn again for a
    while.  Unless we decide to switch to asciidoc or something...:)
 
  - Lots of typo fixes, warning fixes, and more.
 -----BEGIN PGP SIGNATURE-----
 
 iQFDBAABCAAtFiEEIw+MvkEiF49krdp9F0NaE2wMflgFAl8oVkwPHGNvcmJldEBs
 d24ubmV0AAoJEBdDWhNsDH5YoW8H/jJ/xnXFn7tkgVPQAlL3k5HCnK7A5nDP9RVR
 cg1pTx1cEFdjzxPlJyExU6/v+AImOvtweHXC+JDK7YcJ6XFUNYXJI3LxL5KwUXbY
 BL/xRFszDSXH2C7SJF5GECcFYp01e/FWSLN3yWAh+g+XwsKiTJ8q9+CoIDkHfPGO
 7oQsHKFu6s36Af0LfSgxk4sVB7EJbo8e4psuPsP5SUrl+oXRO43Put0rXkR4yJoH
 9oOaB51Do5fZp8I4JVAqGXvpXoExyLMO4yw0mASm6YSZ3KyjR8Fae+HD9Cq4ZuwY
 0uzb9K+9NEhqbfwtyBsi99S64/6Zo/MonwKwevZuhtsDTK4l4iU=
 =JQLZ
 -----END PGP SIGNATURE-----

Merge tag 'docs-5.9' of git://git.lwn.net/linux

Pull documentation updates from Jonathan Corbet:
 "It's been a busy cycle for documentation - hopefully the busiest for a
  while to come. Changes include:

   - Some new Chinese translations

   - Progress on the battle against double words words and non-HTTPS
     URLs

   - Some block-mq documentation

   - More RST conversions from Mauro. At this point, that task is
     essentially complete, so we shouldn't see this kind of churn again
     for a while. Unless we decide to switch to asciidoc or
     something...:)

   - Lots of typo fixes, warning fixes, and more"

* tag 'docs-5.9' of git://git.lwn.net/linux: (195 commits)
  scripts/kernel-doc: optionally treat warnings as errors
  docs: ia64: correct typo
  mailmap: add entry for <alobakin@marvell.com>
  doc/zh_CN: add cpu-load Chinese version
  Documentation/admin-guide: tainted-kernels: fix spelling mistake
  MAINTAINERS: adjust kprobes.rst entry to new location
  devices.txt: document rfkill allocation
  PCI: correct flag name
  docs: filesystems: vfs: correct flag name
  docs: filesystems: vfs: correct sync_mode flag names
  docs: path-lookup: markup fixes for emphasis
  docs: path-lookup: more markup fixes
  docs: path-lookup: fix HTML entity mojibake
  CREDITS: Replace HTTP links with HTTPS ones
  docs: process: Add an example for creating a fixes tag
  doc/zh_CN: add Chinese translation prefer section
  doc/zh_CN: add clearing-warn-once Chinese version
  doc/zh_CN: add admin-guide index
  doc:it_IT: process: coding-style.rst: Correct __maybe_unused compiler label
  futex: MAINTAINERS: Re-add selftests directory
  ...
2020-08-04 22:47:54 -07:00
Linus Torvalds 2ed90dbbf7 dma-mapping updates for 5.9
- make support for dma_ops optional
  - move more code out of line
  - add generic support for a dma_ops bypass mode
  - misc cleanups
 -----BEGIN PGP SIGNATURE-----
 
 iQI/BAABCgApFiEEgdbnc3r/njty3Iq9D55TZVIEUYMFAl8oGscLHGhjaEBsc3Qu
 ZGUACgkQD55TZVIEUYNfEhAAmFwd6BBHGwAhXUchoIue5vdNnuY3GiBFRzUdz67W
 zRYYgZYiPjl+MwflRmwPcoWEnGzmweRa2s6OnyDostiCRauioa8BuQfGqJasf1yZ
 D36dFNVHGW0o6pRDUQkd688k/4A6szwuwpq83qi4e8X2I9QzAITHtW8izjfPM923
 FlJzxEFggbB2TvwfUXOZhmpuG4Dog8S7VZ1Uz4QAg0Z/5FDqIKAAG2aZMqCXBbiX
 01E8tr0AqU/jn2xpc8O+DJGFiYIRhqhyNxQbH6qz1Q3xGFSokcLYm3YqkqVOgpn1
 DLs2UFDxWkly/F+wGnYtju7OD9VGPywzOcW125/LIsApYN5R/rYrtQzK41eq7Mp5
 HY3tqgNTIMdnl4so7QXeU4Vxj+lUdPlI26NZGszcM5AVftdTX8KjGdS+0+PBza6i
 i7trwG7J5/DnwiBCvEKoul7Ul1psUMTSvYwINTXRqsU4mZXhhx/mwyXbtruELnkj
 3agM98u6hoalLNjd2aueh+NjMZi1r+MchTrfRvTcxJ+yQ5BoR5kF+iz7eT/LtZ72
 AqWwimsPGNkLHUa0TrqWql5tv90cdDkBZzWXVbixwxRfgynWYLE6jugeIy8hwjFf
 GjO5XKbBwnWPjdSzFsVMPeuNpmr7ZjVHHewy2Q/jWQAIOyeof0VztEl23LN5yUkx
 pc8=
 =90UK
 -----END PGP SIGNATURE-----

Merge tag 'dma-mapping-5.9' of git://git.infradead.org/users/hch/dma-mapping

Pull dma-mapping updates from Christoph Hellwig:

 - make support for dma_ops optional

 - move more code out of line

 - add generic support for a dma_ops bypass mode

 - misc cleanups

* tag 'dma-mapping-5.9' of git://git.infradead.org/users/hch/dma-mapping:
  dma-contiguous: cleanup dma_alloc_contiguous
  dma-debug: use named initializers for dir2name
  powerpc: use the generic dma_ops_bypass mode
  dma-mapping: add a dma_ops_bypass flag to struct device
  dma-mapping: make support for dma ops optional
  dma-mapping: inline the fast path dma-direct calls
  dma-mapping: move the remaining DMA API calls out of line
2020-08-04 17:29:57 -07:00
Christoph Hellwig 23efed6fa7 dma-debug: use named initializers for dir2name
Make dir2name a little more readable and maintainable by using
named initializers.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
2020-07-19 09:31:13 +02:00
Kees Cook 3f649ab728 treewide: Remove uninitialized_var() usage
Using uninitialized_var() is dangerous as it papers over real bugs[1]
(or can in the future), and suppresses unrelated compiler warnings
(e.g. "unused variable"). If the compiler thinks it is uninitialized,
either simply initialize the variable or make compiler changes.

In preparation for removing[2] the[3] macro[4], remove all remaining
needless uses with the following script:

git grep '\buninitialized_var\b' | cut -d: -f1 | sort -u | \
	xargs perl -pi -e \
		's/\buninitialized_var\(([^\)]+)\)/\1/g;
		 s:\s*/\* (GCC be quiet|to make compiler happy) \*/$::g;'

drivers/video/fbdev/riva/riva_hw.c was manually tweaked to avoid
pathological white-space.

No outstanding warnings were found building allmodconfig with GCC 9.3.0
for x86_64, i386, arm64, arm, powerpc, powerpc64le, s390x, mips, sparc64,
alpha, and m68k.

[1] https://lore.kernel.org/lkml/20200603174714.192027-1-glider@google.com/
[2] https://lore.kernel.org/lkml/CA+55aFw+Vbj0i=1TGqCR5vQkCzWJ0QxK6CernOU6eedsudAixw@mail.gmail.com/
[3] https://lore.kernel.org/lkml/CA+55aFwgbgqhbp1fkxvRKEpzyR5J8n1vKT1VZdz9knmPuXhOeg@mail.gmail.com/
[4] https://lore.kernel.org/lkml/CA+55aFz2500WfbKXAx8s67wrm9=yVJu65TpLgN_ybYNv0VEOKA@mail.gmail.com/

Reviewed-by: Leon Romanovsky <leonro@mellanox.com> # drivers/infiniband and mlx4/mlx5
Acked-by: Jason Gunthorpe <jgg@mellanox.com> # IB
Acked-by: Kalle Valo <kvalo@codeaurora.org> # wireless drivers
Reviewed-by: Chao Yu <yuchao0@huawei.com> # erofs
Signed-off-by: Kees Cook <keescook@chromium.org>
2020-07-16 12:35:15 -07:00
Mauro Carvalho Chehab 985098a05e docs: fix references for DMA*.txt files
As we moved those files to core-api, fix references to point
to their newer locations.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/37b2fd159fbc7655dbf33b3eb1215396a25f6344.1592895969.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2020-06-26 10:01:32 -06:00
Jason Yan 05f099a7d0 dma-debug: make __dma_entry_alloc_check_leak() static
Fix the following sparse warning:

kernel/dma/debug.c:659:6: warning: symbol '__dma_entry_alloc_check_leak'
was not declared. Should it be static?

Reported-by: Hulk Robot <hulkci@huawei.com>
Signed-off-by: Jason Yan <yanaijie@huawei.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
2020-04-20 12:05:18 +02:00
Grygorii Strashko 9bb50ed747 dma-debug: fix displaying of dma allocation type
The commit 2e05ea5cdc ("dma-mapping: implement dma_map_single_attrs using
dma_map_page_attrs") removed "dma_debug_page" enum, but missed to update
type2name string table. This causes incorrect displaying of dma allocation
type.
Fix it by removing "page" string from type2name string table and switch to
use named initializers.

Before (dma_alloc_coherent()):
k3-ringacc 4b800000.ringacc: scather-gather idx 2208 P=d1140000 N=d114 D=d1140000 L=40 DMA_BIDIRECTIONAL dma map error check not applicable
k3-ringacc 4b800000.ringacc: scather-gather idx 2216 P=d1150000 N=d115 D=d1150000 L=40 DMA_BIDIRECTIONAL dma map error check not applicable

After:
k3-ringacc 4b800000.ringacc: coherent idx 2208 P=d1140000 N=d114 D=d1140000 L=40 DMA_BIDIRECTIONAL dma map error check not applicable
k3-ringacc 4b800000.ringacc: coherent idx 2216 P=d1150000 N=d115 D=d1150000 L=40 DMA_BIDIRECTIONAL dma map error check not applicable

Fixes: 2e05ea5cdc ("dma-mapping: implement dma_map_single_attrs using dma_map_page_attrs")
Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
2020-04-08 21:46:57 +02:00
Linus Torvalds 81b6b96475 dma-mapping updates for 5.5-rc1
- improve dma-debug scalability (Eric Dumazet)
  - tiny dma-debug cleanup (Dan Carpenter)
  - check for vmap memory in dma_map_single (Kees Cook)
  - check for dma_addr_t overflows in dma-direct when using
    DMA offsets (Nicolas Saenz Julienne)
  - switch the x86 sta2x11 SOC to use more generic DMA code
    (Nicolas Saenz Julienne)
  - fix arm-nommu dma-ranges handling (Vladimir Murzin)
  - use __initdata in CMA (Shyam Saini)
  - replace the bus dma mask with a limit (Nicolas Saenz Julienne)
  - merge the remapping helpers into the main dma-direct flow (me)
  - switch xtensa to the generic dma remap handling (me)
  - various cleanups around dma_capable (me)
  - remove unused dev arguments to various dma-noncoherent helpers (me)
 -----BEGIN PGP SIGNATURE-----
 
 iQI/BAABCgApFiEEgdbnc3r/njty3Iq9D55TZVIEUYMFAl3f+eULHGhjaEBsc3Qu
 ZGUACgkQD55TZVIEUYPyPg/+PVHCrhmepudQQFHu6wfurE5U77iNnoUifvG+b5z5
 5mHmTMkQwyox6rKDe8NuFApAhz1VJDSUgSelPmvTSOIEIGXCvX1p+GqRSVS5YQON
 aLzGvbWKE8hCpaPdDHKYDauD1FZGMM8L2P5oOMF9X9fQ94xxRqfqJM6c8iD16Sgg
 +aOgPNzTnxQHJFF/Dbt/mjJrKXWI+XF+bgUbH+l9yKa7Dd7ibmJR8yl9hs1jmp0H
 1CZ+CizwnAs57rCd1a6Ybc6gj59tySc03NMnnbTko+KDxrcbD3Ee2tpqHVkkCjYz
 Yl0m4FIpbotrpokL/FIS727bVvkjbWgoeM+kiVPoYzmZea3pq/tFDr6tp/BxDhFj
 TZXSFfgQljlYMD3ppSoklFlfjGriVWV0tPO3arPXwuuMF5EX/IMQmvxei05jpc8n
 iELNXOP9iZZkY4tLHy2hn2uWrxBRrS1WQwlLg9hahlNRzyfFSyHeP0zWlVDt+RgF
 5CCbEI+HQcUqg1FApB30lQNWTn1+dJftrpKVBlgNBIyIa/z2rFbt8GdSnItxjfQX
 /XX8EZbFvF6AcXkgURkYFIoKM/EbYShOSLcYA3PTUtcuTnF6Kk5eimySiGWZTVCS
 prruSFDZJOvL3SnOIMIiYVmBdB7lEbDyLI/VYuhoECXEDCJpVmRktNkJNg4q6/E+
 fjQ=
 =e5wO
 -----END PGP SIGNATURE-----

Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux; tag 'dma-mapping-5.5' of git://git.infradead.org/users/hch/dma-mapping

Pull dma-mapping updates from Christoph Hellwig:

 - improve dma-debug scalability (Eric Dumazet)

 - tiny dma-debug cleanup (Dan Carpenter)

 - check for vmap memory in dma_map_single (Kees Cook)

 - check for dma_addr_t overflows in dma-direct when using DMA offsets
   (Nicolas Saenz Julienne)

 - switch the x86 sta2x11 SOC to use more generic DMA code (Nicolas
   Saenz Julienne)

 - fix arm-nommu dma-ranges handling (Vladimir Murzin)

 - use __initdata in CMA (Shyam Saini)

 - replace the bus dma mask with a limit (Nicolas Saenz Julienne)

 - merge the remapping helpers into the main dma-direct flow (me)

 - switch xtensa to the generic dma remap handling (me)

 - various cleanups around dma_capable (me)

 - remove unused dev arguments to various dma-noncoherent helpers (me)

* 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux:

* tag 'dma-mapping-5.5' of git://git.infradead.org/users/hch/dma-mapping: (22 commits)
  dma-mapping: treat dev->bus_dma_mask as a DMA limit
  dma-direct: exclude dma_direct_map_resource from the min_low_pfn check
  dma-direct: don't check swiotlb=force in dma_direct_map_resource
  dma-debug: clean up put_hash_bucket()
  powerpc: remove support for NULL dev in __phys_to_dma / __dma_to_phys
  dma-direct: avoid a forward declaration for phys_to_dma
  dma-direct: unify the dma_capable definitions
  dma-mapping: drop the dev argument to arch_sync_dma_for_*
  x86/PCI: sta2x11: use default DMA address translation
  dma-direct: check for overflows on 32 bit DMA addresses
  dma-debug: increase HASH_SIZE
  dma-debug: reorder struct dma_debug_entry fields
  xtensa: use the generic uncached segment support
  dma-mapping: merge the generic remapping helpers into dma-direct
  dma-direct: provide mmap and get_sgtable method overrides
  dma-direct: remove the dma_handle argument to __dma_direct_alloc_pages
  dma-direct: remove __dma_direct_free_pages
  usb: core: Remove redundant vmap checks
  kernel: dma-contiguous: mark CMA parameters __initdata/__initconst
  dma-debug: add a schedule point in debug_dma_dump_mappings()
  ...
2019-11-28 11:16:43 -08:00
Dan Carpenter 50f579a239 dma-debug: clean up put_hash_bucket()
The put_hash_bucket() is a bit cleaner if it takes an unsigned long
directly instead of a pointer to unsigned long.

Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
2019-11-20 20:31:41 +01:00
Eric Dumazet 5e76f56457 dma-debug: increase HASH_SIZE
With modern NIC, it is not unusual having about ~256,000 active dma
mappings and a hash size of 1024 buckets is too small.

Forcing full cache line per bucket does not seem useful, especially now
that we have contention on free_entries_lock for allocations and freeing
of entries.  Better use the space to fit more buckets.

Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
2019-11-11 10:52:18 +01:00
Eric Dumazet d3694f3073 dma-debug: reorder struct dma_debug_entry fields
Move all fields used during exact match lookups to the first cache line.
This makes debug_dma_mapping_error() and friends about 50% faster.

Since it removes two 32bit holes, force a cacheline alignment on struct
dma_debug_entry.

Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
2019-11-11 10:52:18 +01:00
Eric Dumazet 9ff6aa027d dma-debug: add a schedule point in debug_dma_dump_mappings()
debug_dma_dump_mappings() can take a lot of cpu cycles :

lpk43:/# time wc -l /sys/kernel/debug/dma-api/dump
163435 /sys/kernel/debug/dma-api/dump

real	0m0.463s
user	0m0.003s
sys	0m0.459s

Let's add a cond_resched() to avoid holding cpu for too long.

Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: Corentin Labbe <clabbe@baylibre.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
2019-10-30 11:09:35 -07:00
Kefeng Wang fc65104c7c dma-debug: Use pr_warn instead of pr_warning
As said in commit f2c2cbcc35 ("powerpc: Use pr_warn instead of
pr_warning"), removing pr_warning so all logging messages use a
consistent <prefix>_warn style. Let's do it.

Link: http://lkml.kernel.org/r/20191018031850.48498-25-wangkefeng.wang@huawei.com
To: linux-kernel@vger.kernel.org
Cc: Christoph Hellwig <hch@lst.de>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
Reviewed-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Signed-off-by: Petr Mladek <pmladek@suse.com>
2019-10-18 15:01:56 +02:00
Thomas Gleixner 4505153954 treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 333
Based on 1 normalized pattern(s):

  this program is free software you can redistribute it and or modify
  it under the terms of the gnu general public license version 2 as
  published by the free software foundation this program is
  distributed in the hope that it will be useful but without any
  warranty without even the implied warranty of merchantability or
  fitness for a particular purpose see the gnu general public license
  for more details you should have received a copy of the gnu general
  public license along with this program if not write to the free
  software foundation inc 59 temple place suite 330 boston ma 02111
  1307 usa

extracted by the scancode license scanner the SPDX license identifier

  GPL-2.0-only

has been chosen to replace the boilerplate/reference in 136 file(s).

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Alexios Zavras <alexios.zavras@intel.com>
Reviewed-by: Allison Randal <allison@lohutok.net>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190530000436.384967451@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-06-05 17:37:06 +02:00
Thomas Gleixner 746017ed8d dma/debug: Simplify stracktrace retrieval
Replace the indirection through struct stack_trace with an invocation of
the storage array based interface.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: iommu@lists.linux-foundation.org
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Alexander Potapenko <glider@google.com>
Cc: Alexey Dobriyan <adobriyan@gmail.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Christoph Lameter <cl@linux.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: linux-mm@kvack.org
Cc: David Rientjes <rientjes@google.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: kasan-dev@googlegroups.com
Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
Cc: Akinobu Mita <akinobu.mita@gmail.com>
Cc: Johannes Thumshirn <jthumshirn@suse.de>
Cc: David Sterba <dsterba@suse.com>
Cc: Chris Mason <clm@fb.com>
Cc: Josef Bacik <josef@toxicpanda.com>
Cc: linux-btrfs@vger.kernel.org
Cc: dm-devel@redhat.com
Cc: Mike Snitzer <snitzer@redhat.com>
Cc: Alasdair Kergon <agk@redhat.com>
Cc: Daniel Vetter <daniel@ffwll.ch>
Cc: intel-gfx@lists.freedesktop.org
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: dri-devel@lists.freedesktop.org
Cc: David Airlie <airlied@linux.ie>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Tom Zanussi <tom.zanussi@linux.intel.com>
Cc: Miroslav Benes <mbenes@suse.cz>
Cc: linux-arch@vger.kernel.org
Link: https://lkml.kernel.org/r/20190425094802.248658135@linutronix.de
2019-04-29 12:37:50 +02:00
Scott Wood 8c5165430c dma-debug: only skip one stackframe entry
With skip set to 1, I get a traceback like this:

[  106.867637] DMA-API: Mapped at:
[  106.870784]  afu_dma_map_region+0x2cd/0x4f0 [dfl_afu]
[  106.875839]  afu_ioctl+0x258/0x380 [dfl_afu]
[  106.880108]  do_vfs_ioctl+0xa9/0x720
[  106.883688]  ksys_ioctl+0x60/0x90
[  106.887007]  __x64_sys_ioctl+0x16/0x20

With the previous value of 2, afu_dma_map_region was being omitted.  I
suspect that the code paths have simply changed since the value of 2 was
chosen a decade ago, but it's also possible that it varies based on which
mapping function was used, compiler inlining choices, etc.  In any case,
it's best to err on the side of skipping less.

Signed-off-by: Scott Wood <swood@redhat.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
2019-04-11 19:29:50 +02:00
Corentin Labbe 0a3b192c26 dma-debug: add dumping facility via debugfs
While debugging a DMA mapping leak, I needed to access
debug_dma_dump_mappings() but easily from user space.

This patch adds a /sys/kernel/debug/dma-api/dump file which contain all
current DMA mapping.

Signed-off-by: Corentin Labbe <clabbe@baylibre.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
2019-02-01 10:06:44 +01:00
Greg Kroah-Hartman 8e4d81b98b dma: debug: no need to check return value of debugfs_create functions
When calling debugfs functions, there is no need to ever check the
return value.  The function can work or not, but the code logic should
never do something different based on this.

Also delete the variables for the file dentries for the debugfs entries
as they are never used at all once they are created.

Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
[hch: moved dma_debug_dent to function scope and renamed it]
Signed-off-by: Christoph Hellwig <hch@lst.de>
2019-02-01 10:02:32 +01:00
Christoph Hellwig 48e638fb68 dma-mapping: remove a few unused exports
Now that the slow path DMA API calls are implemented out of line a few
helpers only used by them don't need to be exported anymore.

Signed-off-by: Christoph Hellwig <hch@lst.de>
2019-01-04 09:03:17 +01:00
Christoph Hellwig 2e05ea5cdc dma-mapping: implement dma_map_single_attrs using dma_map_page_attrs
And also switch the way we implement the unmap side around to stay
consistent.  This ensures dma-debug works again because it records which
function we used for mapping to ensure it is also used for unmapping,
and also reduces further code duplication.  Last but not least this
also officially allows calling dma_sync_single_* for mappings created
using dma_map_page, which is perfectly fine given that the sync calls
only take a dma_addr_t, but not a virtual address or struct page.

Fixes: 7f0fee242e ("dma-mapping: merge dma_unmap_page_attrs and dma_unmap_single_attrs")
Signed-off-by: Christoph Hellwig <hch@lst.de>
Tested-by: LABBE Corentin <clabbe.montjoie@gmail.com>
2019-01-04 09:02:17 +01:00
Christoph Hellwig 8d59b5f2a4 dma-mapping: simplify the dma_sync_single_range_for_{cpu,device} implementation
We can just call the regular calls after adding offset the the address instead
of reimplementing them.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Jesper Dangaard Brouer <brouer@redhat.com>
Tested-by: Jesper Dangaard Brouer <brouer@redhat.com>
Tested-by: Tony Luck <tony.luck@intel.com>
2018-12-13 21:06:08 +01:00
Robin Murphy ad78dee0b6 dma-debug: Batch dma_debug_entry allocation
DMA debug entries are one of those things which aren't that useful
individually - we will always want some larger quantity of them - and
which we don't really need to manage the exact number of - we only care
about having 'enough'. In that regard, the current behaviour of creating
them one-by-one leads to a lot of unwarranted function call overhead and
memory wasted on alignment padding.

Now that we don't have to worry about freeing anything via
dma_debug_resize_entries(), we can optimise the allocation behaviour by
grabbing whole pages at once, which will save considerably on the
aforementioned overheads, and probably offer a little more cache/TLB
locality benefit for traversing the lists under normal operation. This
should also give even less reason for an architecture-level override of
the preallocation size, so make the definition unconditional - if there
is still any desire to change the compile-time value for some platforms
it would be better off as a Kconfig option anyway.

Since freeing a whole page of entries at once becomes enough of a
challenge that it's not really worth complicating dma_debug_init(), we
may as well tweak the preallocation behaviour such that as long as we
manage to allocate *some* pages, we can leave debugging enabled on a
best-effort basis rather than otherwise wasting them.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Tested-by: Qian Cai <cai@lca.pw>
Signed-off-by: Christoph Hellwig <hch@lst.de>
2018-12-11 14:32:13 +01:00
Robin Murphy 0cb0e25e42 dma/debug: Remove dma_debug_resize_entries()
With the only caller now gone, we can clean up this part of dma-debug's
exposed internals and make way to tweak the allocation behaviour.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Tested-by: Qian Cai <cai@lca.pw>
Signed-off-by: Christoph Hellwig <hch@lst.de>
2018-12-11 14:32:13 +01:00
Robin Murphy ceb51173b2 dma-debug: Make leak-like behaviour apparent
Now that we can dynamically allocate DMA debug entries to cope with
drivers maintaining excessively large numbers of live mappings, a driver
which *does* actually have a bug leaking mappings (and is not unloaded)
will no longer trigger the "DMA-API: debugging out of memory - disabling"
message until it gets to actual kernel OOM conditions, which means it
could go unnoticed for a while. To that end, let's inform the user each
time the pool has grown to a multiple of its initial size, which should
make it apparent that they either have a leak or might want to increase
the preallocation size.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Tested-by: Qian Cai <cai@lca.pw>
Signed-off-by: Christoph Hellwig <hch@lst.de>
2018-12-11 14:31:18 +01:00
Robin Murphy 2b9d9ac02b dma-debug: Dynamically expand the dma_debug_entry pool
Certain drivers such as large multi-queue network adapters can use pools
of mapped DMA buffers larger than the default dma_debug_entry pool of
65536 entries, with the result that merely probing such a device can
cause DMA debug to disable itself during boot unless explicitly given an
appropriate "dma_debug_entries=..." option.

Developers trying to debug some other driver on such a system may not be
immediately aware of this, and at worst it can hide bugs if they fail to
realise that dma-debug has already disabled itself unexpectedly by the
time their code of interest gets to run. Even once they do realise, it
can be a bit of a pain to emprirically determine a suitable number of
preallocated entries to configure, short of massively over-allocating.

There's really no need for such a static limit, though, since we can
quite easily expand the pool at runtime in those rare cases that the
preallocated entries are insufficient, which is arguably the least
surprising and most useful behaviour. To that end, refactor the
prealloc_memory() logic a little bit to generalise it for runtime
reallocations as well.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Tested-by: Qian Cai <cai@lca.pw>
Signed-off-by: Christoph Hellwig <hch@lst.de>
2018-12-11 14:31:17 +01:00
Robin Murphy f737b095c6 dma-debug: Use pr_fmt()
Use pr_fmt() to generate the "DMA-API: " prefix consistently. This
results in it being added to a couple of pr_*() messages which were
missing it before, and for the err_printk() calls moves it to the actual
start of the message instead of somewhere in the middle.

Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Tested-by: Qian Cai <cai@lca.pw>
Signed-off-by: Christoph Hellwig <hch@lst.de>
2018-12-11 14:31:17 +01:00
Robin Murphy 9f191555ba dma-debug: Expose nr_total_entries in debugfs
Expose nr_total_entries in debugfs, so that {num,min}_free_entries
become even more meaningful to users interested in current/maximum
utilisation. This becomes even more relevant once nr_total_entries
may change at runtime beyond just the existing AMD GART debug code.

Suggested-by: John Garry <john.garry@huawei.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Tested-by: Qian Cai <cai@lca.pw>
Signed-off-by: Christoph Hellwig <hch@lst.de>
2018-12-11 14:31:17 +01:00