JIRA: https://issues.redhat.com/browse/RHEL-49539
Upstream Status: merged into the linux.git
commit 43698cd6c02ddcf3b4ffb34e5da0be3e801027fe
Author: Tom Zanussi <tom.zanussi@linux.intel.com>
Date: Mon Mar 4 15:20:11 2024 -0600
crypto: iaa - Change iaa statistics to atomic64_t
Change all the iaa statistics to use atomic64_t instead of the current
u64, to avoid potentially inconsistent counts.
Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Vladis Dronov <vdronov@redhat.com>
JIRA: https://issues.redhat.com/browse/RHEL-29685
Upstream Status: merged into the linux.git
commit 3274819b3c81c18c01e3c0e0ea726870ec237ac0
Author: Tom Zanussi <tom.zanussi@linux.intel.com>
Date: Mon Jan 8 16:53:48 2024 -0600
crypto: iaa - Remove header table code
The header table and related code is currently unused - it was
included and used for canned mode, but canned mode has been removed,
so this code can be safely removed as well.
This indirectly fixes a bug reported by Dan Carpenter.
Reported-by: Dan Carpenter <dan.carpenter@linaro.org>
Closes: https://lore.kernel.org/linux-crypto/b2e0bd974981291e16882686a2b9b1db3986abe4.camel@linux.intel.com/T/#m4403253d6a4347a925fab4fc1cdb4ef7c095fb86
Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Vladis Dronov <vdronov@redhat.com>
JIRA: https://issues.redhat.com/browse/RHEL-20145
Upstream Status: merged into the linux.git
commit 93382a91632a5d88bb9bb0ff1fea872fe87f5dc2
Author: Tom Zanussi <tom.zanussi@linux.intel.com>
Date: Tue Dec 5 15:25:29 2023 -0600
crypto: iaa - Add IAA Compression Accelerator stats
Add support for optional debugfs statistics support for the IAA
Compression Accelerator. This is enabled by the kernel config item:
CRYPTO_DEV_IAA_CRYPTO_STATS
When enabled, the IAA crypto driver will generate statistics which can
be accessed at /sys/kernel/debug/iaa-crypto/.
See Documentation/driver-api/crypto/iax/iax-crypto.rst for details.
Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Vladis Dronov <vdronov@redhat.com>
JIRA: https://issues.redhat.com/browse/RHEL-20145
Upstream Status: merged into the linux.git
commit 09646c98d0bfed47930d9eb0d66c323fae70a5e0
Author: Tom Zanussi <tom.zanussi@linux.intel.com>
Date: Tue Dec 5 15:25:28 2023 -0600
crypto: iaa - Add irq support for the crypto async interface
The existing iaa crypto async support provides an implementation that
satisfies the interface but does so in a synchronous manner - it fills
and submits the IDXD descriptor and then waits for it to complete
before returning. This isn't a problem at the moment, since all
existing callers (e.g. zswap) wrap any asynchronous callees in a
synchronous wrapper anyway.
This change makes the iaa crypto async implementation truly
asynchronous: it fills and submits the IDXD descriptor, then returns
immediately with -EINPROGRESS. It also sets the descriptor's 'request
completion irq' bit and sets up a callback with the IDXD driver which
is called when the operation completes and the irq fires. The
existing callers such as zswap use synchronous wrappers to deal with
-EINPROGRESS and so work as expected without any changes.
This mode can be enabled by writing 'async_irq' to the sync_mode
iaa_crypto driver attribute:
echo async_irq > /sys/bus/dsa/drivers/crypto/sync_mode
Async mode without interrupts (caller must poll) can be enabled by
writing 'async' to it:
echo async > /sys/bus/dsa/drivers/crypto/sync_mode
The default sync mode can be enabled by writing 'sync' to it:
echo sync > /sys/bus/dsa/drivers/crypto/sync_mode
The sync_mode value setting at the time the IAA algorithms are
registered is captured in each algorithm's crypto_ctx and used for all
compresses and decompresses when using a given algorithm.
Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Vladis Dronov <vdronov@redhat.com>
JIRA: https://issues.redhat.com/browse/RHEL-20145
Upstream Status: merged into the linux.git
commit 2ec6761df889fdf896fde761abd447596dd8f8c2
Author: Tom Zanussi <tom.zanussi@linux.intel.com>
Date: Tue Dec 5 15:25:27 2023 -0600
crypto: iaa - Add support for deflate-iaa compression algorithm
This patch registers the deflate-iaa deflate compression algorithm and
hooks it up to the IAA hardware using the 'fixed' compression mode
introduced in the previous patch.
Because the IAA hardware has a 4k history-window limitation, only
buffers <= 4k, or that have been compressed using a <= 4k history
window, are technically compliant with the deflate spec, which allows
for a window of up to 32k. Because of this limitation, the IAA fixed
mode deflate algorithm is given its own algorithm name, 'deflate-iaa'.
With this change, the deflate-iaa crypto algorithm is registered and
operational, and compression and decompression operations are fully
enabled following the successful binding of the first IAA workqueue
to the iaa_crypto sub-driver.
when there are no IAA workqueues bound to the driver, the IAA crypto
algorithm can be unregistered by removing the module.
A new iaa_crypto 'verify_compress' driver attribute is also added,
allowing the user to toggle compression verification. If set, each
compress will be internally decompressed and the contents verified,
returning error codes if unsuccessful. This can be toggled with 0/1:
echo 0 > /sys/bus/dsa/drivers/crypto/verify_compress
The default setting is '1' - verify all compresses.
The verify_compress value setting at the time the algorithm is
registered is captured in the algorithm's crypto_ctx and used for all
compresses when using the algorithm.
[ Based on work originally by George Powley, Jing Lin and Kyung Min
Park ]
Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Vladis Dronov <vdronov@redhat.com>
JIRA: https://issues.redhat.com/browse/RHEL-20145
Upstream Status: merged into the linux.git
commit b190447e0fa3ef7355480d641d078962e03768b4
Author: Tom Zanussi <tom.zanussi@linux.intel.com>
Date: Tue Dec 5 15:25:26 2023 -0600
crypto: iaa - Add compression mode management along with fixed mode
Define an in-kernel API for adding and removing compression modes,
which can be used by kernel modules or other kernel code that
implements IAA compression modes.
Also add a separate file, iaa_crypto_comp_fixed.c, containing huffman
tables generated for the IAA 'fixed' compression mode. Future
compression modes can be added in a similar fashion.
One or more crypto compression algorithms will be created for each
compression mode, each of which can be selected as the compression
algorithm to be used by a particular facility.
Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Vladis Dronov <vdronov@redhat.com>
JIRA: https://issues.redhat.com/browse/RHEL-20145
Upstream Status: merged into the linux.git
commit f57bf3f78377d66af89a6d0c6d926ffb1f590b5d
Author: Tom Zanussi <tom.zanussi@linux.intel.com>
Date: Tue Dec 5 15:25:25 2023 -0600
crypto: iaa - Add per-cpu workqueue table with rebalancing
The iaa compression/decompression algorithms in later patches need a
way to retrieve an appropriate IAA workqueue depending on how close
the associated IAA device is to the current cpu.
For this purpose, add a per-cpu array of workqueues such that an
appropriate workqueue can be retrieved by simply accessing the per-cpu
array.
Whenever a new workqueue is bound to or unbound from the iaa_crypto
driver, the available workqueues are 'rebalanced' such that work
submitted from a particular CPU is given to the most appropriate
workqueue available. There currently isn't any way for the user to
tweak the way this is done internally - if necessary, knobs can be
added later for that purpose. Current best practice is to configure
and bind at least one workqueue for each IAA device, but as long as
there is at least one workqueue configured and bound to any IAA device
in the system, the iaa_crypto driver will work, albeit most likely not
as efficiently.
[ Based on work originally by George Powley, Jing Lin and Kyung Min
Park ]
Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Vladis Dronov <vdronov@redhat.com>
JIRA: https://issues.redhat.com/browse/RHEL-20145
Upstream Status: merged into the linux.git
commit ea7a5cbb43696cfacf73e61916d1860ac30b5b2f
Author: Tom Zanussi <tom.zanussi@linux.intel.com>
Date: Tue Dec 5 15:25:24 2023 -0600
crypto: iaa - Add Intel IAA Compression Accelerator crypto driver core
The Intel Analytics Accelerator (IAA) is a hardware accelerator that
provides very high thoughput compression/decompression compatible with
the DEFLATE compression standard described in RFC 1951, which is the
compression/decompression algorithm exported by this module.
Users can select IAA compress/decompress acceleration by specifying
one of the deflate-iaa* algorithms as the compression algorithm to use
by whatever facility allows asynchronous compression algorithms to be
selected.
For example, zswap can select the IAA fixed deflate algorithm
'deflate-iaa' via:
# echo deflate-iaa > /sys/module/zswap/parameters/compressor
This patch adds iaa_crypto as an idxd sub-driver and tracks iaa
devices and workqueues as they are probed or removed.
[ Based on work originally by George Powley, Jing Lin and Kyung Min
Park ]
Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Vladis Dronov <vdronov@redhat.com>
Vladis Dronov