JIRA: https://issues.redhat.com/browse/RHEL-30141
commit 773bb766ca4a05bf363203030b72b10088869224
Author: Leon Romanovsky <leonro@nvidia.com>
Date: Tue Mar 12 13:55:22 2024 +0200
xfrm: Allow UDP encapsulation only in offload modes
The missing check of x->encap caused to the situation where GSO packets
were created with UDP encapsulation.
As a solution return the encap check for non-offloaded SA.
Fixes: 983a73da1f99 ("xfrm: Pass UDP encapsulation in TX packet offload")
Closes: https://lore.kernel.org/all/a650221ae500f0c7cf496c61c96c1b103dcb6f67.camel@redhat.com
Reported-by: Paolo Abeni <pabeni@redhat.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Michal Schmidt <mschmidt@redhat.com>
JIRA: https://issues.redhat.com/browse/RHEL-30141
Conflicts:
net/xfrm/xfrm_device.c
- Context: RHEL has an additional tech-preview mark.
commit 89edf40220be8e68922beb54a06fdfc66f743c39
Author: Leon Romanovsky <leonro@nvidia.com>
Date: Wed Jul 19 12:26:56 2023 +0300
xfrm: Support UDP encapsulation in packet offload mode
Since mlx5 supports UDP encapsulation in packet offload, change the XFRM
core to allow users to configure it.
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Acked-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Signed-off-by: Michal Schmidt <mschmidt@redhat.com>
JIRA: https://issues.redhat.com/browse/RHEL-882
Upstream-status: v6.4-rc1
commit 3e1c957f9a3b248f47f8b39b607002d948fd17d4
Author: Leon Romanovsky <leon@kernel.org>
Date: Thu Mar 30 11:02:26 2023 +0300
xfrm: don't require advance ESN callback for packet offload
In packet offload mode, the hardware is responsible to manage
replay window and advance ESN. In that mode, there won't any
call to .xdo_dev_state_advance_esn callback.
So relax current check for existence of that callback.
Link: https://lore.kernel.org/r/9f3dfc3fef2cfcd191f0c5eee7cf0aa74e7f7786.1680162300.git.leonro@nvidia.com
Reviewed-by: Raed Salem <raeds@nvidia.com>
Acked-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Signed-off-by: Mohammad Kabat <mkabat@redhat.com>
JIRA: https://issues.redhat.com/browse/RHEL-12679
commit d457a0e329b0bfd3a1450e0b1a18cd2b47a25a08
Author: Eric Dumazet <edumazet@google.com>
Date: Thu Jun 8 19:17:37 2023 +0000
net: move gso declarations and functions to their own files
Move declarations into include/net/gso.h and code into net/core/gso.c
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: Stanislav Fomichev <sdf@google.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
Reviewed-by: David Ahern <dsahern@kernel.org>
Link: https://lore.kernel.org/r/20230608191738.3947077-1-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Ivan Vecera <ivecera@redhat.com>
Bugzilla: https://bugzilla.redhat.com/2165364
Upstream-status: v6.3-rc1
commit 3089386db0901ac6ac3d99fbd601212c98217e60
Author: Leon Romanovsky <leon@kernel.org>
Date: Tue Jan 24 13:54:57 2023 +0200
xfrm: extend add policy callback to set failure reason
Almost all validation logic is in the drivers, but they are
missing reliable way to convey failure reason to userspace
applications.
Let's use extack to return this information to users.
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Mohammad Kabat <mkabat@redhat.com>
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2218900
Tested: IPsec tests
commit ec8f32ad9a65a8cbb465b69e154aaec9d2fe45c4
Author: Leon Romanovsky <leon@kernel.org>
Date: Wed Apr 19 15:19:08 2023 +0300
xfrm: Fix leak of dev tracker
At the stage of direction checks, the netdev reference tracker is
already initialized, but released with wrong *_put() call.
Fixes: 919e43fad516 ("xfrm: add an interface to offload policy")
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Sabrina Dubroca <sdubroca@redhat.com>
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2189210
Tested: ipsec tests
commit 4b549ccce941798703f159b227aa28c716aa78fa
Author: Christian Langrock <christian.langrock@secunet.com>
Date: Mon Oct 17 08:34:47 2022 +0200
xfrm: replay: Fix ESN wrap around for GSO
When using GSO it can happen that the wrong seq_hi is used for the last
packets before the wrap around. This can lead to double usage of a
sequence number. To avoid this, we should serialize this last GSO
packet.
Fixes: d7dbefc45c ("xfrm: Add xfrm_replay_overflow functions for offloading")
Co-developed-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Christian Langrock <christian.langrock@secunet.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Sabrina Dubroca <sdubroca@redhat.com>
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2178699
Upstream Status: RHEL only
Tested: kernel gets tainted only when using packet offload
We don't want to provide full support for packet offload just
yet. Mark as tech preview when packet offload. Existing uses of
IPsec (software mode and crypto offload) are supported.
Signed-off-by: Sabrina Dubroca <sdubroca@redhat.com>
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2178699
Tested: basic IPsec tests + offload tests
Conflicts: skip nfp bits, missing ipsec support in driver
commit 7681a4f58fb9c338d6dfe1181607f84c793d77de
Author: Leon Romanovsky <leon@kernel.org>
Date: Tue Jan 24 13:54:59 2023 +0200
xfrm: extend add state callback to set failure reason
Almost all validation logic is in the drivers, but they are
missing reliable way to convey failure reason to userspace
applications.
Let's use extack to return this information to users.
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sabrina Dubroca <sdubroca@redhat.com>
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2178699
Tested: basic IPsec tests + offload tests
commit abe2343d37c2b4361547d5d31e17340ff9ec7356
Author: Colin Ian King <colin.i.king@gmail.com>
Date: Wed Dec 7 09:23:14 2022 +0000
xfrm: Fix spelling mistake "oflload" -> "offload"
There is a spelling mistake in a NL_SET_ERR_MSG message. Fix it.
Signed-off-by: Colin Ian King <colin.i.king@gmail.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Sabrina Dubroca <sdubroca@redhat.com>
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2178699
Tested: basic IPsec tests + offload tests
commit f8a70afafc1759b1fca4baaa891625dde49c10b7
Author: Leon Romanovsky <leon@kernel.org>
Date: Fri Dec 2 20:41:30 2022 +0200
xfrm: add TX datapath support for IPsec packet offload mode
In IPsec packet mode, the device is going to encrypt and encapsulate
packets that are associated with offloaded policy. After successful
policy lookup to indicate if packets should be offloaded or not,
the stack forwards packets to the device to do the magic.
Signed-off-by: Raed Salem <raeds@nvidia.com>
Signed-off-by: Huy Nguyen <huyn@nvidia.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Sabrina Dubroca <sdubroca@redhat.com>
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2178699
Tested: basic IPsec tests + offload tests
commit 919e43fad5163a8ceb39826ecdee897a9f799351
Author: Leon Romanovsky <leon@kernel.org>
Date: Fri Dec 2 20:41:29 2022 +0200
xfrm: add an interface to offload policy
Extend netlink interface to add and delete XFRM policy from the device.
This functionality is a first step to implement packet IPsec offload solution.
Signed-off-by: Raed Salem <raeds@nvidia.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Sabrina Dubroca <sdubroca@redhat.com>
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2178699
Tested: basic IPsec tests + offload tests
Conflicts: skip nfp bits, missing ipsec support in driver
commit 62f6eca5de103c6823f6ca2abbf2ee242e132207
Author: Leon Romanovsky <leon@kernel.org>
Date: Fri Dec 2 20:41:28 2022 +0200
xfrm: allow state packet offload mode
Allow users to configure xfrm states with packet offload mode.
The packet mode must be requested both for policy and state, and
such requires us to do not implement fallback.
We explicitly return an error if requested packet mode can't
be configured.
Reviewed-by: Raed Salem <raeds@nvidia.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Sabrina Dubroca <sdubroca@redhat.com>
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2178699
Tested: basic IPsec tests + offload tests
commit d14f28b8c1de668bab863bf5892a49c824cb110d
Author: Leon Romanovsky <leon@kernel.org>
Date: Fri Dec 2 20:41:27 2022 +0200
xfrm: add new packet offload flag
In the next patches, the xfrm core code will be extended to support
new type of offload - packet offload. In that mode, both policy and state
should be specially configured in order to perform whole offloaded data
path.
Full offload takes care of encryption, decryption, encapsulation and
other operations with headers.
As this mode is new for XFRM policy flow, we can "start fresh" with flag
bits and release first and second bit for future use.
Reviewed-by: Raed Salem <raeds@nvidia.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Sabrina Dubroca <sdubroca@redhat.com>
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2178699
Tested: basic IPsec tests + offload tests
commit 254c4a824c7c6a53360bc4974710e4213b8b7f5d
Author: Leon Romanovsky <leon@kernel.org>
Date: Thu May 5 13:06:45 2022 +0300
xfrm: drop not needed flags variable in XFRM offload struct
After drivers were converted to rely on direction, the flags is not
used anymore and can be removed.
Reviewed-by: Raed Salem <raeds@nvidia.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Sabrina Dubroca <sdubroca@redhat.com>
Bugzilla: https://bugzilla.redhat.com/2175258
Conflicts:
- Removed chunks of unsupported protocol AX.25
- Renamed the funtions also in ipvlan. Commit 40b9d1ab63f5 ("ipvlan: hold lower
dev to avoid possible use-after-free") was backported out of order so it had
to use the old functions names.
commit d62607c3fe45911b2331fac073355a8c914bbde2
Author: Jakub Kicinski <kuba@kernel.org>
Date: Tue Jun 7 21:39:55 2022 -0700
net: rename reference+tracking helpers
Netdev reference helpers have a dev_ prefix for historic
reasons. Renaming the old helpers would be too much churn
but we can rename the tracking ones which are relatively
recent and should be the default for new code.
Rename:
dev_hold_track() -> netdev_hold()
dev_put_track() -> netdev_put()
dev_replace_track() -> netdev_ref_replace()
Link: https://lore.kernel.org/r/20220608043955.919359-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Íñigo Huguet <ihuguet@redhat.com>
MR: https://gitlab.com/redhat/centos-stream/src/kernel/centos-stream-9/-/merge_requests/1723
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2139496
Tested: basic IPsec tests + offload tests
This backports part of the series merged through upstream commit
0f9008e5c510 ("Be explicit with XFRM offload direction"). The driver
patches will be backported later on by their respective maintainers,
and I'll follow up afterwards with the final commit of the series
(commit 254c4a824c7c ("xfrm: drop not needed flags variable in XFRM
offload struct"), once 'flags' is indeed unused).
Commit b01a277a0520 ("xfrm: free not used XFRM_ESP_NO_TRAILER flag")
is also omitted, as the flag is still used in RHEL9.
Signed-off-by: Sabrina Dubroca <sdubroca@redhat.com>
Approved-by: Xin Long <lxin@redhat.com>
Approved-by: Antoine Tenart <atenart@redhat.com>
Signed-off-by: Herton R. Krzesinski <herton@redhat.com>
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2139496
Tested: basic IPsec tests + offload tests
commit 482db2f1dd211f73ad9d71e33ae15c1df6379982
Author: Leon Romanovsky <leon@kernel.org>
Date: Thu May 5 13:06:41 2022 +0300
xfrm: store and rely on direction to construct offload flags
XFRM state doesn't need anything from flags except to understand
direction, so store it separately. For future patches, such change
will allow us to reuse xfrm_dev_offload for policy offload too, which
has three possible directions instead of two.
Reviewed-by: Raed Salem <raeds@nvidia.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Sabrina Dubroca <sdubroca@redhat.com>
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2139496
Tested: basic IPsec tests + offload tests
commit 87e0a94e60ea2e29be9dec6bc146fbc9861a4055
Author: Leon Romanovsky <leon@kernel.org>
Date: Thu May 5 13:06:40 2022 +0300
xfrm: rename xfrm_state_offload struct to allow reuse
The struct xfrm_state_offload has all fields needed to hold information
for offloaded policies too. In order to do not create new struct with
same fields, let's rename existing one and reuse it later.
Reviewed-by: Raed Salem <raeds@nvidia.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Acked-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Sabrina Dubroca <sdubroca@redhat.com>
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2139496
Tested: basic IPsec tests + offload tests
commit a36708e646586f74d073199828ed878b223e988d
Author: Leon Romanovsky <leon@kernel.org>
Date: Thu May 5 13:06:39 2022 +0300
xfrm: delete not used number of external headers
num_exthdrs is set but never used, so delete it.
Reviewed-by: Raed Salem <raeds@nvidia.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Acked-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Sabrina Dubroca <sdubroca@redhat.com>
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2130850
Conflicts:
- drivers/net/vxlan.c: file is not moved to drivers/net/vxlan/vxlan_core.c
due to missing 6765393614ea8 ("vxlan: move to its own directory");
context difference due to missing 4095e0e1328a3 ("drivers: vxlan:
vnifilter: per vni stats")
- net/core/dev.c: code difference in __netif_receive_skb_core due to
already applied 9f8ed577c2881 ("net: skb: rename
SKB_DROP_REASON_PTYPE_ABSENT"). Result is like upstream now.
- net/core/gro_cells.c: context difference due to already applied
5dcd08cd1991 ("net: Fix data-races around netdev_max_backlog.")
commit 625788b5844511cf4c30cffa7fa0bc3a69cebc82
Author: Eric Dumazet <edumazet@google.com>
Date: Thu Mar 10 21:14:20 2022 -0800
net: add per-cpu storage and net->core_stats
Before adding yet another possibly contended atomic_long_t,
it is time to add per-cpu storage for existing ones:
dev->tx_dropped, dev->rx_dropped, and dev->rx_nohandler
Because many devices do not have to increment such counters,
allocate the per-cpu storage on demand, so that dev_get_stats()
does not have to spend considerable time folding zero counters.
Note that some drivers have abused these counters which
were supposed to be only used by core networking stack.
v4: should use per_cpu_ptr() in dev_get_stats() (Jakub)
v3: added a READ_ONCE() in netdev_core_stats_alloc() (Paolo)
v2: add a missing include (reported by kernel test robot <lkp@intel.com>)
Change in netdev_core_stats_alloc() (Jakub)
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: jeffreyji <jeffreyji@google.com>
Reviewed-by: Brian Vazquez <brianvv@google.com>
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
Acked-by: Paolo Abeni <pabeni@redhat.com>
Link: https://lore.kernel.org/r/20220311051420.2608812-1-eric.dumazet@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Felix Maurer <fmaurer@redhat.com>
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2080384
Tested: basic IPsec tests
commit 7c76ecd9c99b6e9a771d813ab1aa7fa428b3ade1
Author: Leon Romanovsky <leon@kernel.org>
Date: Tue Feb 8 16:14:32 2022 +0200
xfrm: enforce validity of offload input flags
struct xfrm_user_offload has flags variable that received user input,
but kernel didn't check if valid bits were provided. It caused a situation
where not sanitized input was forwarded directly to the drivers.
For example, XFRM_OFFLOAD_IPV6 define that was exposed, was used by
strongswan, but not implemented in the kernel at all.
As a solution, check and sanitize input flags to forward
XFRM_OFFLOAD_INBOUND to the drivers.
Fixes: d77e38e612 ("xfrm: Add an IPsec hardware offloading API")
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Sabrina Dubroca <sdubroca@redhat.com>
In case of xfrm offload, if xdo_dev_state_add() of driver returns
-EOPNOTSUPP, xfrm offload fallback is failed.
In xfrm state_add() both xso->dev and xso->real_dev are initialized to
dev and when err(-EOPNOTSUPP) is returned only xso->dev is set to null.
So in this scenario the condition in func validate_xmit_xfrm(),
if ((x->xso.dev != dev) && (x->xso.real_dev == dev))
return skb;
returns true, due to which skb is returned without calling esp_xmit()
below which has fallback code. Hence the CRYPTO_FALLBACK is failing.
So fixing this with by keeping x->xso.real_dev as NULL when err is
returned in func xfrm_dev_state_add().
Fixes: bdfd2d1fa7 ("bonding/xfrm: use real_dev instead of slave_dev")
Signed-off-by: Ayush Sawal <ayush.sawal@chelsio.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Commit 94579ac3f6 ("xfrm: Fix double ESP trailer insertion in IPsec
crypto offload.") added a XFRM_XMIT flag to avoid duplicate ESP trailer
insertion on HW offload. This flag is set on the secpath that is shared
amongst segments. This lead to a situation where some segments are
not transformed correctly when segmentation happens at layer 3.
Fix this by using private skb extensions for segmented and hw offloaded
ESP packets.
Fixes: 94579ac3f6 ("xfrm: Fix double ESP trailer insertion in IPsec crypto offload.")
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Minor overlapping changes in xfrm_device.c, between the double
ESP trailing bug fix setting the XFRM_INIT flag and the changes
in net-next preparing for bonding encryption support.
Signed-off-by: David S. Miller <davem@davemloft.net>
Rather than requiring every hw crypto capable NIC driver to do a check for
slave_dev being set, set real_dev in the xfrm layer and xso init time, and
then override it in the bonding driver as needed. Then NIC drivers can
always use real_dev, and at the same time, we eliminate the use of a
variable name that probably shouldn't have been used in the first place,
particularly given recent current events.
CC: Boris Pismenny <borisp@mellanox.com>
CC: Saeed Mahameed <saeedm@mellanox.com>
CC: Leon Romanovsky <leon@kernel.org>
CC: Jay Vosburgh <j.vosburgh@gmail.com>
CC: Veaceslav Falico <vfalico@gmail.com>
CC: Andy Gospodarek <andy@greyhouse.net>
CC: "David S. Miller" <davem@davemloft.net>
CC: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
CC: Jakub Kicinski <kuba@kernel.org>
CC: Steffen Klassert <steffen.klassert@secunet.com>
CC: Herbert Xu <herbert@gondor.apana.org.au>
CC: netdev@vger.kernel.org
Suggested-by: Saeed Mahameed <saeedm@mellanox.com>
Signed-off-by: Jarod Wilson <jarod@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This is prep work for initial support of bonding hardware encryption
pass-through support. The bonding driver will fill in the slave_dev
pointer, and we use that to know not to skb_push() again on a given
skb that was already processed on the bond device.
CC: Jay Vosburgh <j.vosburgh@gmail.com>
CC: Veaceslav Falico <vfalico@gmail.com>
CC: Andy Gospodarek <andy@greyhouse.net>
CC: "David S. Miller" <davem@davemloft.net>
CC: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
CC: Jakub Kicinski <kuba@kernel.org>
CC: Steffen Klassert <steffen.klassert@secunet.com>
CC: Herbert Xu <herbert@gondor.apana.org.au>
CC: netdev@vger.kernel.org
CC: intel-wired-lan@lists.osuosl.org
Signed-off-by: Jarod Wilson <jarod@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
During IPsec performance testing, we see bad ICMP checksum. The error packet
has duplicated ESP trailer due to double validate_xmit_xfrm calls. The first call
is from ip_output, but the packet cannot be sent because
netif_xmit_frozen_or_stopped is true and the packet gets dev_requeue_skb. The second
call is from NET_TX softirq. However after the first call, the packet already
has the ESP trailer.
Fix by marking the skb with XFRM_XMIT bit after the packet is handled by
validate_xmit_xfrm to avoid duplicate ESP trailer insertion.
Fixes: f6e27114a6 ("net: Add a xfrm validate function to validate_xmit_skb")
Signed-off-by: Huy Nguyen <huyn@mellanox.com>
Reviewed-by: Boris Pismenny <borisp@mellanox.com>
Reviewed-by: Raed Salem <raeds@mellanox.com>
Reviewed-by: Saeed Mahameed <saeedm@mellanox.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
For transport mode, when ipv6 nexthdr is set, the packet format might
be like:
----------------------------------------------------
| | dest | | | | ESP | ESP |
| IP6 hdr| opts.| ESP | TCP | Data | Trailer | ICV |
----------------------------------------------------
and in __xfrm_transport_prep():
pskb_pull(skb, skb->mac_len + sizeof(ip6hdr) + x->props.header_len);
it will pull the data pointer to the wrong position, as it missed the
nexthdrs/dest opts.
This patch is to fix it by using:
pskb_pull(skb, skb_transport_offset(skb) + x->props.header_len);
as we can be sure transport_header points to ESP header at that moment.
It also fixes a panic when packets with ipv6 nexthdr are sent over
esp6 transport mode:
[ 100.473845] kernel BUG at net/core/skbuff.c:4325!
[ 100.478517] RIP: 0010:__skb_to_sgvec+0x252/0x260
[ 100.494355] Call Trace:
[ 100.494829] skb_to_sgvec+0x11/0x40
[ 100.495492] esp6_output_tail+0x12e/0x550 [esp6]
[ 100.496358] esp6_xmit+0x1d5/0x260 [esp6_offload]
[ 100.498029] validate_xmit_xfrm+0x22f/0x2e0
[ 100.499604] __dev_queue_xmit+0x589/0x910
[ 100.502928] ip6_finish_output2+0x2a5/0x5a0
[ 100.503718] ip6_output+0x6c/0x120
[ 100.505198] xfrm_output_resume+0x4bf/0x530
[ 100.508683] xfrm6_output+0x3a/0xc0
[ 100.513446] inet6_csk_xmit+0xa1/0xf0
[ 100.517335] tcp_sendmsg+0x27/0x40
[ 100.517977] sock_sendmsg+0x3e/0x60
[ 100.518648] __sys_sendto+0xee/0x160
Fixes: c35fe4106b ("xfrm: Add mode handlers for IPsec on layer 2")
Signed-off-by: Xin Long <lucien.xin@gmail.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Steffen Klassert says:
====================
pull request (net-next): ipsec-next 2020-03-28
1) Use kmem_cache_zalloc() instead of kmem_cache_alloc()
in xfrm_state_alloc(). From Huang Zijiang.
2) esp_output_fill_trailer() is the same in IPv4 and IPv6,
so share this function to avoide code duplcation.
From Raed Salem.
3) Add offload support for esp beet mode.
From Xin Long.
====================
Signed-off-by: David S. Miller <davem@davemloft.net>
Like __xfrm_transport/mode_tunnel_prep(), this patch is to add
__xfrm_mode_beet_prep() to fix the transport_header for gso
segments, and reset skb mac_len, and pull skb data to the
proto inside esp.
This patch also fixes a panic, reported by ltp:
# modprobe esp4_offload
# runltp -f net_stress.ipsec_tcp
[ 2452.780511] kernel BUG at net/core/skbuff.c:109!
[ 2452.799851] Call Trace:
[ 2452.800298] <IRQ>
[ 2452.800705] skb_push.cold.98+0x14/0x20
[ 2452.801396] esp_xmit+0x17b/0x270 [esp4_offload]
[ 2452.802799] validate_xmit_xfrm+0x22f/0x2e0
[ 2452.804285] __dev_queue_xmit+0x589/0x910
[ 2452.806264] __neigh_update+0x3d7/0xa50
[ 2452.806958] arp_process+0x259/0x810
[ 2452.807589] arp_rcv+0x18a/0x1c
It was caused by the skb going to esp_xmit with a wrong transport
header.
Signed-off-by: Xin Long <lucien.xin@gmail.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Xiumei found a panic in esp offload:
BUG: unable to handle kernel NULL pointer dereference at 0000000000000020
RIP: 0010:esp_output_done+0x101/0x160 [esp4]
Call Trace:
? esp_output+0x180/0x180 [esp4]
cryptd_aead_crypt+0x4c/0x90
cryptd_queue_worker+0x6e/0xa0
process_one_work+0x1a7/0x3b0
worker_thread+0x30/0x390
? create_worker+0x1a0/0x1a0
kthread+0x112/0x130
? kthread_flush_work_fn+0x10/0x10
ret_from_fork+0x35/0x40
It was caused by that skb secpath is used in esp_output_done() after it's
been released elsewhere.
The tx path for esp offload is:
__dev_queue_xmit()->
validate_xmit_skb_list()->
validate_xmit_xfrm()->
esp_xmit()->
esp_output_tail()->
aead_request_set_callback(esp_output_done) <--[1]
crypto_aead_encrypt() <--[2]
In [1], .callback is set, and in [2] it will trigger the worker schedule,
later on a kernel thread will call .callback(esp_output_done), as the call
trace shows.
But in validate_xmit_xfrm():
skb_list_walk_safe(skb, skb2, nskb) {
...
err = x->type_offload->xmit(x, skb2, esp_features); [esp_xmit]
...
}
When the err is -EINPROGRESS, which means this skb2 will be enqueued and
later gets encrypted and sent out by .callback later in a kernel thread,
skb2 should be removed fromt skb chain. Otherwise, it will get processed
again outside validate_xmit_xfrm(), which could release skb secpath, and
cause the panic above.
This patch is to remove the skb from the chain when it's enqueued in
cryptd_wq. While at it, remove the unnecessary 'if (!skb)' check.
Fixes: 3dca3f38cf ("xfrm: Separate ESP handling from segmentation for GRO packets.")
Reported-by: Xiumei Mu <xmu@redhat.com>
Signed-off-by: Xin Long <lucien.xin@gmail.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
This patch to handle the asynchronous unregister
device event so the device IPsec offload resources
could be cleanly released.
Fixes: e4db5b61c5 ("xfrm: policy: remove pcpu policy cache")
Signed-off-by: Raed Salem <raeds@mellanox.com>
Reviewed-by: Boris Pismenny <borisp@mellanox.com>
Reviewed-by: Saeed Mahameed <saeedm@mellanox.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
This is converts xfrm segment iteration to use the new function, keeping
the flow of the existing code as intact as possible. One case is very
straight-forward, whereas the other case has some more subtle code that
likes to peak at ->next and relink skbs. By keeping the variables the
same as before, we can upgrade this code with minimal surgery required.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Steffen Klassert says:
====================
pull request (net-next): ipsec-next 2019-07-05
1) A lot of work to remove indirections from the xfrm code.
From Florian Westphal.
2) Fix a WARN_ON with ipv6 that triggered because of a
forgotten break statement. From Florian Westphal.
3) Remove xfrmi_init_net, it is not needed.
From Li RongQing.
Please pull or let me know if there are problems.
====================
Signed-off-by: David S. Miller <davem@davemloft.net>
esp4_get_mtu and esp6_get_mtu are exactly the same, the only difference
is a single sizeof() (ipv4 vs. ipv6 header).
Merge both into xfrm_state_mtu() and remove the indirection.
Signed-off-by: Florian Westphal <fw@strlen.de>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Based on 1 normalized pattern(s):
this program is free software you can redistribute it and or modify
it under the terms of the gnu general public license as published by
the free software foundation either version 2 of the license or at
your option any later version
extracted by the scancode license scanner the SPDX license identifier
GPL-2.0-or-later
has been chosen to replace the boilerplate/reference in 3029 file(s).
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Allison Randal <allison@lohutok.net>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190527070032.746973796@linutronix.de
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
This structure is now only 4 bytes, so its more efficient
to cache a copy rather than its address.
No significant size difference in allmodconfig vmlinux.
With non-modular kernel that has all XFRM options enabled, this
series reduces vmlinux image size by ~11kb. All xfrm_mode
indirections are gone and all modes are built-in.
before (ipsec-next master):
text data bss dec filename
21071494 7233140 11104324 39408958 vmlinux.master
after this series:
21066448 7226772 11104324 39397544 vmlinux.patched
With allmodconfig kernel, the size increase is only 362 bytes,
even all the xfrm config options removed in this series are
modular.
before:
text data bss dec filename
15731286 6936912 4046908 26715106 vmlinux.master
after this series:
15731492 6937068 4046908 26715468 vmlinux
Signed-off-by: Florian Westphal <fw@strlen.de>
Reviewed-by: Sabrina Dubroca <sd@queasysnail.net>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
There are only two versions (tunnel and transport). The ip/ipv6 versions
are only differ in sizeof(iphdr) vs ipv6hdr.
Place this in the core and use x->outer_mode->encap type to call the
correct adjustment helper.
Before:
text data bss dec filename
15730311 6937008 4046908 26714227 vmlinux
After:
15730428 6937008 4046908 26714344 vmlinux
(about 117 byte increase)
v2: use family from x->outer_mode, not inner
Signed-off-by: Florian Westphal <fw@strlen.de>
Reviewed-by: Sabrina Dubroca <sd@queasysnail.net>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
This patch introduces support for gso partial ESP offload.
Signed-off-by: Boris Pismenny <borisp@mellanox.com>
Signed-off-by: Raed Salem <raeds@mellanox.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
With the following patches, we are going to use __netdev_pick_tx() in
many modules. Rename it to netdev_pick_tx(), to make it clear is
a public API.
Also rename the existing netdev_pick_tx() to netdev_core_pick_tx(),
to avoid name clashes.
Suggested-by: Eric Dumazet <edumazet@google.com>
Suggested-by: David Miller <davem@davemloft.net>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
skb_sec_path gains 'const' qualifier to avoid
xt_policy.c: 'skb_sec_path' discards 'const' qualifier from pointer target type
same reasoning as previous conversions: Won't need to touch these
spots anymore when skb->sp is removed.
Signed-off-by: Florian Westphal <fw@strlen.de>
Signed-off-by: David S. Miller <davem@davemloft.net>
Steffen Klassert says:
====================
pull request (net-next): ipsec-next 2018-10-01
1) Make xfrmi_get_link_net() static to silence a sparse warning.
From Wei Yongjun.
2) Remove a unused esph pointer definition in esp_input().
From Haishuang Yan.
3) Allow the NIC driver to quietly refuse xfrm offload
in case it does not support it, the SA is created
without offload in this case.
From Shannon Nelson.
Please pull or let me know if there are problems.
====================
Signed-off-by: David S. Miller <davem@davemloft.net>
An SKB is not on a list if skb->next is NULL.
Codify this convention into a helper function and use it
where we are dequeueing an SKB and need to mark it as such.
Signed-off-by: David S. Miller <davem@davemloft.net>
If the "offload" attribute is used to create an IPsec SA
and the .xdo_dev_state_add() fails, the SA creation fails.
However, if the "offload" attribute is used on a device that
doesn't offer it, the attribute is quietly ignored and the SA
is created without an offload.
Along the same line of that second case, it would be good to
have a way for the device to refuse to offload an SA without
failing the whole SA creation. This patch adds that feature
by allowing the driver to return -EOPNOTSUPP as a signal that
the SA may be fine, it just can't be offloaded.
This allows the user a little more flexibility in requesting
offloads and not needing to know every detail at all times about
each specific NIC when trying to create SAs.
Signed-off-by: Shannon Nelson <shannon.nelson@oracle.com>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
Michal Schmidt
Mohammad Kabat
Ivan Vecera
Sabrina Dubroca
Jan Stancek
Íñigo Huguet
Herton R. Krzesinski
Felix Maurer
Ayush Sawal
Steffen Klassert
David S. Miller
Jarod Wilson
Huy Nguyen
Xin Long
Raed Salem
Jason A. Donenfeld
Florian Westphal
Thomas Gleixner
Boris Pismenny
Paolo Abeni
Shannon Nelson