JIRA: https://issues.redhat.com/browse/RHEL-57756
Upstream commit(s):
commit 3e51f2cbbc5dc854f89ca37d95d295bfcabb5b43
Author: Jakub Kicinski <kuba@kernel.org>
Date: Thu May 2 09:40:43 2024 -0700
tools: ynl: add --list-ops and --list-msgs to CLI
I often forget the exact naming of ops and have to look at
the spec to find it. Add support for listing the operations:
$ ./cli.py --spec .../netdev.yaml --list-ops
dev-get [ do, dump ]
page-pool-get [ do, dump ]
page-pool-stats-get [ do, dump ]
queue-get [ do, dump ]
napi-get [ do, dump ]
qstats-get [ dump ]
For completeness also support listing all ops (including
notifications:
# ./cli.py --spec .../netdev.yaml --list-msgs
dev-get [ dump, do ]
dev-add-ntf [ notify ]
dev-del-ntf [ notify ]
dev-change-ntf [ notify ]
page-pool-get [ dump, do ]
page-pool-add-ntf [ notify ]
page-pool-del-ntf [ notify ]
page-pool-change-ntf [ notify ]
page-pool-stats-get [ dump, do ]
queue-get [ dump, do ]
napi-get [ dump, do ]
qstats-get [ dump ]
Use double space after the name for slightly easier to read
output.
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20240502164043.2130184-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Petr Oros <poros@redhat.com>
JIRA: https://issues.redhat.com/browse/RHEL-57756
Upstream commit(s):
commit ba8be00f68f5c70eb1df2193251a579923bd9501
Author: Donald Hunter <donald.hunter@gmail.com>
Date: Thu Apr 18 11:47:36 2024 +0100
tools/net/ynl: Add multi message support to ynl
Add a "--multi <do-op> <json>" command line to ynl that makes it
possible to add several operations to a single netlink request payload.
The --multi command line option is repeated for each operation.
This is used by the nftables family for transaction batches. For
example:
./tools/net/ynl/cli.py \
--spec Documentation/netlink/specs/nftables.yaml \
--multi batch-begin '{"res-id": 10}' \
--multi newtable '{"name": "test", "nfgen-family": 1}' \
--multi newchain '{"name": "chain", "table": "test", "nfgen-family": 1}' \
--multi batch-end '{"res-id": 10}'
[None, None, None, None]
It can also be used for bundling get requests:
./tools/net/ynl/cli.py \
--spec Documentation/netlink/specs/nftables.yaml \
--multi gettable '{"name": "test", "nfgen-family": 1}' \
--multi getchain '{"name": "chain", "table": "test", "nfgen-family": 1}' \
--output-json
[{"name": "test", "use": 1, "handle": 1, "flags": [],
"nfgen-family": 1, "version": 0, "res-id": 2},
{"table": "test", "name": "chain", "handle": 1, "use": 0,
"nfgen-family": 1, "version": 0, "res-id": 2}]
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20240418104737.77914-4-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Petr Oros <poros@redhat.com>
JIRA: https://issues.redhat.com/browse/RHEL-57755
Upstream commit(s):
commit c0111878d45e3bb8779886fbf956b574bac8a3aa
Author: Jakub Kicinski <kuba@kernel.org>
Date: Mon Mar 4 21:33:10 2024 -0800
tools: ynl: add --dbg-small-recv for easier kernel testing
Most "production" netlink clients use large buffers to
make dump efficient, which means that handling of dump
continuation in the kernel is not very well tested.
Add an option for debugging / testing handling of dumps.
It enables printing of extra netlink-level debug and
lowers the recv() buffer size in one go. When used
without any argument (--dbg-small-recv) it picks
a very small default (4000), explicit size can be set,
too (--dbg-small-recv 5000).
Example:
$ ./cli.py [...] --dbg-small-recv
Recv: read 3712 bytes, 29 messages
nl_len = 128 (112) nl_flags = 0x0 nl_type = 19
[...]
nl_len = 128 (112) nl_flags = 0x0 nl_type = 19
Recv: read 3968 bytes, 31 messages
nl_len = 128 (112) nl_flags = 0x0 nl_type = 19
[...]
nl_len = 128 (112) nl_flags = 0x0 nl_type = 19
Recv: read 532 bytes, 5 messages
nl_len = 128 (112) nl_flags = 0x0 nl_type = 19
[...]
nl_len = 128 (112) nl_flags = 0x0 nl_type = 19
nl_len = 20 (4) nl_flags = 0x2 nl_type = 3
(the [...] are edits to shorten the commit message).
Note that the first message of the dump is sized conservatively
by the kernel.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Petr Oros <poros@redhat.com>
JIRA: https://issues.redhat.com/browse/RHEL-57764
commit 771b7012e5f3a49739dab4be60b87517a249a1df
Author: Donald Hunter <donald.hunter@gmail.com>
Date: Wed Mar 6 23:10:42 2024 +0000
tools/net/ynl: Report netlink errors without stacktrace
ynl does not handle NlError exceptions so they get reported like program
failures. Handle the NlError exceptions and report the netlink errors
more cleanly.
Example now:
Netlink error: No such file or directory
nl_len = 44 (28) nl_flags = 0x300 nl_type = 2
error: -2 extack: {'bad-attr': '.op'}
Example before:
Traceback (most recent call last):
File "/home/donaldh/net-next/./tools/net/ynl/cli.py", line 81, in <module>
main()
File "/home/donaldh/net-next/./tools/net/ynl/cli.py", line 69, in main
reply = ynl.dump(args.dump, attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/donaldh/net-next/tools/net/ynl/lib/ynl.py", line 906, in dump
return self._op(method, vals, [], dump=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/donaldh/net-next/tools/net/ynl/lib/ynl.py", line 872, in _op
raise NlError(nl_msg)
lib.ynl.NlError: Netlink error: No such file or directory
nl_len = 44 (28) nl_flags = 0x300 nl_type = 2
error: -2 extack: {'bad-attr': '.op'}
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Link: https://lore.kernel.org/r/20240306231046.97158-3-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Ivan Vecera <ivecera@redhat.com>
JIRA: https://issues.redhat.com/browse/RHEL-57764
commit e2ece0bc5ab1f7e0bb00f3b81fd4132b774d880d
Author: Donald Hunter <donald.hunter@gmail.com>
Date: Mon Jan 29 22:34:46 2024 +0000
tools/net/ynl: Add --output-json arg to ynl cli
The ynl cli currently emits python pretty printed structures which is
hard to consume. Add a new --output-json argument to emit JSON.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Breno Leitao <leitao@debian.org>
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20240129223458.52046-2-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Ivan Vecera <ivecera@redhat.com>
JIRA: https://issues.redhat.com/browse/RHEL-30145
Upstream commit(s):
commit d96e48a3d55db7ee62e607ad2d89eee1a8585028
Author: Jiri Pirko <jiri@nvidia.com>
Date: Fri Oct 27 11:25:25 2023 +0200
tools: ynl: introduce option to process unknown attributes or types
In case the kernel sends message back containing attribute not defined
in family spec, following exception is raised to the user:
$ sudo ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/devlink.yaml --do trap-get --json '{"bus-name": "netdevsim", "dev-name": "netdevsim1", "trap-name": "source_mac_is_multicast"}'
Traceback (most recent call last):
File "/home/jiri/work/linux/tools/net/ynl/lib/ynl.py", line 521, in _decode
attr_spec = attr_space.attrs_by_val[attr.type]
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
KeyError: 132
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/jiri/work/linux/./tools/net/ynl/cli.py", line 61, in <module>
main()
File "/home/jiri/work/linux/./tools/net/ynl/cli.py", line 49, in main
reply = ynl.do(args.do, attrs, args.flags)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jiri/work/linux/tools/net/ynl/lib/ynl.py", line 731, in do
return self._op(method, vals, flags)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jiri/work/linux/tools/net/ynl/lib/ynl.py", line 719, in _op
rsp_msg = self._decode(decoded.raw_attrs, op.attr_set.name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jiri/work/linux/tools/net/ynl/lib/ynl.py", line 525, in _decode
raise Exception(f"Space '{space}' has no attribute with value '{attr.type}'")
Exception: Space 'devlink' has no attribute with value '132'
Introduce a command line option "process-unknown" and pass it down to
YnlFamily class constructor to allow user to process unknown
attributes and types and print them as binaries.
$ sudo ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/devlink.yaml --do trap-get --json '{"bus-name": "netdevsim", "dev-name": "netdevsim1", "trap-name": "source_mac_is_multicast"}' --process-unknown
{'UnknownAttr(129)': {'UnknownAttr(0)': b'\x00\x00\x00\x00\x00\x00\x00\x00',
'UnknownAttr(1)': b'\x00\x00\x00\x00\x00\x00\x00\x00',
'UnknownAttr(2)': b'\x0e\x00\x00\x00\x00\x00\x00\x00'},
'UnknownAttr(132)': b'\x00',
'UnknownAttr(133)': b'',
'UnknownAttr(134)': {'UnknownAttr(0)': b''},
'bus-name': 'netdevsim',
'dev-name': 'netdevsim1',
'trap-action': 'drop',
'trap-group-name': 'l2_drops',
'trap-name': 'source_mac_is_multicast'}
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://lore.kernel.org/r/20231027092525.956172-1-jiri@resnulli.us
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Petr Oros <poros@redhat.com>
JIRA: https://issues.redhat.com/browse/RHEL-30656
commit 1768d8a767f8e7049407fe467364e7ea56f0bdce
Author: Donald Hunter <donald.hunter@gmail.com>
Date: Fri Aug 25 13:27:52 2023 +0100
tools/net/ynl: Add support for create flags
Add support for using NLM_F_REPLACE, _EXCL, _CREATE and _APPEND flags
in requests.
Signed-off-by: Donald Hunter <donald.hunter@gmail.com>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Link: https://lore.kernel.org/r/20230825122756.7603-10-donald.hunter@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Ivan Vecera <ivecera@redhat.com>
JIRA: https://issues.redhat.com/browse/RHEL-30344
commit 37d9df224d1eec1b434fe9ffa40104c756478c29
Author: Jakub Kicinski <kuba@kernel.org>
Date: Mon Mar 6 12:04:57 2023 -0800
ynl: re-license uniformly under GPL-2.0 OR BSD-3-Clause
I was intending to make all the Netlink Spec code BSD-3-Clause
to ease the adoption but it appears that:
- I fumbled the uAPI and used "GPL WITH uAPI note" there
- it gives people pause as they expect GPL in the kernel
As suggested by Chuck re-license under dual. This gives us benefit
of full BSD freedom while fulfilling the broad "kernel is under GPL"
expectations.
Link: https://lore.kernel.org/all/20230304120108.05dd44c5@kernel.org/
Link: https://lore.kernel.org/r/20230306200457.3903854-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Ivan Vecera <ivecera@redhat.com>
JIRA: https://issues.redhat.com/browse/RHEL-30344
commit 981cbcb030d919ee49ebbfc2889839c6882d9ea7
Author: Jakub Kicinski <kuba@kernel.org>
Date: Mon Jan 30 18:33:54 2023 -0800
tools: net: use python3 explicitly
The scripts require Python 3 and some distros are dropping
Python 2 support.
Reported-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Ivan Vecera <ivecera@redhat.com>
JIRA: https://issues.redhat.com/browse/RHEL-30344
commit 5c6674f6eb52f7968b805b25c7478b3d96b6b4f7
Author: Jakub Kicinski <kuba@kernel.org>
Date: Mon Jan 30 18:33:50 2023 -0800
tools: ynl: load jsonschema on demand
The CLI script tries to validate jsonschema by default.
It's seems better to validate too many times than too few.
However, when copying the scripts to random servers having
to install jsonschema is tedious. Load jsonschema via
importlib, and let the user opt out.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Ivan Vecera <ivecera@redhat.com>
JIRA: https://issues.redhat.com/browse/RHEL-30344
commit 8dfec0a8886880868802094967c6a769b6d15737
Author: Jakub Kicinski <kuba@kernel.org>
Date: Mon Jan 30 18:33:49 2023 -0800
tools: ynl: use operation names from spec on the CLI
When I wrote the first version of the Python code I was quite
excited that we can generate class methods directly from the
spec. Unfortunately we need to use valid identifiers for method
names (specifically no dashes are allowed). Don't reuse those
names on the CLI, it's much more natural to use the operation
names exactly as listed in the spec.
Instead of:
./cli --do rings_get
use:
./cli --do rings-get
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Ivan Vecera <ivecera@redhat.com>
JIRA: https://issues.redhat.com/browse/RHEL-30344
commit 4e4480e89c47b52b3f4fbc1ddf07a7ce541f0839
Author: Jakub Kicinski <kuba@kernel.org>
Date: Mon Jan 30 18:33:42 2023 -0800
tools: ynl: move the cli and netlink code around
Move the CLI code out of samples/ and the library part
of it into tools/net/ynl/lib/. This way we can start
sharing some code with the code gen.
Initially I thought that code gen is too C-specific to
share anything but basic stuff like calculating values
for enums can easily be shared.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Ivan Vecera <ivecera@redhat.com>
Petr Oros
Ivan Vecera