'optional' fields is the legacy mechanism from 'proto2' times. In
'proto3' schemes this keyword changes the 'presence' handling of
the fields that are marked as 'optional'. By default protobuf
messages(serializer to be precise) treat all fields as optional.
This means that if field has the default value(0, 0.0f, false, nullptr)
the field will not be serialized at all. This mechanism can be
overwritten using the 'optional' attribute or keyword. If the message
field is marked as 'optional' it gets the following methods in addition
to the regular field one:
- clear<fieldName>
- has<fieldName>
These methods allow to clear the value and to check whether the field
was ever set. The optional fields get serialized if they were
explicitly set in the message, even if they have the default value.
This is the key difference that this attribute/keyword makes on
the serialization process.
[ChangeLog][QtProtobuf] All message fieds now have 'has' methods
that indicate if message was initialized and contain value or not.
Task-number: QTBUG-103978
Change-Id: Ia948be85486175bb2d81357352f5c04da12af846
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Konrad Kujawa <konrad.kujawa@qt.io>
Make the functions non-inline and move to .cpp file.
Pick-to: 6.5 6.6
Task-number: QTBUG-116415
Change-Id: I26efb11c23cbf3ee4210cd10b7d3a4cb67219104
Reviewed-by: Tatiana Borisova <tatiana.borisova@qt.io>
Reviewed-by: Konrad Kujawa <konrad.kujawa@qt.io>
The vast majority of users (the tests) will transparently use the new
new overload, because they pass temporaries.
Change-Id: I4630e44ed19f32ac367c48097afe0cda5222bfcf
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
These warnings slipped in during a time period where documentation
testing in the CI was disabled.
Fix incorrect argument type for QAbstractGrpcClient::call() and
remove trailing semi-colons.
Remove documentation for non-existent overload of
QAbstractGrpcClient::startStream().
Add a documentation dependency to qtwidgets to fix broken link to
QLineEdit.
Pick-to: 6.5
Change-Id: Ie429d94535860b3c85aaf95010a49d1fd90e8307
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
This fixes the situation when we received the non-packed repeated field
from the wire, but the used protobuf schema specifies the field as
packed. This situation is possible if the protobuf schema mismatches on
on the wire ends. The explicit indication of this situation is the
wiretype mismatch. So if the wiretype differs of the expected one that
means we need to look for the handler with the 'packed' option opposite
to the one is expected by the actual protobuf schema.
Pick-to: 6.5
Change-Id: I57bf613410c4031f638533f092fd8d36c592db32
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
The actual protobuf serializer has multiple overheads and code
duplicaiton. Serialization functions use individual ways to
detect if field was initialized or not. This patch adds the
general functions that detect if field was initialized or not.
Also serializers previously juggled the protobuf field number to
conclude the need of field header. Now the header is added under
clear conditions.
For STRING and BYTES lists we should use the same way of
serializing/deserialing as for non-packed lists, since they are
non-packed lists.
Map elements should be serialized even if value is not initialized.
But non-initialized keys should still be present in serialization
stream. Avoid using serializeProperty when serializing the map key
and use integated type serilizers directly.
Pick-to: 6.5
Change-Id: I32b512acdaa9f355eea9c41f735bc9722bed0bba
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
This addresses the recent warnings introduced by a change to QDoc:
qtprotobuftypes.h:234: (qdoc) warning: No documentation generated for
function 'qRegisterProtobufTypes' in global scope.
qprotobufserializer.h:319: (qdoc) warning: No documentation generated
for function 'qRegisterProtobufType' in global scope.
qprotobufserializer.h:333: (qdoc) warning: No documentation generated
for function 'qRegisterProtobufMapType' in global scope.
qprotobufserializer.h:361: (qdoc) warning: No documentation generated
for function 'qRegisterProtobufEnumType' in global scope.
Pick-to: 6.5
Change-Id: I694b4a524a31c477bb3e4b52ea89b0c4823c0430
Reviewed-by: Konrad Kujawa <konrad.kujawa@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
QByteArray and QString have the LengthDelimited wiretype, but they
should not be handled as the repeated fields. Exclude them in
isRepeatedField check.
Amends 2e3fdd5c05
Change-Id: If683afe3d16fd7ce039e6c7443a76386e7a0fced
Reviewed-by: Konrad Kujawa <konrad.kujawa@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
If we were unable to read the unknown length-delimited field size we
need to invalidate the further deserializing procedure, since the buffer
is malformed.
Pick-to: 6.5 6.5.0
Change-Id: I6bbc3ed99b8390435aefbee15eefa3dc6a3a2ce9
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Basic type deserializers return the deserializing result. If result
is 'false' that means that there were not enough bytes in the message
data buffer. Need to exit the deserialization process and indicate
the deserialization error for protobuf serializer.
Pick-to: 6.5 6.5.0
Change-Id: I5acebce8afee5260815584db43bb5b13de80312e
Reviewed-by: Konrad Kujawa <konrad.kujawa@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
This patch adds support for 'oneof', a union-like type in protobuf,
that need not contain a value, with the following features:
- Setting a oneof field will automatically clear all other
members of the oneof.
- If the parser encounters multiple members of the same oneof
on the wire, only the last member seen is used in the
parsed message.
- A oneof cannot be repeated.
- A oneof cannot contain repeated or map fields.
- If you set a oneof field to the default value (even the default,
such as 0 for an int32 oneof field), the "case" of that oneof
field will be set, and the value will be serialized on the
wire.
Unlike the reference C++ implementation, the Qt implementation of
'oneof' fields adds the 'has<PropertyName>' functions for every
'oneof' member. The protobuf generator generates a Q_PROPERTY for each
member of the 'oneof'; these properties are collectively implemented
as a single class member.
'oneof' fields are represented by the
QtProtobufPrivate::QProtobufOptional class. The class holds the 'oneof'
field number and the value that it contains as QVariant.
A QExplicitSharedDataPointer is used as reference counter, and to copy
and to free the memory of protobuf messages that are stored inside the
nested QVariant as pointers.
The class could also be used to hold 'optional' fields in follow up
commits, but has small overhead since it holds field number, that
is not used by optional fields.
The generated classes also allows reading field number that is
currently stored in the 'oneof' field.
The QtProtobuf::InvalidFieldNumber constant indicates that oneof fields
is uninitialized.
The serialization and deserialization tests use the expected values
from the results of the reference C++ serialization(protobuf version
3.19).
Task-number: QTBUG-103981
Change-Id: Ie4053cb164bba6bc5f14f8cedb34bad62a638c43
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Avoid serializing values that contain empty or default values. This
reflects the reference protobuf behavior where all fields are
treated as 'optional' by default.
Also this adjusts the basic length-delimited types handling(QString,
QByteArray) to avoid code duplication.
Task-number: QTBUG-103978
Fixes: QTBUG-109291
Change-Id: I91b164a9d8337290087bfa49d8c2570d9de1b1ce
Reviewed-by: Tatiana Borisova <tatiana.borisova@qt.io>
Reviewed-by: Konrad Kujawa <konrad.kujawa@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Use the unknownEntries variable we so cleverly added in
QProtobufMessagePrivate to store the data for the unknown fields.
By using a bytes-to-count mapping we can easily store multiple repeated
entries of a message, which could happen in a repeated list with
multiple identical entries. Since we don't know if a field is actually
marked as _repeated_ or not (since it's an unknown field) we just have
to assume it is.
Fixes: QTBUG-101966
Change-Id: I1ebde5fdaebe2987a68de1632f136bb6606e718a
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Update version to 6.5, since protobuf was not released at 6.4 times.
Change-Id: I1714ca467fb897e9f079cc06cdd94d26b66fe6cb
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
And inherit from it instead of QObject in messages.
Messages are inherently value-types. They don't have any logic
associated with them, they simply store some value(s).
Value-types should be copyable and movable but QObject _is not_ because
this would break connections and parenting.
By making the objects Q_GADGETs we can still retain Q_PROPERTY, and by
having a pointer to the object's staticMetaObject we can implement
property/setProperty functionality in QProtobufMessage with minimal
effort.
As a side-effect of this we no longer emit, or even have, signals
when values on a message changes. But, as mentioned previously, messages
don't have any logic associated to them so they would never update
without users either calling an associated setter function or by
deserializing a new message into an already-existing object.
As another side-effect: propertyCount has decreased because we no longer
have QObject's 'name' property.
Fixes: QTBUG-100986
Change-Id: I0366a180bf5aadd4afb075527b80cc0a3e923ab6
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Protobuf allows non-packed way of serialization of the repeated
fields. This is controlled on the protobuf schema level by adding the
'packed' attribute to a message field descriptor. If the attribute is
not set the default value is 'true', so all repeated messages should
use the 'packed' approach when serializing.
Fixes: QTBUG-103979
Change-Id: I078a18734785865a4b7b34190a642b82b7829755
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
The list of integrated type serializers is predefined and can be
initialized statically. Using std::array allows us to avoid extra
allocations and gives us clean and simple static initialization.
Change-Id: I04290d3b343487d6967341502c893f76a5044139
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
The code handles serialization and deserialization by itself with no
extra dependencies. It also contains a system for registering converters
specific to Protobuf serialization as well as having the ability to
extend it to handle JSON encoding[0] in the future, using QJson.
This patch is a companion to the patch introducing the qtprotobufgen
tool.
The code was originally written by Alexey Edelev, along with various
contributors, on GitHub. Originally under MIT license, but major
contributors have agreed to relicense the code under LGPL/GPL/commercial
for inclusion in Qt. Their copyright notice is retained and the
original source code, still licensed as MIT, can be found in its
original repository[1].
[0]: https://developers.google.com/protocol-buffers/docs/proto3#json
[1]: https://github.com/semlanik/qtprotobuf
Done-with: Alexey Edelev <alexey.edelev@qt.io>
Change-Id: I8b2e7074a19f39bd52ab263e215d2e99ffaf1162
Reviewed-by: Alex Blasche <alexander.blasche@qt.io>
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Tatiana Borisova <tatiana.borisova@qt.io>
Alexey Edelev
Marc Mutz
Topi Reinio
Tatiana Borisova
Mårten Nordheim