Skip to content

HDDS-14748. Implement ScmGeneratedMessageCodec without reflection#9878

Open
Russole wants to merge 9 commits intoapache:masterfrom
Russole:HDDS-14748
Open

HDDS-14748. Implement ScmGeneratedMessageCodec without reflection#9878
Russole wants to merge 9 commits intoapache:masterfrom
Russole:HDDS-14748

Conversation

@Russole
Copy link
Contributor

@Russole Russole commented Mar 7, 2026

What changes were proposed in this pull request?

  • Identify non-shaded protobuf message classes used in @replicate method parameters and register them with ScmNonShadedGeneratedMessageCodec<>(T.parser()), including element types used in collection parameters such as List<T>.
  • Based on this investigation, no classes implementing org.apache.ratis.thirdparty.com.google.protobuf.Message were found.
  • Therefore the generic codec registration codecs.put(Message.class, new ScmGeneratedMessageCodec()) is removed.
  • Change ScmNonShadedGeneratedMessageCodec to support generic type
  • Add a ScmNonShadedGeneratedMessageCodec for each proto class.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-14748

How was this patch tested?

@Russole
Copy link
Contributor Author

Russole commented Mar 7, 2026

Hi @szetszwo, when you have time, could you please help review this PR? Thanks!

Copy link
Contributor

@szetszwo szetszwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Russole , thanks a lot for spending time to figure out the required classes! The change look really simple. I just have a minor comment inlined.

@Russole
Copy link
Contributor Author

Russole commented Mar 7, 2026

Thanks @szetszwo for the review!
I’m happy to update the change based on any comments.

throw new InvalidProtocolBufferException("Message cannot be decoded: " + ex.getMessage());
return parser.parseFrom(value.toByteArray());
} catch (com.google.protobuf.InvalidProtocolBufferException e) {
throw new IllegalArgumentException("Message cannot be decoded", e);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should throw org.apache.ratis.thirdparty.com.google.protobuf.InvalidProtocolBufferException

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry that this comment somehow was not included preiously.

@Russole
Copy link
Contributor Author

Russole commented Mar 7, 2026

Thanks @szetszwo for pointing out the exception type. I’ve updated the change accordingly.

@Russole Russole requested a review from szetszwo March 7, 2026 05:52
Comment on lines +50 to +61
codecs.put(ContainerID.class,
new ScmNonShadedGeneratedMessageCodec<>(ContainerID.parser()));
codecs.put(PipelineID.class,
new ScmNonShadedGeneratedMessageCodec<>(PipelineID.parser()));
codecs.put(Pipeline.class,
new ScmNonShadedGeneratedMessageCodec<>(Pipeline.parser()));
codecs.put(ContainerInfoProto.class,
new ScmNonShadedGeneratedMessageCodec<>(ContainerInfoProto.parser()));
codecs.put(DeletedBlocksTransaction.class,
new ScmNonShadedGeneratedMessageCodec<>(DeletedBlocksTransaction.parser()));
codecs.put(DeletedBlocksTransactionSummary.class,
new ScmNonShadedGeneratedMessageCodec<>(DeletedBlocksTransactionSummary.parser()));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Russole , I forgot to say we may use the getDefaultInstance() trick;

    putProto(ContainerID.getDefaultInstance());
    putProto(PipelineID.getDefaultInstance());
    putProto(Pipeline.getDefaultInstance());
    putProto(ContainerInfoProto.getDefaultInstance());
    putProto(DeletedBlocksTransaction.getDefaultInstance());
    putProto(DeletedBlocksTransactionSummary.getDefaultInstance());
  static <T extends Message> void putProto(T proto) {
    codecs.put(proto.getClass(), new ScmNonShadedGeneratedMessageCodec<>(proto.getParserForType()));
  }

@Russole Russole requested a review from szetszwo March 7, 2026 16:56
@Russole
Copy link
Contributor Author

Russole commented Mar 7, 2026

Thanks @szetszwo for the suggestion. I've updated the implementation accordingly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants