Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KAFKA-12183: Add the KIP-631 metadata record definitions #9876

Merged
merged 1 commit into from
Jan 14, 2021

Conversation

cmccabe
Copy link
Contributor

@cmccabe cmccabe commented Jan 12, 2021

Add the metadata gradle module, which will contain the metadata record
definitions, and other metadata-related broker-side code.

Add MetadataParser, MetadataParseException, etc.

@cmccabe cmccabe added the kraft label Jan 12, 2021
Copy link
Member

@jsancio jsancio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @cmccabe

{ "name": "Id", "type": "int32", "versions": "0 ",
"about": "The broker ID to unfence." },
{ "name": "Epoch", "type": "int64", "versions": "0 ",
"about": "The epoch of the broker to unfence." }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In UnregisterBrokerRecord and RegisterBrokerRecord we call this BrokerId and BrokerEpoch receptively. Should we use the same names here for consistency?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

{ "name": "Id", "type": "int32", "versions": "0 ",
"about": "The broker ID to fence. It will be removed from all ISRs." },
{ "name": "Epoch", "type": "int64", "versions": "0 ",
"about": "The epoch of the broker to fence." }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In UnregisterBrokerRecord and RegisterBrokerRecord we call this BrokerId and BrokerEpoch receptively. Should we use the same names here for consistency?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

"fields": [
{ "name": "Id", "type": "int32", "versions": "0 ",
"about": "The broker ID to fence. It will be removed from all ISRs." },
{ "name": "Epoch", "type": "int64", "versions": "0 ",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extra spaces before { "name":....

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

// limitations under the License.

{
"apiKey": 14,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no message with apiKey 13.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is a message specified in the KIP. I'll add it.

import static org.junit.jupiter.api.Assertions.assertTrue;

@Timeout(value = 40)
public class MetadataParserTest {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add a test that checks that the api keys for metadata records are consecutive numbers?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: you can remove value =

Copy link
Member

@mumrah mumrah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one minor question inline

* SHUTTING_DOWN
*/
@InterfaceStability.Evolving
public enum BrokerState {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not that it does any harm, but is this class needed for this PR? Maybe these values are referenced by the metadata records?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was originally going to remove BrokerStates.scala and replace it with this BrokerState.java file. That was back when KIP-631 serialized broker states in the RPCs. But now that we're not doing that any more, there isn't really a strong reason to do this. So let's just skip it, I guess...

compile libs.jacksonJDK8Datatypes
compileOnly libs.log4j
testCompile libs.junitJupiter
testCompile libs.hamcrest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we actually use hamcrest in this module? If not, we should remove this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, we use it

testCompile libs.junitJupiter
testCompile libs.hamcrest
testCompile libs.slf4jlog4j
testCompile project(':clients').sourceSets.test.output // for org.apache.kafka.test.IntegrationTest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you don't need this if that comment is true.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

later PRs add IntegrationTest to the metadata module... simpler to just keep it in this PR

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not right, the mentioned annotation is only used for JUnit 4. I submitted a minor PR that fixes the comment at least: #9897

Add the metadata gradle module, which will contain the metadata record
definitions, and other metadata-related broker-side code.

Add MetadataParser, MetadataParseException, etc.
@cmccabe cmccabe merged commit 217334b into apache:trunk Jan 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants