{"host": "kafka-main2007.codfw.wmnet", "state": "core_diff", "description": "Differences to core resources", "diff": {"full": {"total": 2971, "only_in_self": [], "only_in_other": ["File[/etc/kafka/super-user-client.properties]"], "resource_diffs": [{"resource": "Class[Confluent::Kafka::Broker]", "parameters": "--- Class[Confluent::Kafka::Broker].orig\n+++ Class[Confluent::Kafka::Broker]\n\n+    super_user_client_properties_template => confluent/kafka/super-user-client.properties.erb\n"}, {"resource": "File[/etc/kafka/super-user-client.properties]", "content": "--- /etc/kafka/super-user-client.properties.orig\n+++ /etc/kafka/super-user-client.properties\n@@ -0,0 +1,13 @@\n+# SPDX-License-Identifier: Apache-2.0\n+# NOTE: This file is managed by Puppet.\n+\n+# These credentials are the super user ones, useful\n+# to make changes to the Cluster's settings without\n+# the need of granting ANONYMOUS too much.\n+\n+security.protocol=SSL\n+ssl.keystore.location=/etc/kafka/ssl/kafka_main-codfw_broker.keystore.p12\n+ssl.keystore.password=nosuchpass\n+ssl.key.password=nosuchpass\n+ssl.truststore.location=/etc/ssl/localcerts/wmf-java-cacerts\n+ssl.truststore.password=changeit", "parameters": "--- File[/etc/kafka/super-user-client.properties].orig\n+++ File[/etc/kafka/super-user-client.properties]\n\n+    group => kafka\n+    mode  => 0440\n+    owner => root\n"}, {"resource": "File[/usr/local/bin/kafka]", "content": "--- /usr/local/bin/kafka.orig\n+++ /usr/local/bin/kafka\n@@ -0,0 +1,113 @@\n+#!/bin/bash\n+# SPDX-License-Identifier: Apache-2.0\n+\n+# NOTE: This file is managed by Puppet.\n+\n+SCRIPT_NAME=$(basename \"$0\")\n+\n+commands=$(ls /usr/bin/kafka-* | xargs -n 1 basename | sed 's@kafka-@  @g')\n+\n+USAGE=\"\n+$SCRIPT_NAME <command> [options]\n+\n+Handy wrapper around various kafka-* scripts.  Set the environment variables\n+KAFKA_ZOOKEEPER_URL, KAFKA_BOOTSTRAP_SERVERS so you don't have to keep typing\n+--broker-list or --bootstrap-server each time you want to\n+use a kafka-* script.\n+\n+Usage:\n+\n+Run $SCRIPT_NAME <command> with zero arguments/options to see command usage.\n+\n+Commands:\n+$commands\n+\n+Environment Variables:\n+  KAFKA_JAVA_HOME         - Value of JAVA_HOME to use for invoking Kafka commands.\n+  KAFKA_ZOOKEEPER_URL     - If this is set, any commands that take a --zookeeper\n+                            flag will be given this value.\n+  KAFKA_BOOTSTRAP_SERVERS - If this is set, any commands that take a --broker-list or\n+                            --bootstrap-server flag will be given this value.\n+                            Also any command that take a --authorizer-properties\n+                            will get the correct zookeeper.connect value.\n+\n+\"\n+\n+# Print usage if no <command> given, or $1 starts with '-'\n+if [ -z \"${1}\" ] || [ \"${1:0:1}\" == '-' ]; then\n+    echo \"${USAGE}\"\n+    exit 1\n+fi\n+\n+# All kafka scripts start with kafka-.\n+command=\"kafka-${1}\"\n+shift\n+\n+# Export JAVA_HOME as KAFKA_JAVA_HOME if it is set.\n+# This makes kafka-run-class use the preferred JAVA_HOME for Kafka.\n+if [ -n \"${KAFKA_JAVA_HOME}\" ]; then\n+    : ${JAVA_HOME=\"$KAFKA_JAVA_HOME\"}\n+    export JAVA_HOME\n+fi\n+\n+# Set BROKER_LIST_OPT if KAFKA_BOOTSTRAP_SERVERS is set and --broker-list has not\n+# also been passed in as a CLI arg.  This will be included\n+# in command functions that take a --broker-list argument.\n+if [ -n \"${KAFKA_BOOTSTRAP_SERVERS}\" ] && ! grep -q -- --broker-list<<<\"$@\"; then\n+    BROKER_LIST_OPT=\"--broker-list ${KAFKA_BOOTSTRAP_SERVERS}\"\n+fi\n+\n+# Set BOOTSTRAP_SERVER_OPT if KAFKA_BOOTSTRAP_SERVERS is set and --bootstrap-server has not\n+# also been passed in as a CLI arg.  This will be included\n+# in command functions that take a --bootstrap-server argument.\n+if [ -n \"${KAFKA_BOOTSTRAP_SERVERS}\" ] && ! grep -q -- --bootstrap-server<<<\"$@\"; then\n+    BOOTSTRAP_SERVER_OPT=\"--bootstrap-server ${KAFKA_BOOTSTRAP_SERVERS}\"\n+fi\n+\n+# Each of these lists signifies that either --broker-list or --bootstrap-server\n+# needs to be given to the $command.  If $command matches one of these,\n+# then we will add the opt if it is not provided already in $@.\n+# Until https://issues.apache.org/jira/browse/KAFKA-4307 is available, there are\n+# inconsistencies in broker CLI parameters.  Some use --bootstrap-server, others\n+# use --broker-list, so we have to support both for now.\n+# --broker-list should be removed in later versions in favor of --bootstrap-server\n+broker_list_commands=\"kafka-replica-verification\"\n+\n+bootstrap_server_commands=\"\n+kafka-acls \\\n+kafka-broker-api-versions \\\n+kafka-client-metrics \\\n+kafka-console-consumer \\\n+kafka-console-producer \\\n+kafka-consumer-perf-test \\\n+kafka-consumer-groups \\\n+kafka-delegation-tokens \\\n+kafka-delete-records \\\n+kafka-leader-election \\\n+kafka-configs \\\n+kafka-features \\\n+kafka-get-offsets \\\n+kafka-log-dirs \\\n+kafka-metadata-quorum \\\n+kafka-preferred-replica-election \\\n+kafka-reassign-partitions \\\n+kafka-replay-log-producer \\\n+kafka-topics \\\n+kafka-streams-application-reset \\\n+kafka-transactions \\\n+kafka-verifiable-consumer \\\n+kafka-verifiable-producer\"\n+\n+COMMAND_CONFIG_OPT=\"\"\n+\n+EXTRA_OPTS=\"\"\n+echo \"${broker_list_commands}\" | /bin/grep -q \"${command}\" && EXTRA_OPTS=\"${BROKER_LIST_OPT} \"\n+echo \"${bootstrap_server_commands}\" | /bin/grep -q \"${command}\" && EXTRA_OPTS=\"${EXTRA_OPTS}${BOOTSTRAP_SERVER_OPT} \"\n+EXTRA_OPTS=\"${EXTRA_OPTS}${COMMAND_CONFIG_OPT}\"\n+\n+# Print out the command we are about to exec, and then run it\n+# set -f to not expand wildcards in command, e.g. --topic '*'\n+set -f\n+echo \"${command} ${EXTRA_OPTS}$*\"\n+# shellcheck disable=SC2086,SC2145\n+${command} ${EXTRA_OPTS}\"$@\"", "parameters": "--- File[/usr/local/bin/kafka].orig\n+++ File[/usr/local/bin/kafka]\n\n-    source => puppet:///modules/confluent/kafka/kafka3.sh\n"}], "perc_changed": "0.13%"}, "core": {"total": 2971, "only_in_self": [], "only_in_other": ["File[/etc/kafka/super-user-client.properties]"], "resource_diffs": [{"resource": "File[/usr/local/bin/kafka]", "content": "--- /usr/local/bin/kafka.orig\n+++ /usr/local/bin/kafka\n@@ -0,0 +1,113 @@\n+#!/bin/bash\n+# SPDX-License-Identifier: Apache-2.0\n+\n+# NOTE: This file is managed by Puppet.\n+\n+SCRIPT_NAME=$(basename \"$0\")\n+\n+commands=$(ls /usr/bin/kafka-* | xargs -n 1 basename | sed 's@kafka-@  @g')\n+\n+USAGE=\"\n+$SCRIPT_NAME <command> [options]\n+\n+Handy wrapper around various kafka-* scripts.  Set the environment variables\n+KAFKA_ZOOKEEPER_URL, KAFKA_BOOTSTRAP_SERVERS so you don't have to keep typing\n+--broker-list or --bootstrap-server each time you want to\n+use a kafka-* script.\n+\n+Usage:\n+\n+Run $SCRIPT_NAME <command> with zero arguments/options to see command usage.\n+\n+Commands:\n+$commands\n+\n+Environment Variables:\n+  KAFKA_JAVA_HOME         - Value of JAVA_HOME to use for invoking Kafka commands.\n+  KAFKA_ZOOKEEPER_URL     - If this is set, any commands that take a --zookeeper\n+                            flag will be given this value.\n+  KAFKA_BOOTSTRAP_SERVERS - If this is set, any commands that take a --broker-list or\n+                            --bootstrap-server flag will be given this value.\n+                            Also any command that take a --authorizer-properties\n+                            will get the correct zookeeper.connect value.\n+\n+\"\n+\n+# Print usage if no <command> given, or $1 starts with '-'\n+if [ -z \"${1}\" ] || [ \"${1:0:1}\" == '-' ]; then\n+    echo \"${USAGE}\"\n+    exit 1\n+fi\n+\n+# All kafka scripts start with kafka-.\n+command=\"kafka-${1}\"\n+shift\n+\n+# Export JAVA_HOME as KAFKA_JAVA_HOME if it is set.\n+# This makes kafka-run-class use the preferred JAVA_HOME for Kafka.\n+if [ -n \"${KAFKA_JAVA_HOME}\" ]; then\n+    : ${JAVA_HOME=\"$KAFKA_JAVA_HOME\"}\n+    export JAVA_HOME\n+fi\n+\n+# Set BROKER_LIST_OPT if KAFKA_BOOTSTRAP_SERVERS is set and --broker-list has not\n+# also been passed in as a CLI arg.  This will be included\n+# in command functions that take a --broker-list argument.\n+if [ -n \"${KAFKA_BOOTSTRAP_SERVERS}\" ] && ! grep -q -- --broker-list<<<\"$@\"; then\n+    BROKER_LIST_OPT=\"--broker-list ${KAFKA_BOOTSTRAP_SERVERS}\"\n+fi\n+\n+# Set BOOTSTRAP_SERVER_OPT if KAFKA_BOOTSTRAP_SERVERS is set and --bootstrap-server has not\n+# also been passed in as a CLI arg.  This will be included\n+# in command functions that take a --bootstrap-server argument.\n+if [ -n \"${KAFKA_BOOTSTRAP_SERVERS}\" ] && ! grep -q -- --bootstrap-server<<<\"$@\"; then\n+    BOOTSTRAP_SERVER_OPT=\"--bootstrap-server ${KAFKA_BOOTSTRAP_SERVERS}\"\n+fi\n+\n+# Each of these lists signifies that either --broker-list or --bootstrap-server\n+# needs to be given to the $command.  If $command matches one of these,\n+# then we will add the opt if it is not provided already in $@.\n+# Until https://issues.apache.org/jira/browse/KAFKA-4307 is available, there are\n+# inconsistencies in broker CLI parameters.  Some use --bootstrap-server, others\n+# use --broker-list, so we have to support both for now.\n+# --broker-list should be removed in later versions in favor of --bootstrap-server\n+broker_list_commands=\"kafka-replica-verification\"\n+\n+bootstrap_server_commands=\"\n+kafka-acls \\\n+kafka-broker-api-versions \\\n+kafka-client-metrics \\\n+kafka-console-consumer \\\n+kafka-console-producer \\\n+kafka-consumer-perf-test \\\n+kafka-consumer-groups \\\n+kafka-delegation-tokens \\\n+kafka-delete-records \\\n+kafka-leader-election \\\n+kafka-configs \\\n+kafka-features \\\n+kafka-get-offsets \\\n+kafka-log-dirs \\\n+kafka-metadata-quorum \\\n+kafka-preferred-replica-election \\\n+kafka-reassign-partitions \\\n+kafka-replay-log-producer \\\n+kafka-topics \\\n+kafka-streams-application-reset \\\n+kafka-transactions \\\n+kafka-verifiable-consumer \\\n+kafka-verifiable-producer\"\n+\n+COMMAND_CONFIG_OPT=\"\"\n+\n+EXTRA_OPTS=\"\"\n+echo \"${broker_list_commands}\" | /bin/grep -q \"${command}\" && EXTRA_OPTS=\"${BROKER_LIST_OPT} \"\n+echo \"${bootstrap_server_commands}\" | /bin/grep -q \"${command}\" && EXTRA_OPTS=\"${EXTRA_OPTS}${BOOTSTRAP_SERVER_OPT} \"\n+EXTRA_OPTS=\"${EXTRA_OPTS}${COMMAND_CONFIG_OPT}\"\n+\n+# Print out the command we are about to exec, and then run it\n+# set -f to not expand wildcards in command, e.g. --topic '*'\n+set -f\n+echo \"${command} ${EXTRA_OPTS}$*\"\n+# shellcheck disable=SC2086,SC2145\n+${command} ${EXTRA_OPTS}\"$@\"", "parameters": "--- File[/usr/local/bin/kafka].orig\n+++ File[/usr/local/bin/kafka]\n\n-    source => puppet:///modules/confluent/kafka/kafka3.sh\n"}], "perc_changed": "0.07%"}, "main": {"total": 2971, "only_in_self": [], "only_in_other": ["File[/etc/kafka/super-user-client.properties]"], "resource_diffs": [{"resource": "Class[Confluent::Kafka::Broker]", "parameters": "--- Class[Confluent::Kafka::Broker].orig\n+++ Class[Confluent::Kafka::Broker]\n\n+    super_user_client_properties_template => confluent/kafka/super-user-client.properties.erb\n"}, {"resource": "File[/usr/local/bin/kafka]", "content": "--- /usr/local/bin/kafka.orig\n+++ /usr/local/bin/kafka\n@@ -0,0 +1,113 @@\n+#!/bin/bash\n+# SPDX-License-Identifier: Apache-2.0\n+\n+# NOTE: This file is managed by Puppet.\n+\n+SCRIPT_NAME=$(basename \"$0\")\n+\n+commands=$(ls /usr/bin/kafka-* | xargs -n 1 basename | sed 's@kafka-@  @g')\n+\n+USAGE=\"\n+$SCRIPT_NAME <command> [options]\n+\n+Handy wrapper around various kafka-* scripts.  Set the environment variables\n+KAFKA_ZOOKEEPER_URL, KAFKA_BOOTSTRAP_SERVERS so you don't have to keep typing\n+--broker-list or --bootstrap-server each time you want to\n+use a kafka-* script.\n+\n+Usage:\n+\n+Run $SCRIPT_NAME <command> with zero arguments/options to see command usage.\n+\n+Commands:\n+$commands\n+\n+Environment Variables:\n+  KAFKA_JAVA_HOME         - Value of JAVA_HOME to use for invoking Kafka commands.\n+  KAFKA_ZOOKEEPER_URL     - If this is set, any commands that take a --zookeeper\n+                            flag will be given this value.\n+  KAFKA_BOOTSTRAP_SERVERS - If this is set, any commands that take a --broker-list or\n+                            --bootstrap-server flag will be given this value.\n+                            Also any command that take a --authorizer-properties\n+                            will get the correct zookeeper.connect value.\n+\n+\"\n+\n+# Print usage if no <command> given, or $1 starts with '-'\n+if [ -z \"${1}\" ] || [ \"${1:0:1}\" == '-' ]; then\n+    echo \"${USAGE}\"\n+    exit 1\n+fi\n+\n+# All kafka scripts start with kafka-.\n+command=\"kafka-${1}\"\n+shift\n+\n+# Export JAVA_HOME as KAFKA_JAVA_HOME if it is set.\n+# This makes kafka-run-class use the preferred JAVA_HOME for Kafka.\n+if [ -n \"${KAFKA_JAVA_HOME}\" ]; then\n+    : ${JAVA_HOME=\"$KAFKA_JAVA_HOME\"}\n+    export JAVA_HOME\n+fi\n+\n+# Set BROKER_LIST_OPT if KAFKA_BOOTSTRAP_SERVERS is set and --broker-list has not\n+# also been passed in as a CLI arg.  This will be included\n+# in command functions that take a --broker-list argument.\n+if [ -n \"${KAFKA_BOOTSTRAP_SERVERS}\" ] && ! grep -q -- --broker-list<<<\"$@\"; then\n+    BROKER_LIST_OPT=\"--broker-list ${KAFKA_BOOTSTRAP_SERVERS}\"\n+fi\n+\n+# Set BOOTSTRAP_SERVER_OPT if KAFKA_BOOTSTRAP_SERVERS is set and --bootstrap-server has not\n+# also been passed in as a CLI arg.  This will be included\n+# in command functions that take a --bootstrap-server argument.\n+if [ -n \"${KAFKA_BOOTSTRAP_SERVERS}\" ] && ! grep -q -- --bootstrap-server<<<\"$@\"; then\n+    BOOTSTRAP_SERVER_OPT=\"--bootstrap-server ${KAFKA_BOOTSTRAP_SERVERS}\"\n+fi\n+\n+# Each of these lists signifies that either --broker-list or --bootstrap-server\n+# needs to be given to the $command.  If $command matches one of these,\n+# then we will add the opt if it is not provided already in $@.\n+# Until https://issues.apache.org/jira/browse/KAFKA-4307 is available, there are\n+# inconsistencies in broker CLI parameters.  Some use --bootstrap-server, others\n+# use --broker-list, so we have to support both for now.\n+# --broker-list should be removed in later versions in favor of --bootstrap-server\n+broker_list_commands=\"kafka-replica-verification\"\n+\n+bootstrap_server_commands=\"\n+kafka-acls \\\n+kafka-broker-api-versions \\\n+kafka-client-metrics \\\n+kafka-console-consumer \\\n+kafka-console-producer \\\n+kafka-consumer-perf-test \\\n+kafka-consumer-groups \\\n+kafka-delegation-tokens \\\n+kafka-delete-records \\\n+kafka-leader-election \\\n+kafka-configs \\\n+kafka-features \\\n+kafka-get-offsets \\\n+kafka-log-dirs \\\n+kafka-metadata-quorum \\\n+kafka-preferred-replica-election \\\n+kafka-reassign-partitions \\\n+kafka-replay-log-producer \\\n+kafka-topics \\\n+kafka-streams-application-reset \\\n+kafka-transactions \\\n+kafka-verifiable-consumer \\\n+kafka-verifiable-producer\"\n+\n+COMMAND_CONFIG_OPT=\"\"\n+\n+EXTRA_OPTS=\"\"\n+echo \"${broker_list_commands}\" | /bin/grep -q \"${command}\" && EXTRA_OPTS=\"${BROKER_LIST_OPT} \"\n+echo \"${bootstrap_server_commands}\" | /bin/grep -q \"${command}\" && EXTRA_OPTS=\"${EXTRA_OPTS}${BOOTSTRAP_SERVER_OPT} \"\n+EXTRA_OPTS=\"${EXTRA_OPTS}${COMMAND_CONFIG_OPT}\"\n+\n+# Print out the command we are about to exec, and then run it\n+# set -f to not expand wildcards in command, e.g. --topic '*'\n+set -f\n+echo \"${command} ${EXTRA_OPTS}$*\"\n+# shellcheck disable=SC2086,SC2145\n+${command} ${EXTRA_OPTS}\"$@\"", "parameters": "--- File[/usr/local/bin/kafka].orig\n+++ File[/usr/local/bin/kafka]\n\n-    source => puppet:///modules/confluent/kafka/kafka3.sh\n"}], "perc_changed": "0.10%"}}}