Compilation results for cirrussearch2089.codfw.wmnet: System changes detected

You can retrieve this result from host.json.

Catalog differences

Summary

Total Resources:	2682
Resources added:	712
Resources removed:	5
Resources modified:	733
Change percentage:	54.06%

Resources only in the new catalog

Exec[java__cacert_wmf:puppetca.pem]
Systemd::Service[prometheus-wmf-elasticsearch-exporter-9200]
File[/usr/local/bin/opensearch-disable-readahead.sh]
Ferm::Service[opensearch-http-9200]
File[/etc/nagios/nrpe.d/check_opensearch_unassigned_shard_check_9200.cfg]
File[/lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer]
Logrotate::Conf[wmf_auto_restart_prometheus-elasticsearch-exporter-9600]
Nrpe::Check[check_opensearch_unassigned_shard_check_9600]
File[/etc/logrotate.d/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]
Systemd::Timer::Job[nrpe2nodexp-check_tcp-mss-clamper_status]
Profile::Prometheus::Wmf_elasticsearch_exporter[cirrussearch2089:9200]
File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet]
File[/lib/systemd/system/prometheus-wmf-elasticsearch-exporter-9600.service]
Service[prometheus-elasticsearch-exporter-9200]
Systemd::Unit[nrpe2nodexp-opensearch_shard_size_check_9600.timer]
Nrpe::Check[check_opensearch_unassigned_shard_check_9200]
Class[Opensearch]
Alternatives::Java[11]
File[/etc/opensearch/instances]
Systemd::Unit[wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer]
Exec[systemd daemon-reload for prometheus-elasticsearch-exporter-9600.service (prometheus-elasticsearch-exporter-9600)]
Prometheus::Alert::Rule[check_opensearch_shard_size_check_9600_6ce4fe694f5df8cba9e0032196c72747]
Monitoring::Service[opensearch_unassigned_shard_check_9600]
Systemd::Timer::Job[wmf_auto_restart_prometheus-elasticsearch-exporter-9200]
File[/lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9600.service]
File[/etc/logrotate.d/opensearch-disable-readahead-production-search-codfw]
File[/var/log/opensearch/opensearch.log]
File[/etc/nagios/nrpe.d/check_check_tcp-mss-clamper_status.cfg]
File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-shard-size-check-9600.conf]
File[/etc/ssh/userkeys/ebernhardson]
Exec[systemd daemon-reload for opensearch-production-search-psi-codfw-gc-log-cleanup.timer (opensearch-production-search-psi-codfw-gc-log-cleanup.timer)]
File[/etc/rsyslog.d/40-wmf-auto-restart-prometheus-elasticsearch-exporter-9600.conf]
Service[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer]
Ferm::Service[opensearch-inter-node-9700]
Exec[systemd daemon-reload for opensearch-production-search-psi-codfw-gc-log-cleanup.service (opensearch-production-search-psi-codfw-gc-log-cleanup.service)]
Monitoring::Service[opensearch_shards_9600]
Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9600.timer (nrpe2nodexp-opensearch_shard_size_check_9600.timer)]
File[/var/log/opensearch/opensearch_index_indexing_slowlog.log]
Systemd::Service[prometheus_lvs_realserver_mss]
Systemd::Timer::Job[wmf_auto_restart_prometheus-elasticsearch-exporter-9600]
Systemd::Service[opensearch-hot-threads-log]
File[/etc/ssh/userkeys/pfischer]
File[/lib/systemd/system/opensearch-disable-readahead.timer]
Interface::Manual[ipip_ipv4]
File[/usr/local/bin/prometheus-lvs-realserver-mss]
File[/lib/systemd/system/clean_up_dlq.timer]
Admin::User[dr0ptp4kt]
Profile::Auto_restarts::Service[prometheus-wmf-elasticsearch-exporter-9200]
Sudo::User[nrpe-check_opensearch_shard_size_check_9600]
Systemd::Unit[nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer]
Systemd::Unit[opensearch-hot-threads-log.timer]
Systemd::Syslog[prometheus_lvs_realserver_mss]
Systemd::Syslog[prometheus_ferm_mss]
File[/etc/ferm/conf.d/10_opensearch-inter-node-9700]
Elasticsearch::Tlsproxy[production-search-codfw]
Nrpe::Check[check_opensearch_shards_9600]
Exec[systemd daemon-reload for opensearch-production-search-codfw-gc-log-cleanup.service (opensearch-production-search-codfw-gc-log-cleanup.service)]
Logrotate::Conf[opensearch-production-search-codfw-gc-log-cleanup]
Systemd::Service[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]
File[/var/log/opensearch-disable-readahead]
File[/etc/ferm/conf.d/10_opensearch-inter-node-9300]
Logstash::Output::Udp[gelf_relay]
File[/etc/logrotate.d/opensearch]
Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer (nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer)]
Exec[systemd daemon-reload for opensearch-hot-threads-log.timer (opensearch-hot-threads-log.timer)]
Prometheus::Alert::Rule[check_opensearch_unassigned_shard_check_9600_8641ad857de0a06c2748b18ea2c135ab]
Systemd::Syslog[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]
Systemd::Unit[nrpe2nodexp-opensearch_unassigned_shard_check_9200.service]
File[/usr/local/bin/ispooled]
Service[prometheus-wmf-elasticsearch-exporter-9600]
Monitoring::Service[check_tcp-mss-clamper_status]
File[/lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer]
Systemd::Unit[nrpe2nodexp-opensearch_shard_size_check_9200.timer]
File[/etc/ssl/dhparam.pem]
Service[logstash]
Sudo::User[nrpe-check_opensearch_shards_9600]
Package[logstash]
Nrpe::Check[check_opensearch_shard_size_check_9200]
File[/var/log/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]
Logrotate::Conf[opensearch-production-search-psi-codfw-gc-log-cleanup]
Systemd::Unit[clean_up_dlq.service]
Systemd::Timer[nrpe2nodexp-opensearch_shards_9200]
File_line[rm_eno12399np0_clsact_eno12399np0]
File[/lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9600.timer]
File[/lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9200.service]
File[/etc/rsyslog.d/40-clean-up-dlq.conf]
File[/etc/opensearch/production-search-codfw/log4j2.properties]
Package[tcp-mss-clamper]
File[/etc/update-motd.d/96-cluster-memberships]
Exec[systemd daemon-reload for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer (wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer)]
Nrpe::Check[check_opensearch_shard_size_check_9600]
Exec[systemd daemon-reload for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service (wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service)]
Class[Profile::Logstash::Gelf_relay]
Class[Java]
File[/etc/rsyslog.d/40-opensearch-production-search-codfw-gc-log-cleanup.conf]
File[/etc/nagios/nrpe.d/check_opensearch_shards_9200.cfg]
Elasticsearch::Tlsproxy[production-search-psi-codfw]
Service[tcp-mss-clamper]
File[/etc/conftool/config.yaml]
File[/usr/local/bin/jheapdump]
Rsyslog::Conf[nrpe2nodexp-opensearch_unassigned_shard_check_9200]
File[/usr/lib/nagios/plugins/check_elasticsearch]
Augeas[ipip60_set_up]
Service[nrpe2nodexp-opensearch_shard_size_check_9200.timer]
File[/etc/logrotate.d/nginx]
Systemd::Service[tcp-mss-clamper]
Nrpe::Check[check_opensearch_shards_9200]
Systemd::Timer[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]
File[/etc/apt/sources.list.d/wikimedia-opensearch-plugins.list]
Systemd::Tmpfile[opensearch-production-search-psi-codfw]
File[/etc/nginx/sites-available]
Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9600.service (nrpe2nodexp-opensearch_unassigned_shard_check_9600.service)]
Systemd::Syslog[clean_up_dlq]
Sudo::Group[elasticsearch-roots]
File[/etc/logstash/log4j2.properties]
Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9200.service (nrpe2nodexp-opensearch_shard_size_check_9200.service)]
Exec[systemd daemon-reload for opensearch-hot-threads-log.service (opensearch-hot-threads-log.service)]
Exec[ip link set up dev ipip60]
File[/etc/rsyslog.d/40-prometheus-ferm-mss.conf]
Systemd::Unit[nrpe2nodexp-opensearch_shard_size_check_9600.service]
User[dcausse]
Package[opensearch]
Monitoring::Service[opensearch_shard_size_check_9600]
Service[opensearch_1@production-search-psi-codfw]
Systemd::Unit[opensearch-disable-readahead-production-search-codfw.service]
Admin::Groupmembers[elasticsearch-roots]
Augeas[ipip60_manual]
Exec[apt_repository_wikimedia-opensearch]
File[/home/pfischer]
File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chained.pem]
Java::Cacert[wmf:Wikimedia_Internal_Root_CA]
Systemd::Timer[opensearch-production-search-psi-codfw-gc-log-cleanup]
File[/lib/systemd/system/nrpe2nodexp-check_tcp-mss-clamper_status.timer]
Class[Profile::Lvs::Configuration]
Exec[systemd daemon-reload for wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer (wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer)]
Logrotate::Conf[opensearch]
Systemd::Timer::Job[nrpe2nodexp-opensearch_unassigned_shard_check_9600]
Exec[systemd daemon-reload for wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service (wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service)]
Service[prometheus-wmf-elasticsearch-exporter-9200]
File[/etc/conftool/schema.yaml]
File[/etc/ferm/conf.d/10_opensearch-https-9643]
Class[Conftool::Config]
Profile::Auto_restarts::Service[prometheus-wmf-elasticsearch-exporter-9600]
Rsyslog::Conf[opensearch-disable-readahead-production-search-psi-codfw]
Logrotate::Conf[wmf_auto_restart_prometheus-elasticsearch-exporter-9200]
Exec[mask_nginx.service]
Exec[elasticsearch-roots_ensure_members]
Systemd::Syslog[opensearch-disable-readahead-production-search-codfw]
Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_unassigned_shard_check_9200]
Etcd::Client::Config[/root/.etcdrc]
File[/lib/systemd/system/prometheus-elasticsearch-exporter-9600.service]
File[/etc/logstash/conf.d/10-input-gelf-gelf_relay.conf]
File[/etc/ssh/userkeys/dcausse]
File[/usr/local/bin/pool]
Class[Conftool::Scripts]
File[/etc/nginx/sites-available/production-search-psi-codfw]
Interface::Ipip[ipip_ipv4]
File[/etc/nginx/sites-enabled/production-search-psi-codfw]
Exec[Refresh tmpfile opensearch-production-search-codfw]
Admin::User[ebernhardson]
Systemd::Service[wmf_auto_restart_prometheus-elasticsearch-exporter-9200]
Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9200.service (nrpe2nodexp-opensearch_unassigned_shard_check_9200.service)]
Systemd::Unit[prometheus-wmf-elasticsearch-exporter-9600]
Tlsproxy::Localssl[production-search-codfw]
Exec[systemd daemon-reload for opensearch-production-search-codfw-gc-log-cleanup.timer (opensearch-production-search-codfw-gc-log-cleanup.timer)]
File[/lib/systemd/system/logstash.service]
Systemd::Unit[opensearch-production-search-psi-codfw-gc-log-cleanup.timer]
Exec[systemd daemon-reload for opensearch-disable-readahead.service (opensearch-disable-readahead.service)]
Profile::Prometheus::Elasticsearch_exporter[cirrussearch2089:9200]
Exec[systemd daemon-reload for opensearch-disable-readahead-production-search-psi-codfw.service (opensearch-disable-readahead-production-search-psi-codfw.service)]
File_line[auto_restart_file_presence_prometheus-wmf-elasticsearch-exporter-9200]
Conftool::Scripts::Safe_service_restart[nginx]
File[/etc/logrotate.d/wmf_auto_restart_prometheus-elasticsearch-exporter-9200]
Systemd::Timer[prometheus_ferm_mss]
Exec[systemd daemon-reload for opensearch_1@.service (opensearch_1@.service)]
Systemd::Service[nrpe2nodexp-opensearch_unassigned_shard_check_9600]
File[/lib/systemd/system/opensearch-hot-threads-log.timer]
Exec[systemd daemon-reload for clean_up_dlq.timer (clean_up_dlq.timer)]
Service[wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer]
Exec[renew certificate - discovery__cirrussearch2089_codfw_wmnet]
Ferm::Service[opensearch-http-9600]
Systemd::Timer[opensearch-disable-readahead]
Systemd::Timer[nrpe2nodexp-opensearch_shard_size_check_9200]
Systemd::Mask[nginx.service]
Prometheus::Elasticsearch_exporter[localhost:9600]
File[/var/log/clean_up_dlq]
Systemd::Timer::Job[prometheus_lvs_realserver_mss]
Prometheus::Alert::Rule[check_opensearch_shard_size_check_9200_ce424a305cb5464806cbf788bde4430a]
Etcd::Client::Config[/etc/etcd/etcdrc]
Systemd::Timer::Job[nrpe2nodexp-opensearch_shards_9600]
Prometheus::Alert::Rule[check_check_tcp-mss-clamper_status_15c652bc7390794191321c22a2885d1f]
Systemd::Syslog[opensearch-production-search-codfw-gc-log-cleanup]
Rsyslog::Conf[nrpe2nodexp-opensearch_shard_size_check_9200]
Systemd::Service[nrpe2nodexp-opensearch_unassigned_shard_check_9200]
Sysctl::Parameters[TCP Fast Open]
Package[wmf-opensearch-search-plugins]
Systemd::Service[nrpe2nodexp-opensearch_shard_size_check_9200]
Interface::Ip[ipip_ipv4 ipv4]
Systemd::Service[opensearch-production-search-codfw-gc-log-cleanup]
Class[Prometheus::Elasticsearch_exporter::Common]
File[/var/log/wmf_auto_restart_prometheus-elasticsearch-exporter-9200]
Systemd::Syslog[wmf_auto_restart_prometheus-elasticsearch-exporter-9600]
Logrotate::Conf[opensearch-disable-readahead]
Exec[systemd daemon-reload for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer (wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer)]
Admin::User[dcausse]
Systemd::Service[nrpe2nodexp-check_tcp-mss-clamper_status]
Service[opensearch-hot-threads-log.timer]
Systemd::Service[opensearch-production-search-psi-codfw-gc-log-cleanup]
Conftool::Scripts::Safe_service_restart[opensearch_1@production-search-psi-codfw]
Exec[systemd daemon-reload for prometheus-wmf-elasticsearch-exporter-9600.service (prometheus-wmf-elasticsearch-exporter-9600)]
Rsyslog::Conf[nrpe2nodexp-check_tcp-mss-clamper_status]
File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.csr]
Systemd::Syslog[opensearch-disable-readahead-production-search-psi-codfw]
File[/usr/local/bin/depool-nginx]
Profile::Prometheus::Wmf_elasticsearch_exporter[cirrussearch2089:9600]
Systemd::Unit[wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service]
File[/etc/sudoers.d/nrpe-check_check_tcp-mss-clamper_status]
Systemd::Unit[prometheus_lvs_realserver_mss.timer]
Class[Icinga::Elasticsearch::Base_plugin]
Systemd::Timer::Job[opensearch-disable-readahead]
Rsyslog::Conf[opensearch-disable-readahead-production-search-codfw]
Service[prometheus-elasticsearch-exporter-9600]
Package[nginx-light]
Monitoring::Service[elasticsearch-https-production-search-psi-codfw]
File[/etc/tmpfiles.d/opensearch-production-search-codfw.conf]
Admin::Group[elasticsearch-roots]
File[/usr/local/bin/cleanup-dlq]
Monitoring::Service[elasticsearch-https-production-search-codfw]
Systemd::Unit[opensearch-production-search-codfw-gc-log-cleanup.service]
File[/var/lib/nginx]
Systemd::Tmpfile[opensearch-production-search-codfw]
File[/lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service]
Exec[symlink /etc/opensearch/production-search-codfw/sudachi]
Class[Poolcounter::Client::Python]
File[/var/log/opensearch-production-search-psi-codfw-gc-log-cleanup]
Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9200.timer (nrpe2nodexp-opensearch_shards_9200.timer)]
File[/etc/opensearch/production-search-codfw/scripts]
Package[logstash-old-name]
Ssh::Userkey[dr0ptp4kt]
Ssh::Userkey[pfischer]
Rsyslog::Conf[opensearch-disable-readahead]
Systemd::Service[opensearch-disable-readahead-production-search-psi-codfw]
Systemd::Unit[opensearch-production-search-psi-codfw-gc-log-cleanup.service]
Systemd::Unit[nrpe2nodexp-check_tcp-mss-clamper_status.service]
Java::Package[openjdk-jdk-11]
File[/srv/opensearch/production-search-codfw]
File[/usr/local/bin/pool-opensearch_1@production-search-psi-codfw]
Opensearch::Keystore[s3-credentials-user-production-search-psi-codfw]
File[/usr/local/sbin/restart-opensearch_1@production-search-psi-codfw]
Systemd::Service[nrpe2nodexp-opensearch_shard_size_check_9600]
Systemd::Service[prometheus-elasticsearch-exporter-9200]
Ferm::Service[opensearch-inter-node-9300]
File[/lib/systemd/system/opensearch-production-search-codfw-gc-log-cleanup.service]
Nrpe::Plugin[check_systemd_unit_status]
Exec[systemd daemon-reload for opensearch-disable-readahead-production-search-codfw.service (opensearch-disable-readahead-production-search-codfw.service)]
File[/lib/systemd/system/opensearch-disable-readahead-production-search-psi-codfw.timer]
Exec[ip addr add 127.0.0.42/32 dev ipip0]
Service[wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer]
Systemd::Service[wmf_auto_restart_prometheus-elasticsearch-exporter-9600]
Nrpe::Monitor_service[opensearch_shards_9200]
File[/lib/systemd/system/tcp-mss-clamper.service]
Class[Passwords::Etcd]
Exec[unmask_nginx.service]
Exec[systemd daemon-reload for opensearch-disable-readahead.timer (opensearch-disable-readahead.timer)]
Systemd::Timer::Job[opensearch-disable-readahead-production-search-psi-codfw]
File[/etc/apt/sources.list.d/wikimedia-opensearch.list]
File[/etc/opensearch/production-search-psi-codfw/java-security.policy]
Systemd::Unit[opensearch-disable-readahead-production-search-codfw.timer]
File[/lib/systemd/system/nrpe2nodexp-check_tcp-mss-clamper_status.service]
Augeas[ipip0_add_up]
Service[clean_up_dlq.timer]
Exec[update_java_alternatives_11]
File[/etc/rsyslog.d/40-wmf-auto-restart-prometheus-wmf-elasticsearch-exporter-9600.conf]
File[/usr/share/opensearch/plugins]
Systemd::Service[nrpe2nodexp-opensearch_shards_9600]
Service[opensearch_1@production-search-codfw]
Systemd::Unit[clean_up_dlq.timer]
Service[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer]
Apt::Repository[wikimedia-opensearch]
Exec[s3-credentials-user-production-search-psi-codfw]
Class[Opensearch::Packages]
Class[Profile::Opensearch::Server]
File[/lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service]
File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-shard-size-check-9200.conf]
Systemd::Timer[nrpe2nodexp-opensearch_shards_9600]
File[/lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer]
Class[Poolcounter::Client]
User[pfischer]
File[/etc/rsyslog.d/40-opensearch-disable-readahead.conf]
Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_shards_9200]
File[/usr/local/sbin/restart-nginx]
Systemd::Timer::Job[nrpe2nodexp-opensearch_shard_size_check_9600]
Systemd::Timer::Job[opensearch-production-search-codfw-gc-log-cleanup]
Systemd::Timer::Job[nrpe2nodexp-opensearch_unassigned_shard_check_9200]
File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet-key.pem]
File[/etc/logrotate.d/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]
File[/etc/rsyslog.d/25-nrpe2nodexp-check-tcp-mss-clamper-status.conf]
File[/etc/ssh/userkeys/dr0ptp4kt]
Rsyslog::Conf[wmf_auto_restart_prometheus-elasticsearch-exporter-9200]
File[/usr/local/sbin/restart-opensearch_1@production-search-codfw]
Systemd::Syslog[opensearch-production-search-psi-codfw-gc-log-cleanup]
File[/etc/opensearch/production-search-psi-codfw/jvm.options]
File_line[rm_lo_clsact_lo]
Class[Profile::Tcp_fast_open]
File[/lib/systemd/system/prometheus-elasticsearch-exporter-9200.service]
Prometheus::Node_lvs_realserver_mss[lvs_clamped_ipport]
Systemd::Timer[prometheus_lvs_realserver_mss]
Interface::Post_up_command[clsact_eno12399np0]
Nrpe::Check[check_check_tcp-mss-clamper_status]
File_line[auto_restart_file_presence_prometheus-wmf-elasticsearch-exporter-9600]
Systemd::Unit[nrpe2nodexp-opensearch_shard_size_check_9200.service]
Exec[apt_repository_wikimedia-opensearch-plugins]
File[/etc/logrotate.d/opensearch-production-search-codfw-gc-log-cleanup]
File[/usr/local/bin/pool-nginx]
Systemd::Service[clean_up_dlq]
Service[nrpe2nodexp-opensearch_shards_9200.timer]
Systemd::Timer[nrpe2nodexp-opensearch_shard_size_check_9600]
File[/usr/local/bin/opensearch_hot_threads_logger.py]
File[/lib/systemd/system/opensearch_1@.service]
Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_shards_9600]
Exec[systemd daemon-reload for wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer (wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer)]
Profile::Prometheus::Elasticsearch_exporter[cirrussearch2089:9600]
File[/etc/nginx/sites-available/production-search-codfw]
Systemd::Unit[wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service]
File[/etc/rsyslog.d/40-opensearch-disable-readahead-production-search-codfw.conf]
Exec[systemd daemon-reload for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service (wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service)]
Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9600.service (nrpe2nodexp-opensearch_shards_9600.service)]
File[/etc/sudoers.d/nrpe-check_opensearch_shard_size_check_9600]
Profile::Auto_restarts::Service[prometheus-elasticsearch-exporter-9600]
File[/etc/nagios/nrpe.d/check_opensearch_shard_size_check_9600.cfg]
Exec[tlsproxy localssl default_server on ports [9243]]
File[/etc/opensearch/production-search-psi-codfw/scripts]
Service[prometheus-elasticsearch-exporter]
Exec[java__cacert_wmf:Wikimedia_Internal_Root_CA]
File[/etc/cfssl/csr/discovery__cirrussearch2089_codfw_wmnet.csr]
Class[Profile::Lvs::Realserver::Ipip]
Monitoring::Exported_nagios_service[cirrussearch2089 elasticsearch-https-production-search-codfw]
Class[Wmflib::Service::Catalog]
Systemd::Service[prometheus_ferm_mss]
Systemd::Timer::Job[opensearch-disable-readahead-production-search-codfw]
File[/lib/systemd/system/clean_up_dlq.service]
File[/etc/conftool/json-schema/]
File[/etc/sysctl.d/70-TCP-Fast-Open.conf]
File[/etc/nagios/nrpe.d/check_opensearch_shard_size_check_9200.cfg]
Rsyslog::Conf[opensearch-production-search-psi-codfw-gc-log-cleanup]
File[/usr/local/bin/drain]
Ssh::Userkey[ebernhardson]
Class[Profile::Opensearch::Cirrus::Server]
Ssh::Userkey[dcausse]
File[/etc/ferm/conf.d/10_ip6ip6]
Class[Sslcert::Dhparam]
File[/etc/ferm/conf.d/10_opensearch-http-9600]
Systemd::Unit[nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer]
Exec[tlsproxy localssl default_server on ports [9643]]
Class[Etcd::Client::Globalconfig]
Interface::Clsact[clsact_eno12399np0]
Logrotate::Conf[prometheus_lvs_realserver_mss]
Rsyslog::Conf[clean_up_dlq]
Systemd::Timer[opensearch-hot-threads-log]
File[/lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service]
File[/srv/opensearch]
Service[prometheus_ferm_mss.timer]
File[/etc/logrotate.d/opensearch-production-search-psi-codfw-gc-log-cleanup]
Logrotate::Conf[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]
File[/etc/sudoers.d/nrpe-check_opensearch_shard_size_check_9200]
Rsyslog::Conf[nrpe2nodexp-opensearch_unassigned_shard_check_9600]
File[/home/dcausse]
Augeas[ipip60_add_up]
Package[nginx-common]
Systemd::Service[opensearch-disable-readahead]
Systemd::Unit[opensearch-hot-threads-log.service]
Exec[systemd reload for nginx systemd fragments]
File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-unassigned-shard-check-9200.conf]
Package[prometheus-elasticsearch-exporter]
Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9200.service (nrpe2nodexp-opensearch_shards_9200.service)]
Augeas[ipip0_set_up]
Service[opensearch]
Monitoring::Exported_nagios_service[cirrussearch2089 elasticsearch-https-production-search-psi-codfw]
Systemd::Syslog[opensearch-disable-readahead]
Sudo::User[nrpe-check_opensearch_unassigned_shard_check_9200]
Sysctl::Conffile[TCP Fast Open]
Systemd::Service[opensearch-disable-readahead-production-search-codfw]
User[ebernhardson]
File[/usr/local/bin/depool]
File[/etc/elasticsearch]
File[/etc/ferm/conf.d/10_ipip]
Node[__node_regexp__cirrussearch2061062069073074075076087088089090091111.codfw.]
Exec[ip link add name ipip0 type ipip external]
Service[opensearch-disable-readahead-production-search-psi-codfw.timer]
File[/etc/systemd/system/nginx.service.d/numa.conf]
Exec[nginx-reload]
Package[openjdk-11-jdk]
File[/etc/opensearch/production-search-psi-codfw/opensearch.yml]
File[/lib/systemd/system/opensearch-disable-readahead-production-search-psi-codfw.service]
File[/usr/lib/nagios/plugins/check_elasticsearch_unassigned_shards.py]
Monitoring::Exported_nagios_service[cirrussearch2089 check_tcp-mss-clamper_status]
Systemd::Timer::Job[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]
Package[edac-utils]
Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_shard_size_check_9200]
Monitoring::Service[opensearch_shard_size_check_9200]
Nrpe::Monitor_service[opensearch_unassigned_shard_check_9200]
Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9200.timer (nrpe2nodexp-opensearch_shard_size_check_9200.timer)]
Admin::Hashuser[dr0ptp4kt]
File[/etc/default/prometheus-elasticsearch-exporter]
File[/lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer]
File[/etc/opensearch/production-search-codfw/jvm.options]
File[/lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9200.timer]
File[/etc/tmpfiles.d/opensearch-production-search-psi-codfw.conf]
File_line[auto_restart_file_presence_prometheus-elasticsearch-exporter-9200]
File[/etc/sudoers.d/nrpe-check_opensearch_shards_9600]
File[/lib/systemd/system/prometheus_lvs_realserver_mss.timer]
Systemd::Timer[wmf_auto_restart_prometheus-elasticsearch-exporter-9600]
File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-shards-9600.conf]
File[/etc/nginx/conf.d]
File[/etc/rsyslog.d/40-wmf-auto-restart-prometheus-wmf-elasticsearch-exporter-9200.conf]
File[/etc/sudoers.d/nrpe-check_opensearch_shards_9200]
Prometheus::Alert::Rule[check_opensearch_shards_9200_8cb398d9f4c08b77d96bfdab16485563]
Class[Profile::Rsyslog::Udp_json_logback_compat]
Opensearch::Instance[production-search-codfw]
Logrotate::Rule[opensearch]
Systemd::Unmask[nginx.service]
Systemd::Timer[opensearch-disable-readahead-production-search-codfw]
Systemd::Timer::Job[opensearch-hot-threads-log]
Udev::Rule[opensearch-readahead]
Exec[systemd daemon-reload for nrpe2nodexp-check_tcp-mss-clamper_status.service (nrpe2nodexp-check_tcp-mss-clamper_status.service)]
File[/etc/rsyslog.d/40-prometheus-lvs-realserver-mss.conf]
File[/lib/systemd/system/prometheus_lvs_realserver_mss.service]
Admin::User[pfischer]
Systemd::Unit[prometheus-elasticsearch-exporter-9200]
Nrpe::Monitor_service[opensearch_shard_size_check_9600]
Motd::Script[cluster_memberships]
File[/etc/opensearch/production-search-psi-codfw]
File[/etc/logrotate.d/prometheus_ferm_mss]
Prometheus::Wmf_elasticsearch_exporter[cirrussearch2089:9600]
Systemd::Timer[nrpe2nodexp-opensearch_unassigned_shard_check_9600]
Class[Role::Cirrus::Opensearch]
Systemd::Unit[prometheus-wmf-elasticsearch-exporter-9200]
Cfssl::Csr[/etc/cfssl/csr/discovery__cirrussearch2089_codfw_wmnet.csr]
Systemd::Unit[nrpe2nodexp-opensearch_shards_9600.service]
File[/lib/systemd/system/opensearch-disable-readahead-production-search-codfw.service]
Service[nrpe2nodexp-opensearch_shard_size_check_9600.timer]
File_line[auto_restart_file_presence_prometheus-elasticsearch-exporter-9600]
Package[wikimedia-lvs-realserver]
File[/etc/sudoers.d/elasticsearch-roots]
Sudo::User[nrpe-check_check_tcp-mss-clamper_status]
Exec[systemd daemon-reload for prometheus-elasticsearch-exporter-9200.service (prometheus-elasticsearch-exporter-9200)]
File[/etc/opensearch/production-search-psi-codfw/logging.yml]
Class[Nginx]
Package[opensearch-madvise]
Class[Wmflib::Service::Conftool]
Systemd::Timer::Job[clean_up_dlq]
Class[Profile::Java]
Systemd::Service[prometheus-elasticsearch-exporter-9600]
Class[Lvs::Realserver]
File[/etc/opensearch/production-search-psi-codfw/log4j2.properties]
Systemd::Unit[opensearch-disable-readahead-production-search-psi-codfw.service]
File[/lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service]
Systemd::Timer[opensearch-production-search-codfw-gc-log-cleanup]
File[/var/log/opensearch]
Nrpe::Monitor_service[opensearch_unassigned_shard_check_9600]
Java::Cacert[wmf:puppetca.pem]
Interface::Clsact[clsact_lo]
Systemd::Service[nrpe2nodexp-opensearch_shards_9200]
File[/usr/local/lib/nagios/plugins/check_systemd_unit_status]
Exec[systemd daemon-reload for prometheus_lvs_realserver_mss.timer (prometheus_lvs_realserver_mss.timer)]
Service[prometheus_lvs_realserver_mss.timer]
Logrotate::Conf[nginx]
Systemd::Unit[nrpe2nodexp-opensearch_unassigned_shard_check_9600.service]
Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9600.service (nrpe2nodexp-opensearch_shard_size_check_9600.service)]
Sudo::User[nrpe-check_opensearch_shard_size_check_9200]
Class[Java::Tools]
Systemd::Timer::Job[nrpe2nodexp-opensearch_shards_9200]
Service[opensearch-production-search-psi-codfw-gc-log-cleanup.timer]
Systemd::Unit[opensearch-disable-readahead.service]
File[/var/log/wmf_auto_restart_prometheus-elasticsearch-exporter-9600]
File[/lib/systemd/system/opensearch-production-search-psi-codfw-gc-log-cleanup.timer]
Exec[ip link add name ipip60 type ip6tnl external]
Profile::Auto_restarts::Service[prometheus-elasticsearch-exporter-9200]
Admin::Hashuser[ebernhardson]
Service[opensearch-production-search-codfw-gc-log-cleanup.timer]
Systemd::Unit[prometheus_ferm_mss.timer]
Nrpe::Monitor_service[check_tcp-mss-clamper_status]
Nginx::Site[production-search-psi-codfw]
File[/etc/nginx]
File[/srv/opensearch/production-search-psi-codfw]
Service[opensearch-disable-readahead-production-search-codfw.timer]
Service[nginx]
Rsyslog::Conf[nrpe2nodexp-opensearch_shards_9200]
Class[Profile::Opensearch::Monitoring::Base_checks]
Package[python3-conftool]
Exec[/usr/sbin/dpkg-reconfigure -p critical -f noninteractive wikimedia-lvs-realserver]
Exec[systemd daemon-reload for wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service (wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service)]
File[/usr/local/bin/prometheus-wmf-elasticsearch-exporter]
File[/lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer]
File[/etc/systemd/system/nginx.service.d/security.conf]
Rsyslog::Conf[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]
File[/var/log/prometheus_ferm_mss]
Exec[ip link set up dev ipip0]
Exec[systemd daemon-reload for tcp-mss-clamper.service (tcp-mss-clamper)]
Nrpe::Monitor_service[opensearch_shard_size_check_9200]
File[/lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9200.service]
Systemd::Unit[opensearch_1@.service]
File[/etc/opensearch/production-search-codfw]
Prometheus::Node_ferm_mss[ferm_clamped_ipport]
File[/etc/ferm/conf.d/10_opensearch-http-9200]
Logrotate::Conf[opensearch-disable-readahead-production-search-psi-codfw]
File[/etc/logrotate.d/opensearch-disable-readahead]
Exec[mask_default_prometheus_elasticsearch_exporter]
File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.pem]
File[/etc/nginx/sites-enabled]
Exec[create chained cert /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chain.pem]
Logrotate::Conf[prometheus_ferm_mss]
File[/etc/sudoers.d/nrpe-check_opensearch_unassigned_shard_check_9600]
File[/etc/conftool/local_services.yaml]
Nginx::Site[production-search-codfw]
Service[nrpe2nodexp-check_tcp-mss-clamper_status.timer]
Ferm::Rule[clamp-mss-ipv4]
File[/var/log/opensearch-disable-readahead-production-search-psi-codfw]
Ferm::Service[opensearch-https-9243]
File[/etc/default/opensearch]
File[/etc/nagios/nrpe.d/check_opensearch_unassigned_shard_check_9600.cfg]
Package[python3-poolcounter]
Logrotate::Conf[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]
File[/var/log/opensearch/opensearch_index_search_slowlog.log]
Exec[systemd daemon-reload for prometheus_ferm_mss.service (prometheus_ferm_mss.service)]
Systemd::Unit[opensearch-disable-readahead-production-search-psi-codfw.timer]
File[/usr/lib/nagios/plugins/check_elasticsearch_shard_size.py]
File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-unassigned-shard-check-9600.conf]
File[/lib/systemd/system/nrpe2nodexp-opensearch_shards_9600.timer]
File[/etc/opensearch/production-search-codfw/logging.yml]
Rsyslog::Conf[prometheus_ferm_mss]
File[/root/.etcdrc]
Class[Profile::Lvs::Realserver]
File[/etc/ferm/conf.d/10_clamp-mss-ipv6]
Admin::Hashuser[pfischer]
File[/etc/opensearch/production-search-codfw/opensearch.keystore]
File[/lib/systemd/system/nrpe2nodexp-opensearch_shards_9600.service]
Systemd::Unit[prometheus_ferm_mss.service]
File[/etc/conftool]
Service[opensearch-disable-readahead.timer]
File[/usr/local/bin/safe-service-restart]
Exec[udev_reload]
Class[Logstash]
Motd::Script[cirrus::opensearch]
Exec[Refresh tmpfile opensearch-production-search-psi-codfw]
Logstash::Conf[output-udp-gelf_relay]
Systemd::Timer::Job[nrpe2nodexp-opensearch_shard_size_check_9200]
File[/etc/logrotate.d/clean_up_dlq]
File[/lib/systemd/system/nrpe2nodexp-opensearch_shards_9200.timer]
File[/etc/opensearch_hot_threads.d]
File[/usr/local/bin/pooler-loop]
Monitoring::Service[opensearch_shards_9200]
File[/etc/nginx/nginx.conf]
Exec[Generate cert discovery__cirrussearch2089_codfw_wmnet refresh]
File[/usr/local/bin/depool-opensearch_1@production-search-codfw]
File[/etc/logrotate.d/opensearch-disable-readahead-production-search-psi-codfw]
Systemd::Unit[opensearch-production-search-codfw-gc-log-cleanup.timer]
Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9600.timer (nrpe2nodexp-opensearch_shards_9600.timer)]
Rsyslog::Conf[nrpe2nodexp-opensearch_shards_9600]
Monitoring::Service[opensearch_unassigned_shard_check_9200]
Prometheus::Wmf_elasticsearch_exporter[cirrussearch2089:9200]
File[/etc/systemd/system/nginx.service.d]
File[/lib/systemd/system/prometheus-wmf-elasticsearch-exporter-9200.service]
Systemd::Unit[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service]
Group[elasticsearch-roots]
File[/etc/etcd]
Exec[systemd daemon-reload for opensearch-disable-readahead-production-search-codfw.timer (opensearch-disable-readahead-production-search-codfw.timer)]
Exec[systemd daemon-reload for clean_up_dlq.service (clean_up_dlq.service)]
Systemd::Unit[nrpe2nodexp-check_tcp-mss-clamper_status.timer]
Rsyslog::Conf[prometheus_lvs_realserver_mss]
Systemd::Timer[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]
Ferm::Rule[ipip]
Service[nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer]
Exec[/usr/sbin/tc qdisc del dev eno12399np0 clsact]
Systemd::Unit[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer]
Class[Prometheus::Wmf_elasticsearch_exporter::Common]
File[/usr/local/bin/logstash-config-test]
Rsyslog::Conf[opensearch-production-search-codfw-gc-log-cleanup]
Systemd::Timer::Job[prometheus_ferm_mss]
Opensearch::Log::Hot_threads_cluster[production-search-codfw]
Opensearch::Keystore[s3-credentials-user-production-search-codfw]
Ferm::Service[opensearch-https-9643]
Systemd::Timer::Job[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]
Service[nrpe2nodexp-opensearch_shards_9600.timer]
File[/etc/udev/rules.d/40-opensearch-readahead.rules]
Systemd::Syslog[wmf_auto_restart_prometheus-elasticsearch-exporter-9200]
File[/lib/systemd/system/opensearch-hot-threads-log.service]
Exec[disable-rp-filter-eno12399np0]
Admin::Hashgroup[elasticsearch-roots]
Systemd::Syslog[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]
File[/lib/systemd/system/nrpe2nodexp-opensearch_shards_9200.service]
File[/usr/lib/nagios/plugins/check_elasticsearch.py]
Rsyslog::Conf[wmf_auto_restart_prometheus-elasticsearch-exporter-9600]
Class[Opensearch::Log::Hot_threads]
Systemd::Service[prometheus-wmf-elasticsearch-exporter-9600]
File[/etc/logstash/jvm.options]
File[/lib/systemd/system/opensearch-production-search-codfw-gc-log-cleanup.timer]
File[/etc/opensearch/production-search-codfw/opensearch.yml]
Exec[systemd daemon-reload for prometheus_ferm_mss.timer (prometheus_ferm_mss.timer)]
Systemd::Unit[nrpe2nodexp-opensearch_shards_9200.timer]
File[/etc/opensearch_hot_threads.d/production-search-psi-codfw.yml]
Systemd::Timer[clean_up_dlq]
Rsyslog::Conf[udp_json_logback_compat]
File[/etc/nginx/sites-enabled/production-search-codfw]
File[/etc/opensearch/production-search-codfw/java-security.policy]
Exec[systemd daemon-reload for opensearch-disable-readahead-production-search-psi-codfw.timer (opensearch-disable-readahead-production-search-psi-codfw.timer)]
File[/etc/opensearch]
Cfssl::Cert[discovery__cirrussearch2089_codfw_wmnet]
File[/etc/sudoers.d/nrpe-check_opensearch_unassigned_shard_check_9200]
Exec[disable-rp-filter-ipip0]
Package[python3-requests]
Service[nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer]
File[/etc/rsyslog.d/40-wmf-auto-restart-prometheus-elasticsearch-exporter-9200.conf]
Systemd::Service[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]
File[/etc/poolcounter-backends.yaml]
File[/usr/local/bin/pool-opensearch_1@production-search-codfw]
File[/etc/rsyslog.d/40-opensearch-production-search-psi-codfw-gc-log-cleanup.conf]
File[/var/log/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]
File[/var/log/prometheus_lvs_realserver_mss]
File[/home/dr0ptp4kt]
File[/etc/rsyslog.d/50-udp-json-logback-compat.conf]
Exec[s3-credentials-user-production-search-codfw]
Nrpe::Monitor_service[opensearch_shards_9600]
File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chain.pem]
Logrotate::Conf[opensearch-disable-readahead-production-search-codfw]
Systemd::Unit[tcp-mss-clamper]
File[/lib/systemd/system/prometheus_ferm_mss.service]
File[/var/log/opensearch-disable-readahead-production-search-codfw]
Opensearch::Instance[production-search-psi-codfw]
Exec[opensearch-create-keystore-production-search-psi-codfw]
File[/etc/opensearch/production-search-psi-codfw/opensearch.keystore]
File[/home/ebernhardson]
Augeas[ipip0_127.0.0.42/32]
Tlsproxy::Localssl[production-search-psi-codfw]
Class[Profile::Conftool::Client]
File[/etc/logstash/conf.d/README]
Systemd::Unit[prometheus-elasticsearch-exporter-9600]
File[/etc/logrotate.d/wmf_auto_restart_prometheus-elasticsearch-exporter-9600]
File[/etc/nagios/nrpe.d/check_opensearch_shards_9600.cfg]
Interface::Post_up_command[clsact_lo]
File[/etc/logrotate.d/prometheus_lvs_realserver_mss]
File[/usr/local/bin/depool-opensearch_1@production-search-psi-codfw]
File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-shards-9200.conf]
File[/etc/ferm/conf.d/10_clamp-mss-ipv4]
Logrotate::Conf[clean_up_dlq]
Systemd::Timer::Job[opensearch-production-search-psi-codfw-gc-log-cleanup]
Augeas[ipip0_manual]
Conftool::Scripts::Safe_service_restart[opensearch_1@production-search-codfw]
Class[Profile::Tlsproxy::Instance]
Systemd::Unit[prometheus_lvs_realserver_mss.service]
Exec[Generate cert discovery__cirrussearch2089_codfw_wmnet]
File[/lib/systemd/system/opensearch-disable-readahead.service]
Class[Udev]
File[/lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9600.service]
File[/etc/default/logstash]
Systemd::Timer[nrpe2nodexp-check_tcp-mss-clamper_status]
Rsyslog::Conf[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]
Systemd::Unit[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service]
Exec[/usr/sbin/tc qdisc del dev lo clsact]
Exec[disable-rp-filter-ipip60]
File[/usr/local/bin/decommission]
Prometheus::Elasticsearch_exporter[localhost:9200]
Exec[systemd daemon-reload for prometheus_lvs_realserver_mss.service (prometheus_lvs_realserver_mss.service)]
File[/etc/rsyslog.d/40-opensearch-disable-readahead-production-search-psi-codfw.conf]
Admin::Hashuser[dcausse]
Motd::Message[cirrus::opensearch]
User[dr0ptp4kt]
File[/etc/default/wikimedia-lvs-realserver]
Mount[/var/lib/nginx]
File[/etc/ferm/conf.d/10_opensearch-https-9243]
File[/usr/local/bin/prometheus-ferm-mss]
Systemd::Unit[nrpe2nodexp-opensearch_shards_9200.service]
Logstash::Input::Gelf[gelf_relay]
Sudo::User[nrpe-check_opensearch_unassigned_shard_check_9600]
Ferm::Rule[ip6ip6]
Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer (nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer)]
Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_shard_size_check_9600]
Interface::Ipip[ipip_ipv6]
File[/var/log/opensearch-production-search-codfw-gc-log-cleanup]
Exec[symlink /etc/opensearch/production-search-psi-codfw/sudachi]
Systemd::Timer[nrpe2nodexp-opensearch_unassigned_shard_check_9200]
Systemd::Monitor[tcp-mss-clamper]
Systemd::Unit[opensearch-disable-readahead.timer]
File[/lib/systemd/system/opensearch-production-search-psi-codfw-gc-log-cleanup.service]
Apt::Repository[wikimedia-opensearch-plugins]
Ferm::Rule[clamp-mss-ipv6]
File[/etc/update-motd.d/05-cirrus--opensearch]
File[/lib/systemd/system/opensearch-disable-readahead-production-search-codfw.timer]
File[/etc/logstash/conf.d/90-output-udp-gelf_relay.conf]
File[/etc/etcd/etcdrc]
Systemd::Unit[wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer]
Logstash::Conf[input-gelf-gelf_relay]
Systemd::Timer[wmf_auto_restart_prometheus-elasticsearch-exporter-9200]
Systemd::Timer[opensearch-disable-readahead-production-search-psi-codfw]
Opensearch::Log::Hot_threads_cluster[production-search-psi-codfw]
File[/etc/opensearch_hot_threads.d/production-search-codfw.yml]
File[/lib/systemd/system/prometheus_ferm_mss.timer]
Systemd::Unit[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer]
Interface::Manual[ipip_ipv6]
Exec[opensearch-create-keystore-production-search-codfw]
Prometheus::Alert::Rule[check_opensearch_unassigned_shard_check_9200_f98d7b74d9467ea0aa193a52c18e30ab]
File[/etc/logstash/logstash.yml]
Exec[systemd daemon-reload for prometheus-wmf-elasticsearch-exporter-9200.service (prometheus-wmf-elasticsearch-exporter-9200)]
File[/lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer]
Systemd::Unit[nrpe2nodexp-opensearch_shards_9600.timer]
Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_unassigned_shard_check_9600]
Exec[stop-default-nginx]
Prometheus::Alert::Rule[check_opensearch_shards_9600_7811fe11f3315f2bd38a6e6de0f1b6b0]
File_line[opensearch_bin_bashrc]
Sudo::User[nrpe-check_opensearch_shards_9200]
Rsyslog::Conf[nrpe2nodexp-opensearch_shard_size_check_9600]
Exec[systemd daemon-reload for nrpe2nodexp-check_tcp-mss-clamper_status.timer (nrpe2nodexp-check_tcp-mss-clamper_status.timer)]

Resources only in the old catalog

File[/etc/update-motd.d/05-insetup--data-platform-ferm]
Node[__node_regexp__cirrussearch2089.codfw.]
Motd::Message[insetup::data_platform_ferm]
Motd::Script[insetup::data_platform_ferm]
Class[Role::Insetup::Data_platform_ferm]

Resources modified

File[/lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer]

Parameters differences:

--- File[/lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer].orig
+++ File[/lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer (wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer.orig
+++ /lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer
@@ -0,0 +1,12 @@
+[Unit]
+Description=Periodic execution of wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service
+
+[Timer]
+Unit=wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service
+# Accuracy sets the maximum time interval around the execution time we want to allow
+AccuracySec=15sec
+OnCalendar=Mon,Tue,Wed,Thu,Fri *-*-* 9:55:00
+RandomizedDelaySec=0
+
+[Install]
+WantedBy=multi-user.target

File[/etc/logrotate.d/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]

Parameters differences:

--- File[/etc/logrotate.d/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600].orig
+++ File[/etc/logrotate.d/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/logrotate.d/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.orig
+++ /etc/logrotate.d/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600
@@ -0,0 +1,12 @@
+# logrotate(8) config for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600
+
+/var/log/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600/*.log {
+    daily
+    copytruncate
+    missingok
+    compress
+    delaycompress
+    notifempty
+    rotate 15
+    size 256M
+}

File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet]

Parameters differences:

--- File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet].orig
+++ File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet]

+    mode    => 0740
+    group   => root
+    owner   => root
+    recurse => True
+    ensure  => directory

Prometheus::Alert::Rule[check_opensearch_shard_size_check_9600_6ce4fe694f5df8cba9e0032196c72747]

Parameters differences:

--- Prometheus::Alert::Rule[check_opensearch_shard_size_check_9600_6ce4fe694f5df8cba9e0032196c72747].orig
+++ Prometheus::Alert::Rule[check_opensearch_shard_size_check_9600_6ce4fe694f5df8cba9e0032196c72747]

+    dashboard          => TODO
+    expr               => (nagios_nrpe_check_result{alert_rule_hash="6ce4fe694f5df8cba9e0032196c72747",check_name="check_opensearch_shard_size_check_9600", status=~"(WARNING|CRITICAL)", severity=~"(warning|critical)"} > 0) * on (instance) group_left (team) role_owner
+    team               => observability
+    description        => OpenSearch shard size check - 9600
+    runbook            => https://wikitech.wikimedia.org/wiki/Search#If_it_has_been_indexed
+    def_label_whitelst => ['team', 'severity']
+    ensure             => absent
+    logs               => https://logstash.wikimedia.org/app/dashboards#/view/2d343ac0-6df8-11f0-8e08-7fab0da52b33?_g=(filters:!((query:(match_phrase:(event.module:check_opensearch_shard_size_check_9600))),(query:(match_phrase:(host.name:{{$labels.instance|stripPort}})))))
+    severity           => info
+    summary            => OpenSearch shard size check - 9600
+    instance           => ops
+    group              => nrpechecks
+    alert_name         => check_opensearch_shard_size_check_9600
+    for                => 259200m
+    site               => codfw

Monitoring::Service[opensearch_unassigned_shard_check_9600]

Parameters differences:

--- Monitoring::Service[opensearch_unassigned_shard_check_9600].orig
+++ Monitoring::Service[opensearch_unassigned_shard_check_9600]

+    check_command  => nrpe_check!check_opensearch_unassigned_shard_check_9600!10
+    retry_interval => 120
+    description    => OpenSearch unassigned shard check - 9600
+    critical       => False
+    notes_url      => https://wikitech.wikimedia.org/wiki/Search#Administration
+    retries        => 1
+    config_dir     => /etc/nagios
+    freshness      => 36000
+    migration_task => T321808
+    ensure         => present
+    check_interval => 720
+    contact_group  => admins,team-discovery
+    host           => cirrussearch2089
+    passive        => False

File[/etc/nagios/nrpe.d/check_check_tcp-mss-clamper_status.cfg]

Parameters differences:

--- File[/etc/nagios/nrpe.d/check_check_tcp-mss-clamper_status.cfg].orig
+++ File[/etc/nagios/nrpe.d/check_check_tcp-mss-clamper_status.cfg]

+    mode    => 0444
+    require => Package[nagios-nrpe-server]
+    tag     => nrpe::check
+    notify  => Service[nagios-nrpe-server]
+    group   => root
+    owner   => root
+    ensure  => absent

Content differences:

--- /etc/nagios/nrpe.d/check_check_tcp-mss-clamper_status.cfg.orig
+++ /etc/nagios/nrpe.d/check_check_tcp-mss-clamper_status.cfg
@@ -0,0 +1,2 @@
+# File generated by puppet. DO NOT edit by hand
+command[check_check_tcp-mss-clamper_status]=/usr/local/lib/nagios/plugins/check_systemd_unit_status tcp-mss-clamper

Exec[systemd daemon-reload for opensearch-production-search-psi-codfw-gc-log-cleanup.timer (opensearch-production-search-psi-codfw-gc-log-cleanup.timer)]

Parameters differences:

--- Exec[systemd daemon-reload for opensearch-production-search-psi-codfw-gc-log-cleanup.timer (opensearch-production-search-psi-codfw-gc-log-cleanup.timer)].orig
+++ Exec[systemd daemon-reload for opensearch-production-search-psi-codfw-gc-log-cleanup.timer (opensearch-production-search-psi-codfw-gc-log-cleanup.timer)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload
+    before      => ['Service[opensearch-production-search-psi-codfw-gc-log-cleanup.timer]']

Elasticsearch::Tlsproxy[production-search-codfw]

Parameters differences:

--- Elasticsearch::Tlsproxy[production-search-codfw].orig
+++ Elasticsearch::Tlsproxy[production-search-codfw]

+    acme_chief        => False
+    upstream_port     => 9200
+    tls_port          => 9243
+    server_aliases    => ['search.discovery.wmnet', 'search-chi.discovery.wmnet', 'search-omega.discovery.wmnet', 'search-psi.discovery.wmnet', 'search.svc.codfw.wmnet', 'search-chi.svc.codfw.wmnet', 'search-omega.svc.codfw.wmnet', 'search-psi.svc.codfw.wmnet']
+    enable_http2      => False
+    read_only         => False
+    server_name       => cirrussearch2089.codfw.wmnet
+    certificate_names => []
+    cfssl_paths       => {'cert': '/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.pem', 'key': '/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet-key.pem', 'chain': '/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chain.pem', 'chained': '/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chained.pem'}

File[/var/log/opensearch-disable-readahead]

Parameters differences:

--- File[/var/log/opensearch-disable-readahead].orig
+++ File[/var/log/opensearch-disable-readahead]

+    mode   => 0755
+    backup => False
+    force  => True
+    owner  => root
+    ensure => absent
+    group  => root

Monitoring::Service[check_tcp-mss-clamper_status]

Parameters differences:

--- Monitoring::Service[check_tcp-mss-clamper_status].orig
+++ Monitoring::Service[check_tcp-mss-clamper_status]

+    check_command  => nrpe_check!check_check_tcp-mss-clamper_status!10
+    retry_interval => 1
+    description    => Check unit status of tcp-mss-clamper
+    critical       => False
+    notes_url      => https://wikitech.wikimedia.org/wiki/LVS#IPIP_encapsulation_experiments
+    retries        => 2
+    config_dir     => /etc/nagios
+    freshness      => 36000
+    migration_task => T321808
+    ensure         => absent
+    check_interval => 10
+    contact_group  => admins
+    host           => cirrussearch2089
+    passive        => False

File[/etc/ssl/dhparam.pem]

Parameters differences:

--- File[/etc/ssl/dhparam.pem].orig
+++ File[/etc/ssl/dhparam.pem]

+    mode   => 0444
+    source => puppet:///modules/sslcert/dhparam.pem
+    group  => root
+    owner  => root
+    ensure => present

Rsyslog::Conf[nrpe2nodexp-opensearch_unassigned_shard_check_9200]

Parameters differences:

--- Rsyslog::Conf[nrpe2nodexp-opensearch_unassigned_shard_check_9200].orig
+++ Rsyslog::Conf[nrpe2nodexp-opensearch_unassigned_shard_check_9200]

+    mode     => 0444
+    priority => 25
+    ensure   => absent

File[/etc/logrotate.d/nginx]

Parameters differences:

--- File[/etc/logrotate.d/nginx].orig
+++ File[/etc/logrotate.d/nginx]

+    mode   => 0444
+    source => puppet:///modules/profile/tlsproxy/logrotate
+    group  => root
+    before => ['Service[nginx]']
+    owner  => root
+    ensure => present

User[dcausse]

Parameters differences:

--- User[dcausse].orig
+++ User[dcausse]

+    allowdupe  => False
+    uid        => 12338
+    shell      => /bin/bash
+    gid        => 500
+    ensure     => present
+    home       => /home/dcausse
+    groups     => []
+    comment    => David Causse
+    managehome => False

File[/etc/rsyslog.d/40-prometheus-ferm-mss.conf]

Parameters differences:

--- File[/etc/rsyslog.d/40-prometheus-ferm-mss.conf].orig
+++ File[/etc/rsyslog.d/40-prometheus-ferm-mss.conf]

+    mode   => 0444
+    notify => Service[rsyslog]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/rsyslog.d/40-prometheus-ferm-mss.conf.orig
+++ /etc/rsyslog.d/40-prometheus-ferm-mss.conf
@@ -0,0 +1,10 @@
+# rsyslog.conf(5) configuration file for services.
+# This file is managed by Puppet.
+if $programname startswith "prometheus_ferm_mss" then {
+    action(
+        type="omfile" file="/var/log/prometheus_ferm_mss/syslog.log"
+        fileOwner="root" fileGroup="root"
+        fileCreateMode="0644"
+    )
+    & stop
+}

Systemd::Timer[opensearch-production-search-psi-codfw-gc-log-cleanup]

Parameters differences:

--- Systemd::Timer[opensearch-production-search-psi-codfw-gc-log-cleanup].orig
+++ Systemd::Timer[opensearch-production-search-psi-codfw-gc-log-cleanup]

+    fixed_random_delay => False
+    unit_name          => opensearch-production-search-psi-codfw-gc-log-cleanup.service
+    timer_intervals    => [{'start': 'OnCalendar', 'interval': '*-*-* 02:12:00'}]
+    splay              => 0
+    accuracy           => 15sec
+    ensure             => present

Class[Conftool::Config]

Parameters differences:

--- Class[Conftool::Config].orig
+++ Class[Conftool::Config]

+    tcpircbot_port       => 9200
+    namespace            => /conftool
+    tcpircbot_host       => icinga.wikimedia.org
+    conftool2git_address => puppetserver1003.eqiad.wmnet:1312
+    hosts                => []

Interface::Ipip[ipip_ipv4]

Parameters differences:

--- Interface::Ipip[ipip_ipv4].orig
+++ Interface::Ipip[ipip_ipv4]

+    family    => inet
+    address   => 127.0.0.42
+    interface => ipip0
+    ensure    => present

Admin::User[ebernhardson]

Parameters differences:

--- Admin::User[ebernhardson].orig
+++ Admin::User[ebernhardson]

+    uid      => 3088
+    groups   => []
+    gid      => 500
+    ensure   => present
+    ssh_keys => ['ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCq5sClypKI1N1T1ZfR2ZZPbWH6gXPrQ9a3HRiFgWpcp+W4YIYT4taNNqd+gFO6aF7UyAS9148ROxivCksSK+MmfULsIz0cD4l8R6haR939kfK5MRC2yONusy8IUkQgPvkBWkIvOhrURNs2sPkxYVwEyHIOUX3Izf1cLjVZenz4lhRqPtstHMZQaa3TGAoCFcymOln/64j5vq9W8MljLBsEWpPdeUwmf6mD0EHb5LUA+2aVB1JwfmViY5Dm77FBh5CDHqdvMWa8ZF0ZybtUAuL0wpcO0h/H1WDhqS5j1Q4AemxtOeRnBJoCsh5Wlf4J4sZptMM2hvdNYnqWXb+3Dto7 ebernhardson@wikimedia.org']
+    home_dir => /home/ebernhardson
+    comment  => Erik Bernhardson
+    shell    => /bin/bash

Exec[systemd daemon-reload for clean_up_dlq.timer (clean_up_dlq.timer)]

Parameters differences:

--- Exec[systemd daemon-reload for clean_up_dlq.timer (clean_up_dlq.timer)].orig
+++ Exec[systemd daemon-reload for clean_up_dlq.timer (clean_up_dlq.timer)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

Systemd::Timer::Job[prometheus_lvs_realserver_mss]

Parameters differences:

--- Systemd::Timer::Job[prometheus_lvs_realserver_mss].orig
+++ Systemd::Timer::Job[prometheus_lvs_realserver_mss]

+    description               => Regular job to collect MSS values of realserver endpoints
+    syslog_match_startswith   => True
+    monitoring_contact_groups => admins
+    logging_enabled           => True
+    success_exit_status       => []
+    fixed_random_delay        => False
+    private_tmp               => False
+    logfile_basedir           => /var/log
+    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+    send_mail_only_on_error   => True
+    ignore_errors             => False
+    logfile_group             => root
+    interval                  => {'start': 'OnCalendar', 'interval': 'minutely'}
+    send_mail                 => False
+    logfile_name              => syslog.log
+    user                      => root
+    ensure                    => present
+    send_mail_to              => root@cirrussearch2089.codfw.wmnet
+    logfile_perms             => all
+    syslog_force_stop         => True
+    environment               => {}
+    command                   => /usr/local/bin/prometheus-lvs-realserver-mss -o /var/lib/prometheus/node.d/lvs-realserver-mss.prom -e 10.2.1.30:9200 -e 10.2.1.30:9243 -e 10.2.1.30:9643
+    monitoring_enabled        => False

Service[opensearch-hot-threads-log.timer]

Parameters differences:

--- Service[opensearch-hot-threads-log.timer].orig
+++ Service[opensearch-hot-threads-log.timer]

+    provider => systemd
+    enable   => True
+    ensure   => running

File[/usr/local/bin/depool-nginx]

Parameters differences:

--- File[/usr/local/bin/depool-nginx].orig
+++ File[/usr/local/bin/depool-nginx]

+    mode   => 0555
+    group  => root
+    before => ['Service[nginx]']
+    owner  => root
+    ensure => present

Content differences:

--- /usr/local/bin/depool-nginx.orig
+++ /usr/local/bin/depool-nginx
@@ -0,0 +1,2 @@
+#!/bin/bash
+/usr/local/bin/safe-service-restart --pools search-https search-psi-https --depool --retries 10 --wait 5

Class[Admin]

Parameters differences:

--- Class[Admin].orig
+++ Class[Admin]

@@
-    groups => []
+    groups => ['elasticsearch-roots']

File[/etc/sudoers.d/nrpe-check_check_tcp-mss-clamper_status]

Parameters differences:

--- File[/etc/sudoers.d/nrpe-check_check_tcp-mss-clamper_status].orig
+++ File[/etc/sudoers.d/nrpe-check_check_tcp-mss-clamper_status]

+    group   => root
+    require => Package[nagios-nrpe-server]
+    owner   => root
+    ensure  => absent

Monitoring::Service[elasticsearch-https-production-search-psi-codfw]

Parameters differences:

--- Monitoring::Service[elasticsearch-https-production-search-psi-codfw].orig
+++ Monitoring::Service[elasticsearch-https-production-search-psi-codfw]

+    check_command  => check_ssl_on_port!cirrussearch2089.codfw.wmnet!9643
+    retry_interval => 1
+    notes_url      => https://wikitech.wikimedia.org/wiki/Search
+    migration_task => T384998
+    description    => Elasticsearch HTTPS for production-search-psi-codfw
+    retries        => 3
+    critical       => False
+    freshness      => 36000
+    config_dir     => /etc/nagios
+    ensure         => present
+    check_interval => 1
+    host           => cirrussearch2089
+    passive        => False

Ssh::Userkey[dr0ptp4kt]

Parameters differences:

--- Ssh::Userkey[dr0ptp4kt].orig
+++ Ssh::Userkey[dr0ptp4kt]

+    user   => dr0ptp4kt
+    ensure => present

Systemd::Unit[nrpe2nodexp-check_tcp-mss-clamper_status.service]

Parameters differences:

--- Systemd::Unit[nrpe2nodexp-check_tcp-mss-clamper_status.service].orig
+++ Systemd::Unit[nrpe2nodexp-check_tcp-mss-clamper_status.service]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => nrpe2nodexp-check_tcp-mss-clamper_status.service
+    ensure            => absent

Class[Profile::Apt]

Parameters differences:

--- Class[Profile::Apt].orig
+++ Class[Profile::Apt]

@@
-    before => ['Package[puppet]', 'Package[facter]', 'Package[augeas-tools]', 'Package[virt-what]', 'Package[puppet-module-puppetlabs-augeas-core]', 'Package[python3-prometheus-client]', 'Package[python3-yaml]', 'Package[ruby-net-ssh]', 'Package[openssl]', 'Package[ssl-cert]', 'Package[ca-certificates]', 'Package[wmf-certificates]', 'Package[ntp]', 'Package[systemd-timesyncd]', 'Package[exim4-config]', 'Package[exim4-daemon-light]', 'Package[logrotate]', 'Package[prometheus-node-exporter]', 'Package[bsdutils]', 'Package[smartmontools]', 'Package[rsyslog]', 'Package[rsyslog-openssl]', 'Package[cadvisor]', 'Package[acct]', 'Package[byobu]', 'Package[colordiff]', 'Package[curl]', 'Package[debian-goodies]', 'Package[ethtool]', 'Package[gdb]', 'Package[gdisk]', 'Package[git]', 'Package[htop]', 'Package[httpry]', 'Package[iotop]', 'Package[iperf]', 'Package[jq]', 'Package[libtemplate-perl]', 'Package[lldpd]', 'Package[lshw]', 'Package[molly-guard]', 'Package[moreutils]', 'Package[net-tools]', 'Package[numactl]', 'Package[ncdu]', 'Package[ngrep]', 'Package[pigz]', 'Package[psmisc]', 'Package[pv]', 'Package[python3]', 'Package[screen]', 'Package[strace]', 'Package[sysstat]', 'Package[tcpdump]', 'Package[tmux]', 'Package[tree]', 'Package[vim]', 'Package[vim-addon-manager]', 'Package[vim-scripts]', 'Package[wipe]', 'Package[xfsprogs]', 'Package[zsh]', 'Package[icdiff]', 'Package[linux-perf]', 'Package[bsd-mailx]', 'Package[ack]', 'Package[netcat-openbsd]', 'Package[tshark]', 'Package[fzf]', 'Package[ripgrep]', 'Package[fd-find]', 'Package[kitty-terminfo]', 'Package[mtr-tiny]', 'Package[tzdata]', 'Package[python3-wmflib]', 'Package[quickstack]', 'Package[bind9-dnsutils]', 'Package[dstat]', 'Package[apport]', 'Package[command-not-found]', 'Package[command-not-found-data]', 'Package[ecryptfs-utils]', 'Package[mlocate]', 'Package[os-prober]', 'Package[python3-apport]', 'Package[wpasupplicant]', 'Package[atop]', 'Package[apt-listchanges]', 'Package[isc-dhcp-client]', 'Package[libpython2.7]', 'Package[libpython2.7-dev]', 'Package[libpython2.7-minimal]', 'Package[python2.7]', 'Package[libpython2.7-stdlib]', 'Package[python2.7-dev]', 'Package[python2.7-minimal]', 'Package[python2.7-dbg]', 'Package[python2.7-doc]', 'Package[python2.7-examples]', 'Package[libpython2.7-testsuite]', 'Package[intel-microcode]', 'Package[rasdaemon]', 'Package[libsnmp30]', 'Package[libdns-export1104]', 'Package[libdns1104]', 'Package[libisc-export1100]', 'Package[libisc1100]', 'Package[multiarch-support]', 'Package[libjson-c3]', 'Package[libpython3.7]', 'Package[libpython3.7-minimal]', 'Package[libpython3.7-stdlib]', 'Package[python3.7]', 'Package[python3.7-minimal]', 'Package[libevent-2.1-6]', 'Package[libwireshark11]', 'Package[libwiretap8]', 'Package[libwsutil9]', 'Package[libwscodecs2]', 'Package[libperl5.28]', 'Package[libmpdec2]', 'Package[perl-modules-5.28]', 'Package[libhogweed4]', 'Package[libnettle6]', 'Package[libprocps7]', 'Package[libip6tc0]', 'Package[libip4tc0]', 'Package[libiptc0]', 'Package[openssh-client]', 'Package[openssh-server]', 'Package[debdeploy-client]', 'Package[python3-dateutil]', 'Package[sudo]', 'Package[golang-cfssl]', 'Package[debmonitor-client]', 'Package[perccli]', 'Package[nagios-nrpe-server]', 'Package[monitoring-plugins]', 'Package[monitoring-plugins-basic]', 'Package[monitoring-plugins-standard]', 'Package[liburiparser1]', 'Package[iucode-tool]', 'Package[freeipmi-tools]', 'Package[freeipmi-ipmiseld]', 'Package[rsyslog-kafka]', 'Package[emacs-nox]', 'Package[prometheus-ipmi-exporter]', 'Package[libnet-dns-perl]', 'Package[iptables]', 'Package[ferm]', 'Package[ulogd2]', 'Package[conntrack]', 'Package[ruby-sys-filesystem]', 'Package[prometheus-rsyslog-exporter]', 'Package[python3-click]', 'Package[python3-box]', 'Package[initramfs-tools]', 'Package[confd]', 'Package[python3-toml]']
+    before => ['Package[puppet]', 'Package[facter]', 'Package[augeas-tools]', 'Package[virt-what]', 'Package[puppet-module-puppetlabs-augeas-core]', 'Package[python3-prometheus-client]', 'Package[python3-yaml]', 'Package[ruby-net-ssh]', 'Package[openssl]', 'Package[ssl-cert]', 'Package[ca-certificates]', 'Package[wmf-certificates]', 'Package[ntp]', 'Package[systemd-timesyncd]', 'Package[exim4-config]', 'Package[exim4-daemon-light]', 'Package[logrotate]', 'Package[prometheus-node-exporter]', 'Package[bsdutils]', 'Package[smartmontools]', 'Package[rsyslog]', 'Package[rsyslog-openssl]', 'Package[cadvisor]', 'Package[acct]', 'Package[byobu]', 'Package[colordiff]', 'Package[curl]', 'Package[debian-goodies]', 'Package[ethtool]', 'Package[gdb]', 'Package[gdisk]', 'Package[git]', 'Package[htop]', 'Package[httpry]', 'Package[iotop]', 'Package[iperf]', 'Package[jq]', 'Package[libtemplate-perl]', 'Package[lldpd]', 'Package[lshw]', 'Package[molly-guard]', 'Package[moreutils]', 'Package[net-tools]', 'Package[numactl]', 'Package[ncdu]', 'Package[ngrep]', 'Package[pigz]', 'Package[psmisc]', 'Package[pv]', 'Package[python3]', 'Package[screen]', 'Package[strace]', 'Package[sysstat]', 'Package[tcpdump]', 'Package[tmux]', 'Package[tree]', 'Package[vim]', 'Package[vim-addon-manager]', 'Package[vim-scripts]', 'Package[wipe]', 'Package[xfsprogs]', 'Package[zsh]', 'Package[icdiff]', 'Package[linux-perf]', 'Package[bsd-mailx]', 'Package[ack]', 'Package[netcat-openbsd]', 'Package[tshark]', 'Package[fzf]', 'Package[ripgrep]', 'Package[fd-find]', 'Package[kitty-terminfo]', 'Package[mtr-tiny]', 'Package[tzdata]', 'Package[python3-wmflib]', 'Package[quickstack]', 'Package[bind9-dnsutils]', 'Package[dstat]', 'Package[apport]', 'Package[command-not-found]', 'Package[command-not-found-data]', 'Package[ecryptfs-utils]', 'Package[mlocate]', 'Package[os-prober]', 'Package[python3-apport]', 'Package[wpasupplicant]', 'Package[atop]', 'Package[apt-listchanges]', 'Package[isc-dhcp-client]', 'Package[libpython2.7]', 'Package[libpython2.7-dev]', 'Package[libpython2.7-minimal]', 'Package[python2.7]', 'Package[libpython2.7-stdlib]', 'Package[python2.7-dev]', 'Package[python2.7-minimal]', 'Package[python2.7-dbg]', 'Package[python2.7-doc]', 'Package[python2.7-examples]', 'Package[libpython2.7-testsuite]', 'Package[intel-microcode]', 'Package[rasdaemon]', 'Package[libsnmp30]', 'Package[libdns-export1104]', 'Package[libdns1104]', 'Package[libisc-export1100]', 'Package[libisc1100]', 'Package[multiarch-support]', 'Package[libjson-c3]', 'Package[libpython3.7]', 'Package[libpython3.7-minimal]', 'Package[libpython3.7-stdlib]', 'Package[python3.7]', 'Package[python3.7-minimal]', 'Package[libevent-2.1-6]', 'Package[libwireshark11]', 'Package[libwiretap8]', 'Package[libwsutil9]', 'Package[libwscodecs2]', 'Package[libperl5.28]', 'Package[libmpdec2]', 'Package[perl-modules-5.28]', 'Package[libhogweed4]', 'Package[libnettle6]', 'Package[libprocps7]', 'Package[libip6tc0]', 'Package[libip4tc0]', 'Package[libiptc0]', 'Package[openssh-client]', 'Package[openssh-server]', 'Package[debdeploy-client]', 'Package[python3-dateutil]', 'Package[sudo]', 'Package[golang-cfssl]', 'Package[debmonitor-client]', 'Package[perccli]', 'Package[nagios-nrpe-server]', 'Package[monitoring-plugins]', 'Package[monitoring-plugins-basic]', 'Package[monitoring-plugins-standard]', 'Package[liburiparser1]', 'Package[iucode-tool]', 'Package[freeipmi-tools]', 'Package[freeipmi-ipmiseld]', 'Package[rsyslog-kafka]', 'Package[emacs-nox]', 'Package[prometheus-ipmi-exporter]', 'Package[libnet-dns-perl]', 'Package[iptables]', 'Package[ferm]', 'Package[ulogd2]', 'Package[conntrack]', 'Package[wikimedia-lvs-realserver]', 'Package[python3-conftool]', 'Package[python3-poolcounter]', 'Package[tcp-mss-clamper]', 'Package[edac-utils]', 'Package[opensearch]', 'Package[python3-requests]', 'Package[wmf-opensearch-search-plugins]', 'Package[opensearch-madvise]', 'Package[logstash]', 'Package[logstash-old-name]', 'Package[ruby-sys-filesystem]', 'Package[prometheus-rsyslog-exporter]', 'Package[python3-click]', 'Package[python3-box]', 'Package[initramfs-tools]', 'Package[confd]', 'Package[python3-toml]', 'Package[openjdk-11-jdk]', 'Package[prometheus-elasticsearch-exporter]', 'Package[nginx-light]', 'Package[nginx-common]']

User[pfischer]

Parameters differences:

--- User[pfischer].orig
+++ User[pfischer]

+    allowdupe  => False
+    uid        => 41223
+    shell      => /bin/bash
+    gid        => 500
+    ensure     => present
+    home       => /home/pfischer
+    groups     => []
+    comment    => Peter Fischer
+    managehome => False

Systemd::Timer::Job[opensearch-production-search-codfw-gc-log-cleanup]

Parameters differences:

--- Systemd::Timer::Job[opensearch-production-search-codfw-gc-log-cleanup].orig
+++ Systemd::Timer::Job[opensearch-production-search-codfw-gc-log-cleanup]

+    description               => Cleanup GC logs
+    syslog_match_startswith   => True
+    monitoring_contact_groups => admins
+    logging_enabled           => True
+    success_exit_status       => []
+    fixed_random_delay        => False
+    private_tmp               => False
+    logfile_basedir           => /var/log
+    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+    send_mail_only_on_error   => True
+    ignore_errors             => False
+    logfile_group             => root
+    interval                  => {'start': 'OnCalendar', 'interval': '*-*-* 02:12:00'}
+    send_mail                 => False
+    logfile_name              => syslog.log
+    user                      => root
+    ensure                    => present
+    send_mail_to              => root@cirrussearch2089.codfw.wmnet
+    logfile_perms             => all
+    syslog_force_stop         => True
+    environment               => {}
+    command                   => /usr/bin/find /var/log/opensearch -name 'production-search-codfw_jvm_gc.*.log*' -mtime +30 -delete
+    monitoring_enabled        => False

File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet-key.pem]

Parameters differences:

--- File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet-key.pem].orig
+++ File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet-key.pem]

+    mode      => 0440
+    backup    => False
+    notify    => ['Exec[nginx-reload]', 'Exec[nginx-reload]']
+    show_diff => False
+    group     => root
+    owner     => root
+    ensure    => file

File[/etc/opensearch/production-search-psi-codfw/jvm.options]

Parameters differences:

--- File[/etc/opensearch/production-search-psi-codfw/jvm.options].orig
+++ File[/etc/opensearch/production-search-psi-codfw/jvm.options]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => file

Content differences:

--- /etc/opensearch/production-search-psi-codfw/jvm.options.orig
+++ /etc/opensearch/production-search-psi-codfw/jvm.options
@@ -0,0 +1,109 @@
+## JVM configuration
+
+################################################################
+## IMPORTANT: JVM heap size
+################################################################
+##
+## You should always set the min and max JVM heap
+## size to the same value. For example, to set
+## the heap to 4 GB, set:
+##
+## -Xms4g
+## -Xmx4g
+##
+## See https://opensearch.org/docs/opensearch/install/important-settings/
+## for more information
+##
+################################################################
+
+# Xms represents the initial size of total heap space
+# Xmx represents the maximum size of total heap space
+
+-Xms10G
+-Xmx10G
+
+################################################################
+## Numa Awareness
+################################################################
+-XX:+UseNUMA
+
+################################################################
+## Expert settings
+################################################################
+##
+## All settings below this section are considered
+## expert settings. Don't tamper with them unless
+## you understand what you are doing
+##
+################################################################
+
+
+## optimizations
+
+# disable calls to System#gc
+-XX:+DisableExplicitGC
+
+# pre-touch memory pages used by the JVM during initialization
+-XX:+AlwaysPreTouch
+
+## basic
+
+# force the server VM (remove on 32-bit client JVMs)
+-server
+
+# explicitly set the stack size (reduce to 320k on 32-bit client JVMs)
+-Xss1m
+
+# log4j 2
+-Dlog4j.skipJansi=true
+-Dlog4j2.formatMsgNoLookups=true
+
+# security policy, needed for OpenSearch to access non-default files/sockets/ports T395571
+
+-Djava.security.policy=/etc/opensearch/production-search-psi-codfw/java-security.policy
+
+## heap dumps
+
+# generate a heap dump when an allocation from the Java heap fails
+# heap dumps are created in the working directory of the JVM
+-XX:+HeapDumpOnOutOfMemoryError
+
+# specify an alternative path for heap dumps
+# ensure the directory exists and has sufficient space
+-XX:HeapDumpPath=/srv/opensearch/production-search-psi-codfw
+
+## GC logging
+
+#-XX:+PrintGCDetails
+#-XX:+PrintGCTimeStamps
+#-XX:+PrintGCDateStamps
+#-XX:+PrintClassHistogram
+#-XX:+PrintTenuringDistribution
+#-XX:+PrintGCApplicationStoppedTime
+
+# log GC status to a file with time stamps
+# ensure the directory exists
+#-Xloggc:${loggc}
+
+## Custom Wikimedia configuration
+
+# Defines a limit of proportion of time spent in GC before
+# an OutOfMemory error is thrown.  The goal is to bail and
+# restart the server if throughput is suffering due to
+# GC stalls.
+# TODO: Why 70? why not 60 or 80?
+-XX:GCTimeLimit=70
+
+# Defines minimum percentage of free space after a full GC
+# before an OutOfMemoryError is thrown. Typically a full GC
+# should leave > 50% of heap free. If we are down to 10%
+# something is wrong and bailing is a reasonable solution.
+-XX:GCHeapFreeLimit=10
+
+# Exit on OutOfMemoryError, since in most cases the JVM does not recover
+-XX:+ExitOnOutOfMemoryError
+
+# Custom WMF GC flags
+-Xlog:gc*:file=/var/log/opensearch/production-search-psi-codfw_jvm_gc.%p.log::filecount=10,filesize=20000
+-Xlog:gc+age=trace
+-Xlog:safepoint

Interface::Post_up_command[clsact_eno12399np0]

Parameters differences:

--- Interface::Post_up_command[clsact_eno12399np0].orig
+++ Interface::Post_up_command[clsact_eno12399np0]

+    command   => /usr/sbin/tc qdisc add dev eno12399np0 clsact
+    interface => eno12399np0
+    ensure    => absent

Exec[apt_repository_wikimedia-opensearch-plugins]

Parameters differences:

--- Exec[apt_repository_wikimedia-opensearch-plugins].orig
+++ Exec[apt_repository_wikimedia-opensearch-plugins]

+    refreshonly => True
+    command     => /usr/bin/apt-get update

File[/etc/rsyslog.d/40-opensearch-disable-readahead-production-search-codfw.conf]

Parameters differences:

--- File[/etc/rsyslog.d/40-opensearch-disable-readahead-production-search-codfw.conf].orig
+++ File[/etc/rsyslog.d/40-opensearch-disable-readahead-production-search-codfw.conf]

+    mode   => 0444
+    notify => Service[rsyslog]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/rsyslog.d/40-opensearch-disable-readahead-production-search-codfw.conf.orig
+++ /etc/rsyslog.d/40-opensearch-disable-readahead-production-search-codfw.conf
@@ -0,0 +1,10 @@
+# rsyslog.conf(5) configuration file for services.
+# This file is managed by Puppet.
+if $programname startswith "opensearch-disable-readahead-production-search-codfw" then {
+    action(
+        type="omfile" file="/var/log/opensearch-disable-readahead-production-search-codfw/syslog.log"
+        fileOwner="root" fileGroup="root"
+        fileCreateMode="0644"
+    )
+    & stop
+}

Class[Profile::Lvs::Realserver::Ipip]

Parameters differences:

--- Class[Profile::Lvs::Realserver::Ipip].orig
+++ Class[Profile::Lvs::Realserver::Ipip]

+    pools             => {'search': {'services': ['opensearch_1@production-search-codfw']}, 'search-https': {'services': ['nginx', 'opensearch_1@production-search-codfw']}, 'search-psi-https': {'services': ['nginx', 'opensearch_1@production-search-psi-codfw']}}
+    firewall_provider => ferm
+    enabled           => True
+    ipv4_mss          => 1440
+    ipv6_mss          => 1400
+    interfaces        => ['eno12399np0', 'lo']

File[/lib/systemd/system/clean_up_dlq.service]

Parameters differences:

--- File[/lib/systemd/system/clean_up_dlq.service].orig
+++ File[/lib/systemd/system/clean_up_dlq.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for clean_up_dlq.service (clean_up_dlq.service)]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /lib/systemd/system/clean_up_dlq.service.orig
+++ /lib/systemd/system/clean_up_dlq.service
@@ -0,0 +1,8 @@
+[Unit]
+Description=Clean up dead letter queue and restart logstash
+Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+
+[Service]
+Type=oneshot
+User=root
+ExecStart=/usr/local/bin/cleanup-dlq

Exec[apt-get update]

Parameters differences:

--- Exec[apt-get update].orig
+++ Exec[apt-get update]

+    before => ['Class[Opensearch]']

Rsyslog::Conf[clean_up_dlq]

Parameters differences:

--- Rsyslog::Conf[clean_up_dlq].orig
+++ Rsyslog::Conf[clean_up_dlq]

+    mode     => 0444
+    require  => File[/var/log/clean_up_dlq]
+    priority => 40
+    ensure   => absent

Systemd::Unit[opensearch-hot-threads-log.service]

Parameters differences:

--- Systemd::Unit[opensearch-hot-threads-log.service].orig
+++ Systemd::Unit[opensearch-hot-threads-log.service]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => opensearch-hot-threads-log.service
+    ensure            => present

Systemd::Syslog[opensearch-disable-readahead]

Parameters differences:

--- Systemd::Syslog[opensearch-disable-readahead].orig
+++ Systemd::Syslog[opensearch-disable-readahead]

+    programname_comparison => startswith
+    force_stop             => True
+    ensure                 => absent
+    base_dir               => /var/log
+    readable_by            => all
+    log_filename           => syslog.log
+    owner                  => root
+    group                  => root

Sysctl::Conffile[TCP Fast Open]

Parameters differences:

--- Sysctl::Conffile[TCP Fast Open].orig
+++ Sysctl::Conffile[TCP Fast Open]

+    priority => 70
+    ensure   => present

Exec[nginx-reload]

Parameters differences:

--- Exec[nginx-reload].orig
+++ Exec[nginx-reload]

+    refreshonly => True
+    command     => /usr/sbin/service nginx reload

File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-shards-9600.conf]

Parameters differences:

--- File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-shards-9600.conf].orig
+++ File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-shards-9600.conf]

+    mode   => 0444
+    notify => Service[rsyslog]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /etc/rsyslog.d/25-nrpe2nodexp-opensearch-shards-9600.conf.orig
+++ /etc/rsyslog.d/25-nrpe2nodexp-opensearch-shards-9600.conf
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: Apache-2.0
+if $programname contains "nrpe2nodexp-opensearch_shards_9600" then {
+    if ($msg contains "\"ecs.version\": \"1.7.0\"") then {
+        # Send logs to kafka
+        set $.log_outputs = "kafka ecs_170 local";
+    } else {
+        # Filter out non-relevant nrpe2nodexp messages
+        stop
+    }
+}

Prometheus::Wmf_elasticsearch_exporter[cirrussearch2089:9600]

Parameters differences:

--- Prometheus::Wmf_elasticsearch_exporter[cirrussearch2089:9600].orig
+++ Prometheus::Wmf_elasticsearch_exporter[cirrussearch2089:9600]

+    indices_to_monitor => []
+    prometheus_port    => 9121
+    elasticsearch_port => 9600

File[/lib/systemd/system/opensearch-disable-readahead-production-search-codfw.service]

Parameters differences:

--- File[/lib/systemd/system/opensearch-disable-readahead-production-search-codfw.service].orig
+++ File[/lib/systemd/system/opensearch-disable-readahead-production-search-codfw.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for opensearch-disable-readahead-production-search-codfw.service (opensearch-disable-readahead-production-search-codfw.service)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/opensearch-disable-readahead-production-search-codfw.service.orig
+++ /lib/systemd/system/opensearch-disable-readahead-production-search-codfw.service
@@ -0,0 +1,8 @@
+[Unit]
+Description=Disables readahead on all open files every 30 minutes to alleviate Cirrussearch / opensearch IO load spikes
+Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+
+[Service]
+Type=oneshot
+User=root
+ExecStart=/usr/local/bin/opensearch-disable-readahead.sh production-search-codfw /srv/opensearch

File[/var/log/opensearch]

Parameters differences:

--- File[/var/log/opensearch].orig
+++ File[/var/log/opensearch]

+    mode    => 0755
+    require => Package[opensearch]
+    group   => opensearch
+    owner   => opensearch
+    ensure  => directory

File[/usr/local/lib/nagios/plugins/check_systemd_unit_status]

Parameters differences:

--- File[/usr/local/lib/nagios/plugins/check_systemd_unit_status].orig
+++ File[/usr/local/lib/nagios/plugins/check_systemd_unit_status]

+    mode    => 0555
+    require => File[/usr/local/lib/nagios/plugins/]
+    tag     => nrpe::plugin
+    source  => puppet:///modules/systemd/check_systemd_unit_status
+    group   => root
+    owner   => root
+    ensure  => file

Exec[systemd daemon-reload for tcp-mss-clamper.service (tcp-mss-clamper)]

Parameters differences:

--- Exec[systemd daemon-reload for tcp-mss-clamper.service (tcp-mss-clamper)].orig
+++ Exec[systemd daemon-reload for tcp-mss-clamper.service (tcp-mss-clamper)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

Nrpe::Monitor_service[opensearch_shard_size_check_9200]

Parameters differences:

--- Nrpe::Monitor_service[opensearch_shard_size_check_9200].orig
+++ Nrpe::Monitor_service[opensearch_shard_size_check_9200]

+    timeout                     => 10
+    nrpe2nodexp_parse_perf_data => False
+    retry_interval              => 180
+    description                 => OpenSearch shard size check - 9200
+    notes_url                   => https://wikitech.wikimedia.org/wiki/Search#If_it_has_been_indexed
+    critical                    => False
+    migration_task              => T371083
+    retries                     => 3
+    ensure                      => present
+    enable_nrpe2nodexp          => False
+    enable_icinga_check         => True
+    check_interval              => 1440
+    contact_group               => admins,team-discovery
+    nrpe_command                => /usr/lib/nagios/plugins/check_elasticsearch_shard_size.py --url http://localhost:9200 --shard-size-warning 110 --shard-size-critical 140 --timeout 4

Systemd::Unit[opensearch_1@.service]

Parameters differences:

--- Systemd::Unit[opensearch_1@.service].orig
+++ Systemd::Unit[opensearch_1@.service]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => opensearch_1@.service
+    ensure            => present

File[/etc/opensearch/production-search-codfw]

Parameters differences:

--- File[/etc/opensearch/production-search-codfw].orig
+++ File[/etc/opensearch/production-search-codfw]

+    mode   => 0755
+    group  => root
+    owner  => root
+    ensure => directory

Prometheus::Node_ferm_mss[ferm_clamped_ipport]

Parameters differences:

--- Prometheus::Node_ferm_mss[ferm_clamped_ipport].orig
+++ Prometheus::Node_ferm_mss[ferm_clamped_ipport]

+    clamped_ipport => ['10.2.1.30:9200', '10.2.1.30:9243', '10.2.1.30:9643']
+    ensure         => present
+    outfile        => /var/lib/prometheus/node.d/ferm-mss.prom

Logrotate::Conf[opensearch-disable-readahead-production-search-psi-codfw]

Parameters differences:

--- Logrotate::Conf[opensearch-disable-readahead-production-search-psi-codfw].orig
+++ Logrotate::Conf[opensearch-disable-readahead-production-search-psi-codfw]

+    ensure => present

File[/usr/lib/nagios/plugins/check_elasticsearch_shard_size.py]

Parameters differences:

--- File[/usr/lib/nagios/plugins/check_elasticsearch_shard_size.py].orig
+++ File[/usr/lib/nagios/plugins/check_elasticsearch_shard_size.py]

+    mode   => 0755
+    group  => root
+    owner  => root
+    source => puppet:///modules/icinga/elasticsearch/check_elasticsearch_shard_size.py

File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-unassigned-shard-check-9600.conf]

Parameters differences:

--- File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-unassigned-shard-check-9600.conf].orig
+++ File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-unassigned-shard-check-9600.conf]

+    mode   => 0444
+    notify => Service[rsyslog]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /etc/rsyslog.d/25-nrpe2nodexp-opensearch-unassigned-shard-check-9600.conf.orig
+++ /etc/rsyslog.d/25-nrpe2nodexp-opensearch-unassigned-shard-check-9600.conf
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: Apache-2.0
+if $programname contains "nrpe2nodexp-opensearch_unassigned_shard_check_9600" then {
+    if ($msg contains "\"ecs.version\": \"1.7.0\"") then {
+        # Send logs to kafka
+        set $.log_outputs = "kafka ecs_170 local";
+    } else {
+        # Filter out non-relevant nrpe2nodexp messages
+        stop
+    }
+}

File[/lib/systemd/system/nrpe2nodexp-opensearch_shards_9200.timer]

Parameters differences:

--- File[/lib/systemd/system/nrpe2nodexp-opensearch_shards_9200.timer].orig
+++ File[/lib/systemd/system/nrpe2nodexp-opensearch_shards_9200.timer]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9200.timer (nrpe2nodexp-opensearch_shards_9200.timer)]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /lib/systemd/system/nrpe2nodexp-opensearch_shards_9200.timer.orig
+++ /lib/systemd/system/nrpe2nodexp-opensearch_shards_9200.timer
@@ -0,0 +1,13 @@
+[Unit]
+Description=Periodic execution of nrpe2nodexp-opensearch_shards_9200.service
+
+[Timer]
+Unit=nrpe2nodexp-opensearch_shards_9200.service
+# Accuracy sets the maximum time interval around the execution time we want to allow
+AccuracySec=15sec
+OnUnitInactiveSec=1min
+OnActiveSec=1s
+RandomizedDelaySec=0
+
+[Install]
+WantedBy=multi-user.target

File[/etc/nginx/nginx.conf]

Parameters differences:

--- File[/etc/nginx/nginx.conf].orig
+++ File[/etc/nginx/nginx.conf]

+    owner  => root
+    group  => root
+    before => ['Service[nginx]']
+    tag    => nginx

Content differences:

--- /etc/nginx/nginx.conf.orig
+++ /etc/nginx/nginx.conf
@@ -0,0 +1,67 @@
+# This file is managed by Puppet!
+
+user www-data www-data;
+worker_processes 48;
+worker_cpu_affinity 000000000000000000000001000000000000000000000001 000000000000000000000001000000000000000000000001 000000000000000000000010000000000000000000000010 000000000000000000000010000000000000000000000010 000000000000000000000100000000000000000000000100 000000000000000000000100000000000000000000000100 000000000000000000001000000000000000000000001000 000000000000000000001000000000000000000000001000 000000000000000000010000000000000000000000010000 000000000000000000010000000000000000000000010000 000000000000000000100000000000000000000000100000 000000000000000000100000000000000000000000100000 000000000000000001000000000000000000000001000000 000000000000000001000000000000000000000001000000 000000000000000010000000000000000000000010000000 000000000000000010000000000000000000000010000000 000000000000000100000000000000000000000100000000 000000000000000100000000000000000000000100000000 000000000000001000000000000000000000001000000000 000000000000001000000000000000000000001000000000 000000000000010000000000000000000000010000000000 000000000000010000000000000000000000010000000000 000000000000100000000000000000000000100000000000 000000000000100000000000000000000000100000000000 000000000001000000000000000000000001000000000000 000000000001000000000000000000000001000000000000 000000000010000000000000000000000010000000000000 000000000010000000000000000000000010000000000000 000000000100000000000000000000000100000000000000 000000000100000000000000000000000100000000000000 000000001000000000000000000000001000000000000000 000000001000000000000000000000001000000000000000 000000010000000000000000000000010000000000000000 000000010000000000000000000000010000000000000000 000000100000000000000000000000100000000000000000 000000100000000000000000000000100000000000000000 000001000000000000000000000001000000000000000000 000001000000000000000000000001000000000000000000 000010000000000000000000000010000000000000000000 000010000000000000000000000010000000000000000000 000100000000000000000000000100000000000000000000 000100000000000000000000000100000000000000000000 001000000000000000000000001000000000000000000000 001000000000000000000000001000000000000000000000 010000000000000000000000010000000000000000000000 010000000000000000000000010000000000000000000000 100000000000000000000000100000000000000000000000 100000000000000000000000100000000000000000000000;
+worker_rlimit_nofile 262144;
+
+error_log  /var/log/nginx/error.log;
+pid        /run/nginx.pid;
+
+events {
+    worker_connections 131072;
+    multi_accept off; # accept 1 conn per epoll hit, basically
+    accept_mutex off; # better latencies at high connection rates
+}
+
+http {
+    include       /etc/nginx/mime.types;
+
+    access_log	/var/log/nginx/access.log;
+
+    client_max_body_size 100m;
+    client_body_buffer_size 64k;
+    http2_max_field_size 8k;
+
+    sendfile        on;
+    #tcp_nopush     on;
+
+    #keepalive_timeout  0;
+    keepalive_timeout  65;
+    tcp_nodelay        on;
+
+    gzip  off;
+    gzip_disable "MSIE [1-6]\.(?!.*SV1)";
+
+    # SSL sessionid caching.  1m is ~4000 sessions.
+    # Still experimenting!
+    ssl_session_cache shared:SSL:1024m;
+    ssl_session_timeout 25h;
+
+    # Disable RFC5077 tickets (may revisit later when client support is better)
+    ssl_session_tickets off;
+
+    # nginx's $http2 is either 'h2' or '' (empty string) for http/1
+    # Map it to $h2 as a boolean string value '1' or '0' for use in headers
+    map $http2 $h2 {
+        default '1';
+        '' '0';
+    }
+
+    # Nginx indicates session reuse with 'r' and '.', which is rather cryptic.
+    # Map this value to '1' and '0' instead.
+    map $ssl_session_reused $session_reused {
+        'r' '1';
+        '.' '0';
+    }
+
+
+    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
+    ssl_ciphers -ALL:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_256_GCM_SHA384:TLS_AES_128_GCM_SHA256:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA;
+    ssl_ecdh_curve X25519:prime256v1;
+    ssl_prefer_server_ciphers on;
+    ssl_dhparam /etc/ssl/dhparam.pem;
+
+    include /etc/nginx/conf.d/*.conf;
+    include /etc/nginx/sites-enabled/*;
+}

Monitoring::Service[opensearch_unassigned_shard_check_9200]

Parameters differences:

--- Monitoring::Service[opensearch_unassigned_shard_check_9200].orig
+++ Monitoring::Service[opensearch_unassigned_shard_check_9200]

+    check_command  => nrpe_check!check_opensearch_unassigned_shard_check_9200!10
+    retry_interval => 120
+    description    => OpenSearch unassigned shard check - 9200
+    critical       => False
+    notes_url      => https://wikitech.wikimedia.org/wiki/Search#Administration
+    retries        => 1
+    config_dir     => /etc/nagios
+    freshness      => 36000
+    migration_task => T321808
+    ensure         => present
+    check_interval => 720
+    contact_group  => admins,team-discovery
+    host           => cirrussearch2089
+    passive        => False

Systemd::Unit[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service]

Parameters differences:

--- Systemd::Unit[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service].orig
+++ Systemd::Unit[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service
+    ensure            => present

File[/lib/systemd/system/opensearch-hot-threads-log.service]

Parameters differences:

--- File[/lib/systemd/system/opensearch-hot-threads-log.service].orig
+++ File[/lib/systemd/system/opensearch-hot-threads-log.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for opensearch-hot-threads-log.service (opensearch-hot-threads-log.service)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/opensearch-hot-threads-log.service.orig
+++ /lib/systemd/system/opensearch-hot-threads-log.service
@@ -0,0 +1,8 @@
+[Unit]
+Description=Archive exception logs of hot opensearch threads
+Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+
+[Service]
+Type=oneshot
+User=opensearch
+ExecStart=/usr/local/bin/opensearch_hot_threads_logger.py

Systemd::Syslog[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]

Parameters differences:

--- Systemd::Syslog[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600].orig
+++ Systemd::Syslog[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]

+    programname_comparison => startswith
+    force_stop             => True
+    ensure                 => present
+    base_dir               => /var/log
+    readable_by            => all
+    log_filename           => syslog.log
+    owner                  => root
+    group                  => root

File[/lib/systemd/system/nrpe2nodexp-opensearch_shards_9200.service]

Parameters differences:

--- File[/lib/systemd/system/nrpe2nodexp-opensearch_shards_9200.service].orig
+++ File[/lib/systemd/system/nrpe2nodexp-opensearch_shards_9200.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9200.service (nrpe2nodexp-opensearch_shards_9200.service)]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /lib/systemd/system/nrpe2nodexp-opensearch_shards_9200.service.orig
+++ /lib/systemd/system/nrpe2nodexp-opensearch_shards_9200.service
@@ -0,0 +1,11 @@
+[Unit]
+Description=execution of nrpe2nodexp for the check_opensearch_shards_9200 command.
+Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+
+[Service]
+Type=oneshot
+User=nagios
+
+Group=prometheus-node-exporter
+SyslogIdentifier=nrpe2nodexp-opensearch_shards_9200
+ExecStart=-/usr/local/bin/nrpe2nodexp --alert-rule-hash "8cb398d9f4c08b77d96bfdab16485563" --timeout 10 --check-command "check_opensearch_shards_9200"

Package[python3-requests]

Parameters differences:

--- Package[python3-requests].orig
+++ Package[python3-requests]

+    provider => apt
+    ensure   => installed

Systemd::Service[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]

Parameters differences:

--- Systemd::Service[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600].orig
+++ Systemd::Service[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => present
+    monitoring_critical      => False
+    require                  => Systemd::Unit[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service]
+    monitoring_enabled       => False
+    unit_type                => timer
+    restart                  => False

File[/etc/rsyslog.d/40-opensearch-production-search-psi-codfw-gc-log-cleanup.conf]

Parameters differences:

--- File[/etc/rsyslog.d/40-opensearch-production-search-psi-codfw-gc-log-cleanup.conf].orig
+++ File[/etc/rsyslog.d/40-opensearch-production-search-psi-codfw-gc-log-cleanup.conf]

+    mode   => 0444
+    notify => Service[rsyslog]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/rsyslog.d/40-opensearch-production-search-psi-codfw-gc-log-cleanup.conf.orig
+++ /etc/rsyslog.d/40-opensearch-production-search-psi-codfw-gc-log-cleanup.conf
@@ -0,0 +1,10 @@
+# rsyslog.conf(5) configuration file for services.
+# This file is managed by Puppet.
+if $programname startswith "opensearch-production-search-psi-codfw-gc-log-cleanup" then {
+    action(
+        type="omfile" file="/var/log/opensearch-production-search-psi-codfw-gc-log-cleanup/syslog.log"
+        fileOwner="root" fileGroup="root"
+        fileCreateMode="0644"
+    )
+    & stop
+}

File[/var/log/prometheus_lvs_realserver_mss]

Parameters differences:

--- File[/var/log/prometheus_lvs_realserver_mss].orig
+++ File[/var/log/prometheus_lvs_realserver_mss]

+    mode   => 0755
+    backup => False
+    force  => True
+    owner  => root
+    ensure => directory
+    group  => root

File[/etc/opensearch/production-search-psi-codfw/opensearch.keystore]

Parameters differences:

--- File[/etc/opensearch/production-search-psi-codfw/opensearch.keystore].orig
+++ File[/etc/opensearch/production-search-psi-codfw/opensearch.keystore]

+    mode   => 0640
+    group  => opensearch
+    owner  => root
+    ensure => present

File[/etc/nagios/nrpe.d/check_opensearch_shards_9600.cfg]

Parameters differences:

--- File[/etc/nagios/nrpe.d/check_opensearch_shards_9600.cfg].orig
+++ File[/etc/nagios/nrpe.d/check_opensearch_shards_9600.cfg]

+    mode    => 0444
+    require => Package[nagios-nrpe-server]
+    tag     => nrpe::check
+    notify  => Service[nagios-nrpe-server]
+    group   => root
+    owner   => root
+    ensure  => present

Content differences:

--- /etc/nagios/nrpe.d/check_opensearch_shards_9600.cfg.orig
+++ /etc/nagios/nrpe.d/check_opensearch_shards_9600.cfg
@@ -0,0 +1,2 @@
+# File generated by puppet. DO NOT edit by hand
+command[check_opensearch_shards_9600]=/usr/lib/nagios/plugins/check_elasticsearch.py --ignore-status --url http://localhost:9600 --shards-inactive '>=0.15' --timeout 4

Systemd::Unit[prometheus_lvs_realserver_mss.service]

Parameters differences:

--- Systemd::Unit[prometheus_lvs_realserver_mss.service].orig
+++ Systemd::Unit[prometheus_lvs_realserver_mss.service]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => prometheus_lvs_realserver_mss.service
+    ensure            => present

Rsyslog::Conf[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]

Parameters differences:

--- Rsyslog::Conf[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200].orig
+++ Rsyslog::Conf[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]

+    mode     => 0444
+    require  => File[/var/log/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]
+    priority => 40
+    ensure   => present

Systemd::Monitor[tcp-mss-clamper]

Parameters differences:

--- Systemd::Monitor[tcp-mss-clamper].orig
+++ Systemd::Monitor[tcp-mss-clamper]

+    check_interval => 10
+    contact_group  => admins
+    notes_url      => https://wikitech.wikimedia.org/wiki/LVS#IPIP_encapsulation_experiments
+    migration_task => T321808
+    critical       => False
+    retries        => 2
+    ensure         => absent

File[/lib/systemd/system/opensearch-production-search-psi-codfw-gc-log-cleanup.service]

Parameters differences:

--- File[/lib/systemd/system/opensearch-production-search-psi-codfw-gc-log-cleanup.service].orig
+++ File[/lib/systemd/system/opensearch-production-search-psi-codfw-gc-log-cleanup.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for opensearch-production-search-psi-codfw-gc-log-cleanup.service (opensearch-production-search-psi-codfw-gc-log-cleanup.service)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/opensearch-production-search-psi-codfw-gc-log-cleanup.service.orig
+++ /lib/systemd/system/opensearch-production-search-psi-codfw-gc-log-cleanup.service
@@ -0,0 +1,8 @@
+[Unit]
+Description=Cleanup GC logs
+Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+
+[Service]
+Type=oneshot
+User=root
+ExecStart=/usr/bin/find /var/log/opensearch -name 'production-search-psi-codfw_jvm_gc.*.log*' -mtime +30 -delete

Systemd::Unit[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer]

Parameters differences:

--- Systemd::Unit[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer].orig
+++ Systemd::Unit[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer
+    ensure            => present

File[/lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer]

Parameters differences:

--- File[/lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer].orig
+++ File[/lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer (nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer)]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer.orig
+++ /lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer
@@ -0,0 +1,13 @@
+[Unit]
+Description=Periodic execution of nrpe2nodexp-opensearch_unassigned_shard_check_9600.service
+
+[Timer]
+Unit=nrpe2nodexp-opensearch_unassigned_shard_check_9600.service
+# Accuracy sets the maximum time interval around the execution time we want to allow
+AccuracySec=15sec
+OnUnitInactiveSec=720min
+OnActiveSec=1s
+RandomizedDelaySec=0
+
+[Install]
+WantedBy=multi-user.target

Sudo::User[nrpe-check_opensearch_shards_9200]

Parameters differences:

--- Sudo::User[nrpe-check_opensearch_shards_9200].orig
+++ Sudo::User[nrpe-check_opensearch_shards_9200]

+    privileges => []
+    tag        => nrpe::check
+    require    => ['Class[Sudo]']
+    user       => nagios
+    ensure     => absent

Ferm::Service[opensearch-http-9200]

Parameters differences:

--- Ferm::Service[opensearch-http-9200].orig
+++ Ferm::Service[opensearch-http-9200]

+    proto   => tcp
+    port    => 9200
+    prio    => 10
+    desc    => 
+    notrack => True
+    srange  => $DOMAIN_NETWORKS
+    ensure  => present

File[/etc/logrotate.d/opensearch-disable-readahead-production-search-codfw]

Parameters differences:

--- File[/etc/logrotate.d/opensearch-disable-readahead-production-search-codfw].orig
+++ File[/etc/logrotate.d/opensearch-disable-readahead-production-search-codfw]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/logrotate.d/opensearch-disable-readahead-production-search-codfw.orig
+++ /etc/logrotate.d/opensearch-disable-readahead-production-search-codfw
@@ -0,0 +1,12 @@
+# logrotate(8) config for opensearch-disable-readahead-production-search-codfw
+
+/var/log/opensearch-disable-readahead-production-search-codfw/*.log {
+    daily
+    copytruncate
+    missingok
+    compress
+    delaycompress
+    notifempty
+    rotate 15
+    size 256M
+}

File[/var/log/opensearch/opensearch.log]

Parameters differences:

--- File[/var/log/opensearch/opensearch.log].orig
+++ File[/var/log/opensearch/opensearch.log]

+    group  => root
+    owner  => root
+    ensure => absent

Ferm::Service[opensearch-inter-node-9700]

Parameters differences:

--- Ferm::Service[opensearch-inter-node-9700].orig
+++ Ferm::Service[opensearch-inter-node-9700]

+    proto   => tcp
+    port    => 9700
+    prio    => 10
+    desc    => 
+    notrack => True
+    srange  => @resolve((cirrussearch2061.codfw.wmnet cirrussearch2062.codfw.wmnet cirrussearch2063.codfw.wmnet cirrussearch2064.codfw.wmnet cirrussearch2065.codfw.wmnet cirrussearch2066.codfw.wmnet cirrussearch2067.codfw.wmnet cirrussearch2068.codfw.wmnet cirrussearch2069.codfw.wmnet cirrussearch2070.codfw.wmnet cirrussearch2071.codfw.wmnet cirrussearch2072.codfw.wmnet cirrussearch2073.codfw.wmnet cirrussearch2074.codfw.wmnet cirrussearch2075.codfw.wmnet cirrussearch2076.codfw.wmnet cirrussearch2077.codfw.wmnet cirrussearch2078.codfw.wmnet cirrussearch2079.codfw.wmnet cirrussearch2080.codfw.wmnet cirrussearch2081.codfw.wmnet cirrussearch2082.codfw.wmnet cirrussearch2083.codfw.wmnet cirrussearch2084.codfw.wmnet cirrussearch2085.codfw.wmnet cirrussearch2086.codfw.wmnet cirrussearch2087.codfw.wmnet cirrussearch2088.codfw.wmnet cirrussearch2089.codfw.wmnet cirrussearch2090.codfw.wmnet cirrussearch2091.codfw.wmnet cirrussearch2092.codfw.wmnet cirrussearch2093.codfw.wmnet cirrussearch2094.codfw.wmnet cirrussearch2095.codfw.wmnet cirrussearch2096.codfw.wmnet cirrussearch2097.codfw.wmnet cirrussearch2098.codfw.wmnet cirrussearch2099.codfw.wmnet cirrussearch2100.codfw.wmnet cirrussearch2101.codfw.wmnet cirrussearch2102.codfw.wmnet cirrussearch2103.codfw.wmnet cirrussearch2104.codfw.wmnet cirrussearch2105.codfw.wmnet cirrussearch2106.codfw.wmnet cirrussearch2107.codfw.wmnet cirrussearch2108.codfw.wmnet cirrussearch2109.codfw.wmnet cirrussearch2110.codfw.wmnet cirrussearch2111.codfw.wmnet cirrussearch2112.codfw.wmnet cirrussearch2113.codfw.wmnet cirrussearch2114.codfw.wmnet cirrussearch2115.codfw.wmnet))
+    ensure  => present

Systemd::Timer::Job[wmf_auto_restart_prometheus-elasticsearch-exporter-9600]

Parameters differences:

--- Systemd::Timer::Job[wmf_auto_restart_prometheus-elasticsearch-exporter-9600].orig
+++ Systemd::Timer::Job[wmf_auto_restart_prometheus-elasticsearch-exporter-9600]

+    description               => Auto restart job: prometheus-elasticsearch-exporter-9600
+    syslog_match_startswith   => True
+    monitoring_contact_groups => admins
+    logging_enabled           => True
+    success_exit_status       => []
+    require                   => File[/usr/local/sbin/wmf-auto-restart]
+    fixed_random_delay        => False
+    private_tmp               => False
+    logfile_basedir           => /var/log
+    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+    send_mail_only_on_error   => True
+    ignore_errors             => False
+    logfile_group             => root
+    interval                  => {'start': 'OnCalendar', 'interval': 'Mon,Tue,Wed,Thu,Fri *-*-* 5:3:00'}
+    send_mail                 => False
+    logfile_name              => syslog.log
+    user                      => root
+    ensure                    => present
+    send_mail_to              => root@cirrussearch2089.codfw.wmnet
+    logfile_perms             => all
+    syslog_force_stop         => True
+    environment               => {}
+    command                   => /usr/local/sbin/wmf-auto-restart -s prometheus-elasticsearch-exporter-9600
+    monitoring_enabled        => False

Systemd::Service[opensearch-hot-threads-log]

Parameters differences:

--- Systemd::Service[opensearch-hot-threads-log].orig
+++ Systemd::Service[opensearch-hot-threads-log]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => present
+    monitoring_critical      => False
+    require                  => Systemd::Unit[opensearch-hot-threads-log.service]
+    monitoring_enabled       => False
+    unit_type                => timer
+    restart                  => False

Profile::Auto_restarts::Service[prometheus-wmf-elasticsearch-exporter-9200]

Parameters differences:

--- Profile::Auto_restarts::Service[prometheus-wmf-elasticsearch-exporter-9200].orig
+++ Profile::Auto_restarts::Service[prometheus-wmf-elasticsearch-exporter-9200]

+    ensure => present

Prometheus::Alert::Rule[check_opensearch_unassigned_shard_check_9600_8641ad857de0a06c2748b18ea2c135ab]

Parameters differences:

--- Prometheus::Alert::Rule[check_opensearch_unassigned_shard_check_9600_8641ad857de0a06c2748b18ea2c135ab].orig
+++ Prometheus::Alert::Rule[check_opensearch_unassigned_shard_check_9600_8641ad857de0a06c2748b18ea2c135ab]

+    dashboard          => TODO
+    expr               => (nagios_nrpe_check_result{alert_rule_hash="8641ad857de0a06c2748b18ea2c135ab",check_name="check_opensearch_unassigned_shard_check_9600", status=~"(WARNING|CRITICAL)", severity=~"(warning|critical)"} > 0) * on (instance) group_left (team) role_owner
+    team               => observability
+    description        => OpenSearch unassigned shard check - 9600
+    runbook            => https://wikitech.wikimedia.org/wiki/Search#Administration
+    def_label_whitelst => ['team', 'severity']
+    ensure             => absent
+    logs               => https://logstash.wikimedia.org/app/dashboards#/view/2d343ac0-6df8-11f0-8e08-7fab0da52b33?_g=(filters:!((query:(match_phrase:(event.module:check_opensearch_unassigned_shard_check_9600))),(query:(match_phrase:(host.name:{{$labels.instance|stripPort}})))))
+    severity           => info
+    summary            => OpenSearch unassigned shard check - 9600
+    instance           => ops
+    group              => nrpechecks
+    alert_name         => check_opensearch_unassigned_shard_check_9600
+    for                => 86400m
+    site               => codfw

Package[logstash]

Parameters differences:

--- Package[logstash].orig
+++ Package[logstash]

+    require  => Package[openjdk-11-jdk]
+    name     => logstash-oss
+    ensure   => present
+    provider => apt

Package[tcp-mss-clamper]

Parameters differences:

--- Package[tcp-mss-clamper].orig
+++ Package[tcp-mss-clamper]

+    provider => apt
+    ensure   => absent

Elasticsearch::Tlsproxy[production-search-psi-codfw]

Parameters differences:

--- Elasticsearch::Tlsproxy[production-search-psi-codfw].orig
+++ Elasticsearch::Tlsproxy[production-search-psi-codfw]

+    acme_chief        => False
+    upstream_port     => 9600
+    tls_port          => 9643
+    server_aliases    => ['search.discovery.wmnet', 'search-chi.discovery.wmnet', 'search-omega.discovery.wmnet', 'search-psi.discovery.wmnet', 'search.svc.codfw.wmnet', 'search-chi.svc.codfw.wmnet', 'search-omega.svc.codfw.wmnet', 'search-psi.svc.codfw.wmnet']
+    enable_http2      => False
+    read_only         => False
+    server_name       => cirrussearch2089.codfw.wmnet
+    certificate_names => []
+    cfssl_paths       => {'cert': '/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.pem', 'key': '/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet-key.pem', 'chain': '/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chain.pem', 'chained': '/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chained.pem'}

File[/usr/lib/nagios/plugins/check_elasticsearch]

Parameters differences:

--- File[/usr/lib/nagios/plugins/check_elasticsearch].orig
+++ File[/usr/lib/nagios/plugins/check_elasticsearch]

+    mode   => 0755
+    group  => root
+    owner  => root
+    source => puppet:///modules/icinga/elasticsearch/check_elasticsearch

File[/etc/apt/sources.list.d/wikimedia-opensearch-plugins.list]

Parameters differences:

--- File[/etc/apt/sources.list.d/wikimedia-opensearch-plugins.list].orig
+++ File[/etc/apt/sources.list.d/wikimedia-opensearch-plugins.list]

+    mode   => 0444
+    notify => Exec[apt_repository_wikimedia-opensearch-plugins]
+    group  => root
+    owner  => root
+    ensure => file

Content differences:

--- /etc/apt/sources.list.d/wikimedia-opensearch-plugins.list.orig
+++ /etc/apt/sources.list.d/wikimedia-opensearch-plugins.list
@@ -0,0 +1,2 @@
+deb http://apt.wikimedia.org/wikimedia bullseye-wikimedia component/opensearch13
+deb-src http://apt.wikimedia.org/wikimedia bullseye-wikimedia component/opensearch13

Exec[ip link set up dev ipip60]

Parameters differences:

--- Exec[ip link set up dev ipip60].orig
+++ Exec[ip link set up dev ipip60]

+    unless  => ip link show ipip60 | grep -q UP
+    path    => /bin:/usr/bin
+    returns => [0, 2]

Exec[systemd daemon-reload for wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer (wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer)]

Parameters differences:

--- Exec[systemd daemon-reload for wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer (wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer)].orig
+++ Exec[systemd daemon-reload for wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer (wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload
+    before      => ['Service[wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer]']

Logrotate::Conf[opensearch]

Parameters differences:

--- Logrotate::Conf[opensearch].orig
+++ Logrotate::Conf[opensearch]

+    ensure => present

Class[Monitoring]

Parameters differences:

--- Class[Monitoring].orig
+++ Class[Monitoring]

@@
-    nagios_group          => insetup_codfw
+    nagios_group          => elasticsearch_codfw
@@
-    notifications_enabled => False
+    notifications_enabled => True
@@
-    cluster               => insetup
+    cluster               => elasticsearch
@@
-    contact_group         => admins
+    contact_group         => team-data-platform,team-discovery

Systemd::Syslog[opensearch-disable-readahead-production-search-codfw]

Parameters differences:

--- Systemd::Syslog[opensearch-disable-readahead-production-search-codfw].orig
+++ Systemd::Syslog[opensearch-disable-readahead-production-search-codfw]

+    programname_comparison => startswith
+    force_stop             => True
+    ensure                 => present
+    base_dir               => /var/log
+    readable_by            => all
+    log_filename           => syslog.log
+    owner                  => root
+    group                  => root

Etcd::Client::Config[/root/.etcdrc]

Parameters differences:

--- Etcd::Client::Config[/root/.etcdrc].orig
+++ Etcd::Client::Config[/root/.etcdrc]

+    world_readable => False
+    settings       => {'username': 'conftool', 'password': 'another_secret'}
+    owner          => root
+    ensure         => present
+    group          => root

Ferm::Service[opensearch-http-9600]

Parameters differences:

--- Ferm::Service[opensearch-http-9600].orig
+++ Ferm::Service[opensearch-http-9600]

+    proto   => tcp
+    port    => 9600
+    prio    => 10
+    desc    => 
+    notrack => True
+    srange  => $DOMAIN_NETWORKS
+    ensure  => present

Systemd::Timer[nrpe2nodexp-opensearch_shard_size_check_9200]

Parameters differences:

--- Systemd::Timer[nrpe2nodexp-opensearch_shard_size_check_9200].orig
+++ Systemd::Timer[nrpe2nodexp-opensearch_shard_size_check_9200]

+    fixed_random_delay => False
+    unit_name          => nrpe2nodexp-opensearch_shard_size_check_9200.service
+    timer_intervals    => [{'start': 'OnUnitInactiveSec', 'interval': '1440min'}, {'interval': '1s', 'start': 'OnActiveSec'}]
+    splay              => 0
+    accuracy           => 15sec
+    ensure             => absent

Prometheus::Elasticsearch_exporter[localhost:9600]

Parameters differences:

--- Prometheus::Elasticsearch_exporter[localhost:9600].orig
+++ Prometheus::Elasticsearch_exporter[localhost:9600]

+    extra_config       => 
+    prometheus_port    => 9109
+    elasticsearch_port => 9600

Systemd::Syslog[wmf_auto_restart_prometheus-elasticsearch-exporter-9600]

Parameters differences:

--- Systemd::Syslog[wmf_auto_restart_prometheus-elasticsearch-exporter-9600].orig
+++ Systemd::Syslog[wmf_auto_restart_prometheus-elasticsearch-exporter-9600]

+    programname_comparison => startswith
+    force_stop             => True
+    ensure                 => present
+    base_dir               => /var/log
+    readable_by            => all
+    log_filename           => syslog.log
+    owner                  => root
+    group                  => root

Systemd::Service[opensearch-production-search-psi-codfw-gc-log-cleanup]

Parameters differences:

--- Systemd::Service[opensearch-production-search-psi-codfw-gc-log-cleanup].orig
+++ Systemd::Service[opensearch-production-search-psi-codfw-gc-log-cleanup]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => present
+    monitoring_critical      => False
+    require                  => Systemd::Unit[opensearch-production-search-psi-codfw-gc-log-cleanup.service]
+    monitoring_enabled       => False
+    unit_type                => timer
+    restart                  => False

Exec[systemd daemon-reload for prometheus-wmf-elasticsearch-exporter-9600.service (prometheus-wmf-elasticsearch-exporter-9600)]

Parameters differences:

--- Exec[systemd daemon-reload for prometheus-wmf-elasticsearch-exporter-9600.service (prometheus-wmf-elasticsearch-exporter-9600)].orig
+++ Exec[systemd daemon-reload for prometheus-wmf-elasticsearch-exporter-9600.service (prometheus-wmf-elasticsearch-exporter-9600)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload
+    notify      => ['Service[prometheus-wmf-elasticsearch-exporter-9600]']

Systemd::Unit[prometheus_lvs_realserver_mss.timer]

Parameters differences:

--- Systemd::Unit[prometheus_lvs_realserver_mss.timer].orig
+++ Systemd::Unit[prometheus_lvs_realserver_mss.timer]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => prometheus_lvs_realserver_mss.timer
+    ensure            => present

File[/etc/tmpfiles.d/opensearch-production-search-codfw.conf]

Parameters differences:

--- File[/etc/tmpfiles.d/opensearch-production-search-codfw.conf].orig
+++ File[/etc/tmpfiles.d/opensearch-production-search-codfw.conf]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/tmpfiles.d/opensearch-production-search-codfw.conf.orig
+++ /etc/tmpfiles.d/opensearch-production-search-codfw.conf
@@ -0,0 +1 @@
+d    /run/opensearch-production-search-codfw  0755 opensearch opensearch - -

File[/lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service]

Parameters differences:

--- File[/lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service].orig
+++ File[/lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service (wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service.orig
+++ /lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service
@@ -0,0 +1,8 @@
+[Unit]
+Description=Auto restart job: prometheus-wmf-elasticsearch-exporter-9200
+Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+
+[Service]
+Type=oneshot
+User=root
+ExecStart=/usr/local/sbin/wmf-auto-restart -s prometheus-wmf-elasticsearch-exporter-9200

Package[logstash-old-name]

Parameters differences:

--- Package[logstash-old-name].orig
+++ Package[logstash-old-name]

+    before   => Package[logstash]
+    name     => logstash
+    ensure   => absent
+    provider => apt

Opensearch::Keystore[s3-credentials-user-production-search-psi-codfw]

Parameters differences:

--- Opensearch::Keystore[s3-credentials-user-production-search-psi-codfw].orig
+++ Opensearch::Keystore[s3-credentials-user-production-search-psi-codfw]

+    value        => search:platform
+    cluster_name => production-search-psi-codfw
+    key          => s3.client.default.access_key

File[/usr/local/sbin/restart-opensearch_1@production-search-psi-codfw]

Parameters differences:

--- File[/usr/local/sbin/restart-opensearch_1@production-search-psi-codfw].orig
+++ File[/usr/local/sbin/restart-opensearch_1@production-search-psi-codfw]

+    mode   => 0555
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /usr/local/sbin/restart-opensearch_1@production-search-psi-codfw.orig
+++ /usr/local/sbin/restart-opensearch_1@production-search-psi-codfw
@@ -0,0 +1,2 @@
+#!/bin/bash
+/usr/local/bin/safe-service-restart --pools search-psi-https --services opensearch_1@production-search-psi-codfw --retries 10 --wait 5 --max-concurrency 5 $@

Systemd::Service[prometheus-elasticsearch-exporter-9200]

Parameters differences:

--- Systemd::Service[prometheus-elasticsearch-exporter-9200].orig
+++ Systemd::Service[prometheus-elasticsearch-exporter-9200]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {'ensure': 'running'}
+    ensure                   => present
+    monitoring_critical      => False
+    require                  => Package[prometheus-elasticsearch-exporter]
+    monitoring_enabled       => False
+    unit_type                => service
+    restart                  => False

Exec[systemd daemon-reload for opensearch-disable-readahead-production-search-codfw.service (opensearch-disable-readahead-production-search-codfw.service)]

Parameters differences:

--- Exec[systemd daemon-reload for opensearch-disable-readahead-production-search-codfw.service (opensearch-disable-readahead-production-search-codfw.service)].orig
+++ Exec[systemd daemon-reload for opensearch-disable-readahead-production-search-codfw.service (opensearch-disable-readahead-production-search-codfw.service)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

File[/lib/systemd/system/tcp-mss-clamper.service]

Parameters differences:

--- File[/lib/systemd/system/tcp-mss-clamper.service].orig
+++ File[/lib/systemd/system/tcp-mss-clamper.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for tcp-mss-clamper.service (tcp-mss-clamper)]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /lib/systemd/system/tcp-mss-clamper.service.orig
+++ /lib/systemd/system/tcp-mss-clamper.service
@@ -0,0 +1,11 @@
+[Unit]
+Description=eBPF based TCP MSS clamper
+After=network.target
+
+[Install]
+WantedBy=multi-user.target
+
+[Service]
+LimitMEMLOCK=infinity
+ExecStart=/usr/bin/tcp-mss-clamper --ipv4-mss 1440 --ipv6-mss 1400 -p :2200 -s "10.2.1.30:9200,10.2.1.30:9243,10.2.1.30:9643" -i eno12399np0,lo
+Restart=on-failure

Systemd::Timer::Job[opensearch-disable-readahead-production-search-psi-codfw]

Parameters differences:

--- Systemd::Timer::Job[opensearch-disable-readahead-production-search-psi-codfw].orig
+++ Systemd::Timer::Job[opensearch-disable-readahead-production-search-psi-codfw]

+    description               => Disables readahead on all open files every 30 minutes to alleviate Cirrussearch / opensearch IO load spikes
+    syslog_match_startswith   => True
+    monitoring_contact_groups => admins
+    logging_enabled           => True
+    success_exit_status       => []
+    fixed_random_delay        => False
+    private_tmp               => False
+    logfile_basedir           => /var/log
+    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+    send_mail_only_on_error   => True
+    ignore_errors             => False
+    logfile_group             => root
+    interval                  => [{'start': 'OnUnitActiveSec', 'interval': '30min'}, {'start': 'OnBootSec', 'interval': '1min'}]
+    send_mail                 => False
+    logfile_name              => syslog.log
+    user                      => root
+    ensure                    => present
+    send_mail_to              => root@cirrussearch2089.codfw.wmnet
+    logfile_perms             => all
+    syslog_force_stop         => True
+    environment               => {}
+    command                   => /usr/local/bin/opensearch-disable-readahead.sh production-search-psi-codfw /srv/opensearch
+    monitoring_enabled        => False

File[/etc/opensearch/production-search-psi-codfw/java-security.policy]

Parameters differences:

--- File[/etc/opensearch/production-search-psi-codfw/java-security.policy].orig
+++ File[/etc/opensearch/production-search-psi-codfw/java-security.policy]

+    mode   => 0440
+    source => puppet:///modules/profile/opensearch/java-security.policy
+    group  => opensearch
+    owner  => opensearch
+    ensure => file

Systemd::Unit[opensearch-disable-readahead-production-search-codfw.timer]

Parameters differences:

--- Systemd::Unit[opensearch-disable-readahead-production-search-codfw.timer].orig
+++ Systemd::Unit[opensearch-disable-readahead-production-search-codfw.timer]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => opensearch-disable-readahead-production-search-codfw.timer
+    ensure            => present

Service[opensearch_1@production-search-codfw]

Parameters differences:

--- Service[opensearch_1@production-search-codfw].orig
+++ Service[opensearch_1@production-search-codfw]

+    require  => ['Systemd::Unit[opensearch_1@.service]', 'File[/etc/opensearch/production-search-codfw/opensearch.yml]', 'File[/etc/opensearch/production-search-codfw/logging.yml]', 'File[/etc/opensearch/production-search-codfw/log4j2.properties]', 'File[/etc/opensearch/production-search-codfw/jvm.options]', 'File[/srv/opensearch/production-search-codfw]', 'Systemd::Tmpfile[opensearch-production-search-codfw]']
+    tag      => opensearch_services
+    enable   => True
+    provider => systemd
+    ensure   => running

Service[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer]

Parameters differences:

--- Service[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer].orig
+++ Service[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer]

+    provider => systemd
+    enable   => True
+    ensure   => running

Systemd::Timer::Job[nrpe2nodexp-opensearch_unassigned_shard_check_9200]

Parameters differences:

--- Systemd::Timer::Job[nrpe2nodexp-opensearch_unassigned_shard_check_9200].orig
+++ Systemd::Timer::Job[nrpe2nodexp-opensearch_unassigned_shard_check_9200]

+    description               => execution of nrpe2nodexp for the check_opensearch_unassigned_shard_check_9200 command.
+    syslog_match_startswith   => True
+    monitoring_contact_groups => admins
+    logging_enabled           => False
+    success_exit_status       => []
+    fixed_random_delay        => False
+    private_tmp               => False
+    logfile_basedir           => /var/log
+    group                     => prometheus-node-exporter
+    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+    syslog_identifier         => nrpe2nodexp-opensearch_unassigned_shard_check_9200
+    send_mail_only_on_error   => True
+    ignore_errors             => True
+    logfile_group             => root
+    interval                  => [{'start': 'OnUnitInactiveSec', 'interval': '720min'}]
+    send_mail                 => False
+    logfile_name              => syslog.log
+    user                      => nagios
+    ensure                    => absent
+    send_mail_to              => root@cirrussearch2089.codfw.wmnet
+    logfile_perms             => all
+    syslog_force_stop         => True
+    environment               => {}
+    command                   => /usr/local/bin/nrpe2nodexp --alert-rule-hash "f98d7b74d9467ea0aa193a52c18e30ab" --timeout 10 --check-command "check_opensearch_unassigned_shard_check_9200"
+    monitoring_enabled        => False

File[/usr/local/sbin/restart-opensearch_1@production-search-codfw]

Parameters differences:

--- File[/usr/local/sbin/restart-opensearch_1@production-search-codfw].orig
+++ File[/usr/local/sbin/restart-opensearch_1@production-search-codfw]

+    mode   => 0555
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /usr/local/sbin/restart-opensearch_1@production-search-codfw.orig
+++ /usr/local/sbin/restart-opensearch_1@production-search-codfw
@@ -0,0 +1,2 @@
+#!/bin/bash
+/usr/local/bin/safe-service-restart --pools search search-https --services opensearch_1@production-search-codfw --retries 10 --wait 5 --max-concurrency 5 $@

Systemd::Service[clean_up_dlq]

Parameters differences:

--- Systemd::Service[clean_up_dlq].orig
+++ Systemd::Service[clean_up_dlq]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => absent
+    monitoring_critical      => False
+    require                  => Systemd::Unit[clean_up_dlq.service]
+    monitoring_enabled       => False
+    unit_type                => timer
+    restart                  => False

Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_shards_9600]

Parameters differences:

--- Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_shards_9600].orig
+++ Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_shards_9600]

+    passive_checks_enabled => 1
+    is_volatile            => 0
+    contact_groups         => admins,team-discovery
+    notification_options   => c,r,f
+    service_description    => OpenSearch health check for shards on 9600
+    max_check_attempts     => 3
+    active_checks_enabled  => 1
+    notification_interval  => 0
+    check_command          => nrpe_check!check_opensearch_shards_9600!10
+    retry_interval         => 1
+    check_freshness        => 0
+    notes_url              => https://wikitech.wikimedia.org/wiki/Search#Administration
+    notifications_enabled  => 1
+    check_period           => 24x7
+    ensure                 => present
+    check_interval         => 1
+    notification_period    => 24x7
+    host_name              => cirrussearch2089
+    servicegroups          => elasticsearch_codfw

File[/etc/opensearch/production-search-psi-codfw/scripts]

Parameters differences:

--- File[/etc/opensearch/production-search-psi-codfw/scripts].orig
+++ File[/etc/opensearch/production-search-psi-codfw/scripts]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => directory

Class[Etcd::Client::Globalconfig]

Parameters differences:

--- Class[Etcd::Client::Globalconfig].orig
+++ Class[Etcd::Client::Globalconfig]

+    srv_domain => conftool.codfw.wmnet

Logrotate::Conf[prometheus_lvs_realserver_mss]

Parameters differences:

--- Logrotate::Conf[prometheus_lvs_realserver_mss].orig
+++ Logrotate::Conf[prometheus_lvs_realserver_mss]

+    ensure => present

File[/etc/sudoers.d/nrpe-check_opensearch_shard_size_check_9200]

Parameters differences:

--- File[/etc/sudoers.d/nrpe-check_opensearch_shard_size_check_9200].orig
+++ File[/etc/sudoers.d/nrpe-check_opensearch_shard_size_check_9200]

+    group   => root
+    require => Package[nagios-nrpe-server]
+    owner   => root
+    ensure  => absent

File[/etc/sysctl.d/10-ubuntu-defaults.conf]

Content differences:

--- /etc/sysctl.d/10-ubuntu-defaults.conf.orig
+++ /etc/sysctl.d/10-ubuntu-defaults.conf
@@ -4,7 +4,7 @@
 kernel.kptr_restrict = 1
 kernel.printk = 4 4 1 7
 kernel.yama.ptrace_scope = 1
-net.ipv4.conf.all.rp_filter = 1
-net.ipv4.conf.default.rp_filter = 1
+net.ipv4.conf.all.rp_filter = 0
+net.ipv4.conf.default.rp_filter = 0
 net.ipv4.tcp_syncookies = 1
 vm.mmap_min_addr = 65536

Monitoring::Exported_nagios_service[cirrussearch2089 raid_broadcom_raid]

Parameters differences:

--- Monitoring::Exported_nagios_service[cirrussearch2089 raid_broadcom_raid].orig
+++ Monitoring::Exported_nagios_service[cirrussearch2089 raid_broadcom_raid]

@@
-    notifications_enabled => 0
+    notifications_enabled => 1
@@
-    servicegroups         => insetup_codfw
+    servicegroups         => elasticsearch_codfw
@@
-    contact_groups        => admins
+    contact_groups        => team-data-platform,team-discovery

File[/lib/systemd/system/opensearch-disable-readahead-production-search-psi-codfw.service]

Parameters differences:

--- File[/lib/systemd/system/opensearch-disable-readahead-production-search-psi-codfw.service].orig
+++ File[/lib/systemd/system/opensearch-disable-readahead-production-search-psi-codfw.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for opensearch-disable-readahead-production-search-psi-codfw.service (opensearch-disable-readahead-production-search-psi-codfw.service)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/opensearch-disable-readahead-production-search-psi-codfw.service.orig
+++ /lib/systemd/system/opensearch-disable-readahead-production-search-psi-codfw.service
@@ -0,0 +1,8 @@
+[Unit]
+Description=Disables readahead on all open files every 30 minutes to alleviate Cirrussearch / opensearch IO load spikes
+Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+
+[Service]
+Type=oneshot
+User=root
+ExecStart=/usr/local/bin/opensearch-disable-readahead.sh production-search-psi-codfw /srv/opensearch

File[/lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9200.timer]

Parameters differences:

--- File[/lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9200.timer].orig
+++ File[/lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9200.timer]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9200.timer (nrpe2nodexp-opensearch_shard_size_check_9200.timer)]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9200.timer.orig
+++ /lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9200.timer
@@ -0,0 +1,13 @@
+[Unit]
+Description=Periodic execution of nrpe2nodexp-opensearch_shard_size_check_9200.service
+
+[Timer]
+Unit=nrpe2nodexp-opensearch_shard_size_check_9200.service
+# Accuracy sets the maximum time interval around the execution time we want to allow
+AccuracySec=15sec
+OnUnitInactiveSec=1440min
+OnActiveSec=1s
+RandomizedDelaySec=0
+
+[Install]
+WantedBy=multi-user.target

File[/etc/nginx/conf.d]

Parameters differences:

--- File[/etc/nginx/conf.d].orig
+++ File[/etc/nginx/conf.d]

+    purge   => True
+    force   => True
+    before  => ['Service[nginx]']
+    owner   => root
+    recurse => True
+    ensure  => directory
+    group   => root

Logrotate::Rule[opensearch]

Parameters differences:

--- Logrotate::Rule[opensearch].orig
+++ Logrotate::Rule[opensearch]

+    date_ext       => False
+    no_create      => False
+    file_glob      => /var/log/opensearch/*.log
+    compress       => True
+    date_yesterday => False
+    rotate         => 7
+    post_rotate    => []
+    ensure         => present
+    frequency      => daily
+    missing_ok     => True
+    copy_truncate  => True
+    not_if_empty   => True

Systemd::Timer::Job[opensearch-hot-threads-log]

Parameters differences:

--- Systemd::Timer::Job[opensearch-hot-threads-log].orig
+++ Systemd::Timer::Job[opensearch-hot-threads-log]

+    description               => Archive exception logs of hot opensearch threads
+    syslog_match_startswith   => True
+    monitoring_contact_groups => admins
+    logging_enabled           => False
+    success_exit_status       => []
+    require                   => ['Package[opensearch]', 'File[/usr/local/bin/opensearch_hot_threads_logger.py]']
+    fixed_random_delay        => False
+    private_tmp               => False
+    logfile_basedir           => /var/log
+    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+    send_mail_only_on_error   => True
+    ignore_errors             => False
+    logfile_group             => root
+    interval                  => {'start': 'OnCalendar', 'interval': '*-*-* *:00/5:00'}
+    send_mail                 => False
+    logfile_name              => syslog.log
+    user                      => opensearch
+    ensure                    => present
+    send_mail_to              => root@cirrussearch2089.codfw.wmnet
+    logfile_perms             => all
+    syslog_force_stop         => True
+    environment               => {}
+    command                   => /usr/local/bin/opensearch_hot_threads_logger.py
+    monitoring_enabled        => False

File[/etc/rsyslog.d/40-prometheus-lvs-realserver-mss.conf]

Parameters differences:

--- File[/etc/rsyslog.d/40-prometheus-lvs-realserver-mss.conf].orig
+++ File[/etc/rsyslog.d/40-prometheus-lvs-realserver-mss.conf]

+    mode   => 0444
+    notify => Service[rsyslog]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/rsyslog.d/40-prometheus-lvs-realserver-mss.conf.orig
+++ /etc/rsyslog.d/40-prometheus-lvs-realserver-mss.conf
@@ -0,0 +1,10 @@
+# rsyslog.conf(5) configuration file for services.
+# This file is managed by Puppet.
+if $programname startswith "prometheus_lvs_realserver_mss" then {
+    action(
+        type="omfile" file="/var/log/prometheus_lvs_realserver_mss/syslog.log"
+        fileOwner="root" fileGroup="root"
+        fileCreateMode="0644"
+    )
+    & stop
+}

File[/lib/systemd/system/prometheus_lvs_realserver_mss.service]

Parameters differences:

--- File[/lib/systemd/system/prometheus_lvs_realserver_mss.service].orig
+++ File[/lib/systemd/system/prometheus_lvs_realserver_mss.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for prometheus_lvs_realserver_mss.service (prometheus_lvs_realserver_mss.service)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/prometheus_lvs_realserver_mss.service.orig
+++ /lib/systemd/system/prometheus_lvs_realserver_mss.service
@@ -0,0 +1,8 @@
+[Unit]
+Description=Regular job to collect MSS values of realserver endpoints
+Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+
+[Service]
+Type=oneshot
+User=root
+ExecStart=/usr/local/bin/prometheus-lvs-realserver-mss -o /var/lib/prometheus/node.d/lvs-realserver-mss.prom -e 10.2.1.30:9200 -e 10.2.1.30:9243 -e 10.2.1.30:9643

Systemd::Unit[prometheus-elasticsearch-exporter-9200]

Parameters differences:

--- Systemd::Unit[prometheus-elasticsearch-exporter-9200].orig
+++ Systemd::Unit[prometheus-elasticsearch-exporter-9200]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => prometheus-elasticsearch-exporter-9200
+    ensure            => present

Systemd::Timer::Job[clean_up_dlq]

Parameters differences:

--- Systemd::Timer::Job[clean_up_dlq].orig
+++ Systemd::Timer::Job[clean_up_dlq]

+    description               => Clean up dead letter queue and restart logstash
+    syslog_match_startswith   => True
+    monitoring_contact_groups => admins
+    logging_enabled           => True
+    success_exit_status       => []
+    fixed_random_delay        => False
+    private_tmp               => False
+    logfile_basedir           => /var/log
+    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+    send_mail_only_on_error   => True
+    ignore_errors             => False
+    logfile_group             => root
+    interval                  => {'start': 'OnCalendar', 'interval': '1h'}
+    send_mail                 => False
+    logfile_name              => syslog.log
+    user                      => root
+    ensure                    => absent
+    send_mail_to              => root@cirrussearch2089.codfw.wmnet
+    logfile_perms             => all
+    syslog_force_stop         => True
+    environment               => {}
+    command                   => /usr/local/bin/cleanup-dlq
+    monitoring_enabled        => False

Class[Profile::Java]

Parameters differences:

--- Class[Profile::Java].orig
+++ Class[Profile::Java]

+    extra_args      => {}
+    egd_source      => /dev/random
+    enable_dbg      => False
+    java_packages   => []
+    trust_puppet_ca => True
+    hardened_tls    => False

Nrpe::Monitor_service[opensearch_unassigned_shard_check_9600]

Parameters differences:

--- Nrpe::Monitor_service[opensearch_unassigned_shard_check_9600].orig
+++ Nrpe::Monitor_service[opensearch_unassigned_shard_check_9600]

+    timeout                     => 10
+    nrpe2nodexp_parse_perf_data => False
+    retry_interval              => 120
+    description                 => OpenSearch unassigned shard check - 9600
+    notes_url                   => https://wikitech.wikimedia.org/wiki/Search#Administration
+    critical                    => False
+    retries                     => 1
+    migration_task              => T371083
+    ensure                      => present
+    enable_nrpe2nodexp          => False
+    enable_icinga_check         => True
+    check_interval              => 720
+    contact_group               => admins,team-discovery
+    nrpe_command                => /usr/lib/nagios/plugins/check_elasticsearch_unassigned_shards.py --url http://localhost:9600 --timeout 4

Interface::Clsact[clsact_lo]

Parameters differences:

--- Interface::Clsact[clsact_lo].orig
+++ Interface::Clsact[clsact_lo]

+    interface => lo
+    ensure    => absent

Exec[ip link add name ipip60 type ip6tnl external]

Parameters differences:

--- Exec[ip link add name ipip60 type ip6tnl external].orig
+++ Exec[ip link add name ipip60 type ip6tnl external]

+    unless  => ip link show ipip60
+    path    => /bin:/usr/bin
+    returns => [0, 2]

Systemd::Unit[prometheus_ferm_mss.timer]

Parameters differences:

--- Systemd::Unit[prometheus_ferm_mss.timer].orig
+++ Systemd::Unit[prometheus_ferm_mss.timer]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => prometheus_ferm_mss.timer
+    ensure            => present

File[/srv/opensearch/production-search-psi-codfw]

Parameters differences:

--- File[/srv/opensearch/production-search-psi-codfw].orig
+++ File[/srv/opensearch/production-search-psi-codfw]

+    mode   => 0755
+    group  => opensearch
+    owner  => opensearch
+    ensure => directory

Rsyslog::Conf[nrpe2nodexp-opensearch_shards_9200]

Parameters differences:

--- Rsyslog::Conf[nrpe2nodexp-opensearch_shards_9200].orig
+++ Rsyslog::Conf[nrpe2nodexp-opensearch_shards_9200]

+    mode     => 0444
+    priority => 25
+    ensure   => absent

Rsyslog::Conf[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]

Parameters differences:

--- Rsyslog::Conf[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600].orig
+++ Rsyslog::Conf[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]

+    mode     => 0444
+    require  => File[/var/log/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]
+    priority => 40
+    ensure   => present

File[/lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9200.service]

Parameters differences:

--- File[/lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9200.service].orig
+++ File[/lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9200.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9200.service (nrpe2nodexp-opensearch_unassigned_shard_check_9200.service)]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9200.service.orig
+++ /lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9200.service
@@ -0,0 +1,11 @@
+[Unit]
+Description=execution of nrpe2nodexp for the check_opensearch_unassigned_shard_check_9200 command.
+Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+
+[Service]
+Type=oneshot
+User=nagios
+
+Group=prometheus-node-exporter
+SyslogIdentifier=nrpe2nodexp-opensearch_unassigned_shard_check_9200
+ExecStart=-/usr/local/bin/nrpe2nodexp --alert-rule-hash "f98d7b74d9467ea0aa193a52c18e30ab" --timeout 10 --check-command "check_opensearch_unassigned_shard_check_9200"

File[/etc/conftool/local_services.yaml]

Parameters differences:

--- File[/etc/conftool/local_services.yaml].orig
+++ File[/etc/conftool/local_services.yaml]

+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/conftool/local_services.yaml.orig
+++ /etc/conftool/local_services.yaml
@@ -0,0 +1,19 @@
+---
+search:
+  cluster: elasticsearch
+  service: elasticsearch
+  servers:
+  - pybal-low-traffic.svc.codfw.wmnet
+  port: 9200
+search-https:
+  cluster: elasticsearch
+  service: elasticsearch-ssl
+  servers:
+  - pybal-low-traffic.svc.codfw.wmnet
+  port: 9243
+search-psi-https:
+  cluster: elasticsearch
+  service: elasticsearch-psi-ssl
+  servers:
+  - pybal-low-traffic.svc.codfw.wmnet
+  port: 9643

Nginx::Site[production-search-codfw]

Parameters differences:

--- Nginx::Site[production-search-codfw].orig
+++ Nginx::Site[production-search-codfw]

+    require => Exec[tlsproxy localssl default_server on ports [9243]]
+    ensure  => present

Service[nrpe2nodexp-check_tcp-mss-clamper_status.timer]

Parameters differences:

--- Service[nrpe2nodexp-check_tcp-mss-clamper_status.timer].orig
+++ Service[nrpe2nodexp-check_tcp-mss-clamper_status.timer]

+    before   => ['Exec[systemd daemon-reload for nrpe2nodexp-check_tcp-mss-clamper_status.timer (nrpe2nodexp-check_tcp-mss-clamper_status.timer)]']
+    provider => systemd
+    enable   => False
+    ensure   => stopped

Logrotate::Conf[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]

Parameters differences:

--- Logrotate::Conf[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200].orig
+++ Logrotate::Conf[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]

+    ensure => present

File[/var/log/opensearch/opensearch_index_search_slowlog.log]

Parameters differences:

--- File[/var/log/opensearch/opensearch_index_search_slowlog.log].orig
+++ File[/var/log/opensearch/opensearch_index_search_slowlog.log]

+    group  => root
+    owner  => root
+    ensure => absent

Class[Profile::Lvs::Realserver]

Parameters differences:

--- Class[Profile::Lvs::Realserver].orig
+++ Class[Profile::Lvs::Realserver]

+    pools                => {'search': {'services': ['opensearch_1@production-search-codfw']}, 'search-https': {'services': ['nginx', 'opensearch_1@production-search-codfw']}, 'search-psi-https': {'services': ['nginx', 'opensearch_1@production-search-psi-codfw']}}
+    use_conftool         => True
+    require              => ['Class[Profile::Conftool::Client]']
+    poolcounter_backends => [{'label': 'pc1', 'fqdn': 'poolcounter2005.codfw.wmnet'}, {'label': 'pc2', 'fqdn': 'poolcounter2006.codfw.wmnet'}]

Service[opensearch-disable-readahead.timer]

Parameters differences:

--- Service[opensearch-disable-readahead.timer].orig
+++ Service[opensearch-disable-readahead.timer]

+    before   => ['Exec[systemd daemon-reload for opensearch-disable-readahead.timer (opensearch-disable-readahead.timer)]']
+    provider => systemd
+    enable   => False
+    ensure   => stopped

Class[Logstash]

Parameters differences:

--- Class[Logstash].orig
+++ Class[Logstash]

+    pipeline_batch_size  => 125
+    heap_memory          => 192m
+    enable_dlq           => False
+    logstash_package     => logstash-oss
+    java_package         => openjdk-11-jdk
+    pipeline_batch_delay => 50
+    pipeline_workers     => 2
+    dlq_hosts            => []
+    log_format           => json
+    gc_log               => True
+    manage_service       => True
+    logstash_version     => 6
+    dlq_max_bytes        => 1024mb

File[/lib/systemd/system/prometheus-wmf-elasticsearch-exporter-9200.service]

Parameters differences:

--- File[/lib/systemd/system/prometheus-wmf-elasticsearch-exporter-9200.service].orig
+++ File[/lib/systemd/system/prometheus-wmf-elasticsearch-exporter-9200.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for prometheus-wmf-elasticsearch-exporter-9200.service (prometheus-wmf-elasticsearch-exporter-9200)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/prometheus-wmf-elasticsearch-exporter-9200.service.orig
+++ /lib/systemd/system/prometheus-wmf-elasticsearch-exporter-9200.service
@@ -0,0 +1,13 @@
+[Unit]
+Description=Prometheus WMF Elasticsearch Exporter
+
+[Service]
+Restart=always
+User=prometheus
+ExecStart=/usr/local/bin/prometheus-wmf-elasticsearch-exporter \
+    --listen :9120 \
+    --port 9200 \
+    --indices commonswiki_file wikidatawiki_content enwiki_general enwiki_content viwiki_general commonswiki_general cebwiki_content metawiki_general dewiki_content frwiki_content
+
+[Install]
+WantedBy=multi-user.target

Opensearch::Keystore[s3-credentials-user-production-search-codfw]

Parameters differences:

--- Opensearch::Keystore[s3-credentials-user-production-search-codfw].orig
+++ Opensearch::Keystore[s3-credentials-user-production-search-codfw]

+    value        => search:platform
+    cluster_name => production-search-codfw
+    key          => s3.client.default.access_key

File[/usr/lib/nagios/plugins/check_elasticsearch.py]

Parameters differences:

--- File[/usr/lib/nagios/plugins/check_elasticsearch.py].orig
+++ File[/usr/lib/nagios/plugins/check_elasticsearch.py]

+    mode   => 0755
+    group  => root
+    owner  => root
+    source => puppet:///modules/icinga/elasticsearch/check_elasticsearch.py

Rsyslog::Conf[wmf_auto_restart_prometheus-elasticsearch-exporter-9600]

Parameters differences:

--- Rsyslog::Conf[wmf_auto_restart_prometheus-elasticsearch-exporter-9600].orig
+++ Rsyslog::Conf[wmf_auto_restart_prometheus-elasticsearch-exporter-9600]

+    mode     => 0444
+    require  => File[/var/log/wmf_auto_restart_prometheus-elasticsearch-exporter-9600]
+    priority => 40
+    ensure   => present

Systemd::Service[prometheus-wmf-elasticsearch-exporter-9600]

Parameters differences:

--- Systemd::Service[prometheus-wmf-elasticsearch-exporter-9600].orig
+++ Systemd::Service[prometheus-wmf-elasticsearch-exporter-9600]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => present
+    monitoring_critical      => False
+    require                  => File[/usr/local/bin/prometheus-wmf-elasticsearch-exporter]
+    subscribe                => File[/usr/local/bin/prometheus-wmf-elasticsearch-exporter]
+    monitoring_enabled       => False
+    unit_type                => service
+    restart                  => True

File[/etc/opensearch_hot_threads.d/production-search-psi-codfw.yml]

Parameters differences:

--- File[/etc/opensearch_hot_threads.d/production-search-psi-codfw.yml].orig
+++ File[/etc/opensearch_hot_threads.d/production-search-psi-codfw.yml]

+    mode   => 0755
+    group  => root
+    owner  => root
+    ensure => file

Content differences:

--- /etc/opensearch_hot_threads.d/production-search-psi-codfw.yml.orig
+++ /etc/opensearch_hot_threads.d/production-search-psi-codfw.yml
@@ -0,0 +1,3 @@
+---
+port: 9600
+log_file: "/var/log/opensearch/opensearch_hot_threads-production-search-psi-codfw.log"

Exec[systemd daemon-reload for opensearch-disable-readahead-production-search-psi-codfw.timer (opensearch-disable-readahead-production-search-psi-codfw.timer)]

Parameters differences:

--- Exec[systemd daemon-reload for opensearch-disable-readahead-production-search-psi-codfw.timer (opensearch-disable-readahead-production-search-psi-codfw.timer)].orig
+++ Exec[systemd daemon-reload for opensearch-disable-readahead-production-search-psi-codfw.timer (opensearch-disable-readahead-production-search-psi-codfw.timer)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload
+    before      => ['Service[opensearch-disable-readahead-production-search-psi-codfw.timer]']

File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chain.pem]

Parameters differences:

--- File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chain.pem].orig
+++ File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chain.pem]

+    mode   => 0440
+    source => puppet:///modules/profile/pki/intermediates/discovery-cert.pem
+    notify => ['Exec[nginx-reload]', 'Exec[nginx-reload]']
+    group  => root
+    owner  => root
+    ensure => file

Systemd::Unit[tcp-mss-clamper]

Parameters differences:

--- Systemd::Unit[tcp-mss-clamper].orig
+++ Systemd::Unit[tcp-mss-clamper]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => tcp-mss-clamper
+    ensure            => absent

Class[Profile::Conftool::Client]

Parameters differences:

--- Class[Profile::Conftool::Client].orig
+++ Class[Profile::Conftool::Client]

+    conftool2git_host      => puppetserver1003.eqiad.wmnet
+    tcpircbot_host         => icinga.wikimedia.org
+    conftool2git_bind_addr => 0.0.0.0:1312
+    tcpircbot_port         => 9200
+    namespace              => /conftool
+    etcd_user              => __auto__
+    require                => ['Class[Passwords::Etcd]']
+    pool_pwd_seed          => 21}@/
+    srv_domain             => conftool.codfw.wmnet

Interface::Post_up_command[clsact_lo]

Parameters differences:

--- Interface::Post_up_command[clsact_lo].orig
+++ Interface::Post_up_command[clsact_lo]

+    command   => /usr/sbin/tc qdisc add dev lo clsact
+    interface => lo
+    ensure    => absent

Exec[Generate cert discovery__cirrussearch2089_codfw_wmnet]

Parameters differences:

--- Exec[Generate cert discovery__cirrussearch2089_codfw_wmnet].orig
+++ Exec[Generate cert discovery__cirrussearch2089_codfw_wmnet]

+    environment => ['GODEBUG=x509ignoreCN=0']
+    unless      => /usr/bin/test "$(/usr/bin/openssl x509 -in /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.pem -noout -pubkey 2>&1)" == "$(/usr/bin/openssl pkey -pubout -in /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet-key.pem 2>&1)"

+    command     => /usr/bin/cfssl gencert -config /etc/cfssl/client-cfssl.conf -tls-remote-ca /etc/ssl/certs/wmf-ca-certificates.crt -mutual-tls-client-cert /etc/cfssl/mutual_tls_client_cert.pem -mutual-tls-client-key /var/lib/puppet/ssl/private_keys/cirrussearch2089.codfw.wmnet.pem -label discovery  /etc/cfssl/csr/discovery__cirrussearch2089_codfw_wmnet.csr | /usr/bin/cfssljson -bare /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet

+    require     => Cfssl::Csr[/etc/cfssl/csr/discovery__cirrussearch2089_codfw_wmnet.csr]

File[/lib/systemd/system/opensearch-disable-readahead.service]

Parameters differences:

--- File[/lib/systemd/system/opensearch-disable-readahead.service].orig
+++ File[/lib/systemd/system/opensearch-disable-readahead.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for opensearch-disable-readahead.service (opensearch-disable-readahead.service)]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /lib/systemd/system/opensearch-disable-readahead.service.orig
+++ /lib/systemd/system/opensearch-disable-readahead.service
@@ -0,0 +1,8 @@
+[Unit]
+Description=Disables readahead on all open files every 30 minutes to alleviate Cirrussearch / opensearch IO load spikes
+Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+
+[Service]
+Type=oneshot
+User=root
+ExecStart=/usr/local/bin/opensearch-disable-readahead.sh

File[/etc/default/logstash]

Parameters differences:

--- File[/etc/default/logstash].orig
+++ File[/etc/default/logstash]

+    mode    => 0444
+    require => Package[logstash]
+    notify  => Service[logstash]
+    group   => root
+    owner   => root

File[/etc/ferm/conf.d/10_opensearch-https-9243]

Parameters differences:

--- File[/etc/ferm/conf.d/10_opensearch-https-9243].orig
+++ File[/etc/ferm/conf.d/10_opensearch-https-9243]

+    mode    => 0400
+    require => File[/etc/ferm/conf.d]
+    tag     => ferm
+    notify  => Service[ferm]
+    group   => root
+    owner   => root
+    ensure  => present

Content differences:

--- /etc/ferm/conf.d/10_opensearch-https-9243.orig
+++ /etc/ferm/conf.d/10_opensearch-https-9243
@@ -0,0 +1,6 @@
+# Autogenerated by puppet. DO NOT EDIT BY HAND!
+#
+# 
+&R_SERVICE(tcp, 9243, $DOMAIN_NETWORKS);
+
+

Ferm::Rule[ip6ip6]

Parameters differences:

--- Ferm::Rule[ip6ip6].orig
+++ Ferm::Rule[ip6ip6]

+    prio   => 10
+    desc   => 
+    table  => filter
+    rule   => saddr 0100::/64 proto ipv6 ACCEPT;
+    chain  => INPUT
+    ensure => present
+    domain => (ip6)

File[/var/log/opensearch-production-search-codfw-gc-log-cleanup]

Parameters differences:

--- File[/var/log/opensearch-production-search-codfw-gc-log-cleanup].orig
+++ File[/var/log/opensearch-production-search-codfw-gc-log-cleanup]

+    mode   => 0755
+    backup => False
+    force  => True
+    owner  => root
+    ensure => directory
+    group  => root

Systemd::Unit[opensearch-disable-readahead.timer]

Parameters differences:

--- Systemd::Unit[opensearch-disable-readahead.timer].orig
+++ Systemd::Unit[opensearch-disable-readahead.timer]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => opensearch-disable-readahead.timer
+    ensure            => absent

Monitoring::Service[disk_space]

Parameters differences:

--- Monitoring::Service[disk_space].orig
+++ Monitoring::Service[disk_space]

@@
-    contact_group => admins
+    contact_group => team-data-platform,team-discovery

Systemd::Unit[wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer]

Parameters differences:

--- Systemd::Unit[wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer].orig
+++ Systemd::Unit[wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer
+    ensure            => present

Interface::Manual[ipip_ipv6]

Parameters differences:

--- Interface::Manual[ipip_ipv6].orig
+++ Interface::Manual[ipip_ipv6]

+    hotplug   => False
+    family    => inet6
+    interface => ipip60
+    ensure    => present

Monitoring::Service[raid_broadcom_raid]

Parameters differences:

--- Monitoring::Service[raid_broadcom_raid].orig
+++ Monitoring::Service[raid_broadcom_raid]

@@
-    contact_group => admins
+    contact_group => team-data-platform,team-discovery

Prometheus::Alert::Rule[check_opensearch_shards_9600_7811fe11f3315f2bd38a6e6de0f1b6b0]

Parameters differences:

--- Prometheus::Alert::Rule[check_opensearch_shards_9600_7811fe11f3315f2bd38a6e6de0f1b6b0].orig
+++ Prometheus::Alert::Rule[check_opensearch_shards_9600_7811fe11f3315f2bd38a6e6de0f1b6b0]

+    dashboard          => TODO
+    expr               => (nagios_nrpe_check_result{alert_rule_hash="7811fe11f3315f2bd38a6e6de0f1b6b0",check_name="check_opensearch_shards_9600", status=~"(WARNING|CRITICAL)", severity=~"(warning|critical)"} > 0) * on (instance) group_left (team) role_owner
+    team               => observability
+    description        => OpenSearch health check for shards on 9600
+    runbook            => https://wikitech.wikimedia.org/wiki/Search#Administration
+    def_label_whitelst => ['team', 'severity']
+    ensure             => absent
+    logs               => https://logstash.wikimedia.org/app/dashboards#/view/2d343ac0-6df8-11f0-8e08-7fab0da52b33?_g=(filters:!((query:(match_phrase:(event.module:check_opensearch_shards_9600))),(query:(match_phrase:(host.name:{{$labels.instance|stripPort}})))))
+    severity           => info
+    summary            => OpenSearch health check for shards on 9600
+    instance           => ops
+    group              => nrpechecks
+    alert_name         => check_opensearch_shards_9600
+    for                => 1m
+    site               => codfw

Service[prometheus-elasticsearch-exporter-9200]

Parameters differences:

--- Service[prometheus-elasticsearch-exporter-9200].orig
+++ Service[prometheus-elasticsearch-exporter-9200]

+    enable => True
+    ensure => running

Class[Opensearch]

Parameters differences:

--- Class[Opensearch].orig
+++ Class[Opensearch]

+    instances               => {'production-search-codfw': {'awareness_attributes': 'row', 'plugins_mandatory': ['analysis-hebrew', 'analysis-icu', 'analysis-smartcn', 'analysis-stempel', 'analysis-ukrainian', 'cirrus-highlighter', 'opensearch-analysis-stconvert', 'opensearch-extra', 'opensearch-extra-analysis-esperanto', 'opensearch-extra-analysis-serbian', 'opensearch-extra-analysis-slovak', 'ltr'], 'heap_memory': '30G', 'recover_after_time': '5m', 'search_shard_count_limit': 5000, 'ltr_cache_size': '100mb', 'auto_create_index': '+apifeatureusage-*,+glent_*,-*', 'script_max_compilations_per_minute': 10000, 'bulk_thread_pool_executors': 6, 'bulk_thread_pool_capacity': 1000, 'filter_cache_size': '20%', 'cluster_name': 'production-search-codfw', 'short_cluster_name': 'chi', 'http_port': 9200, 'tls_port': 9243, 'certificate_name': 'search.discovery.wmnet', 'transport_tcp_port': 9300, 'unicast_hosts': ['cirrussearch2093.codfw.wmnet', 'cirrussearch2061.codfw.wmnet', 'cirrussearch2081.codfw.wmnet', 'cirrussearch2084.codfw.wmnet', 'cirrussearch2111.codfw.wmnet'], 'cluster_hosts': ['cirrussearch2061.codfw.wmnet', 'cirrussearch2062.codfw.wmnet', 'cirrussearch2063.codfw.wmnet', 'cirrussearch2064.codfw.wmnet', 'cirrussearch2065.codfw.wmnet', 'cirrussearch2066.codfw.wmnet', 'cirrussearch2067.codfw.wmnet', 'cirrussearch2068.codfw.wmnet', 'cirrussearch2069.codfw.wmnet', 'cirrussearch2070.codfw.wmnet', 'cirrussearch2071.codfw.wmnet', 'cirrussearch2072.codfw.wmnet', 'cirrussearch2073.codfw.wmnet', 'cirrussearch2074.codfw.wmnet', 'cirrussearch2075.codfw.wmnet', 'cirrussearch2076.codfw.wmnet', 'cirrussearch2077.codfw.wmnet', 'cirrussearch2078.codfw.wmnet', 'cirrussearch2079.codfw.wmnet', 'cirrussearch2080.codfw.wmnet', 'cirrussearch2081.codfw.wmnet', 'cirrussearch2082.codfw.wmnet', 'cirrussearch2083.codfw.wmnet', 'cirrussearch2084.codfw.wmnet', 'cirrussearch2085.codfw.wmnet', 'cirrussearch2086.codfw.wmnet', 'cirrussearch2087.codfw.wmnet', 'cirrussearch2088.codfw.wmnet', 'cirrussearch2089.codfw.wmnet', 'cirrussearch2090.codfw.wmnet', 'cirrussearch2091.codfw.wmnet', 'cirrussearch2092.codfw.wmnet', 'cirrussearch2093.codfw.wmnet', 'cirrussearch2094.codfw.wmnet', 'cirrussearch2095.codfw.wmnet', 'cirrussearch2096.codfw.wmnet', 'cirrussearch2097.codfw.wmnet', 'cirrussearch2098.codfw.wmnet', 'cirrussearch2099.codfw.wmnet', 'cirrussearch2100.codfw.wmnet', 'cirrussearch2101.codfw.wmnet', 'cirrussearch2102.codfw.wmnet', 'cirrussearch2103.codfw.wmnet', 'cirrussearch2104.codfw.wmnet', 'cirrussearch2105.codfw.wmnet', 'cirrussearch2106.codfw.wmnet', 'cirrussearch2107.codfw.wmnet', 'cirrussearch2108.codfw.wmnet', 'cirrussearch2109.codfw.wmnet', 'cirrussearch2110.codfw.wmnet', 'cirrussearch2111.codfw.wmnet', 'cirrussearch2112.codfw.wmnet', 'cirrussearch2113.codfw.wmnet', 'cirrussearch2114.codfw.wmnet', 'cirrussearch2115.codfw.wmnet'], 'expected_nodes': 36, 'recover_after_nodes': 24, 'indices_to_monitor': ['commonswiki_file', 'wikidatawiki_content', 'enwiki_general', 'enwiki_content', 'viwiki_general', 'commonswiki_general', 'cebwiki_content', 'metawiki_general', 'dewiki_content', 'frwiki_content']}, 'production-search-psi-codfw': {'awareness_attributes': 'row', 'plugins_mandatory': ['analysis-hebrew', 'analysis-icu', 'analysis-smartcn', 'analysis-stempel', 'analysis-ukrainian', 'cirrus-highlighter', 'opensearch-analysis-stconvert', 'opensearch-extra', 'opensearch-extra-analysis-esperanto', 'opensearch-extra-analysis-serbian', 'opensearch-extra-analysis-slovak', 'ltr'], 'heap_memory': '10G', 'recover_after_time': '5m', 'search_shard_count_limit': 5000, 'ltr_cache_size': '100mb', 'auto_create_index': '+apifeatureusage-*,+glent_*,-*', 'script_max_compilations_per_minute': 10000, 'bulk_thread_pool_executors': 6, 'bulk_thread_pool_capacity': 1000, 'filter_cache_size': '20%', 'cluster_name': 'production-search-psi-codfw', 'short_cluster_name': 'psi', 'http_port': 9600, 'tls_port': 9643, 'certificate_name': 'search.discovery.wmnet', 'transport_tcp_port': 9700, 'unicast_hosts': ['cirrussearch2108.codfw.wmnet', 'cirrussearch2076.codfw.wmnet', 'cirrussearch2080.codfw.wmnet', 'cirrussearch2083.codfw.wmnet', 'cirrussearch2115.codfw.wmnet'], 'cluster_hosts': ['cirrussearch2089.codfw.wmnet', 'cirrussearch2062.codfw.wmnet', 'cirrussearch2069.codfw.wmnet', 'cirrussearch2075.codfw.wmnet', 'cirrussearch2076.codfw.wmnet', 'cirrussearch2091.codfw.wmnet', 'cirrussearch2094.codfw.wmnet', 'cirrussearch2095.codfw.wmnet', 'cirrussearch2064.codfw.wmnet', 'cirrussearch2078.codfw.wmnet', 'cirrussearch2079.codfw.wmnet', 'cirrussearch2080.codfw.wmnet', 'cirrussearch2096.codfw.wmnet', 'cirrussearch2110.codfw.wmnet', 'cirrussearch2101.codfw.wmnet', 'cirrussearch2066.codfw.wmnet', 'cirrussearch2071.codfw.wmnet', 'cirrussearch2072.codfw.wmnet', 'cirrussearch2082.codfw.wmnet', 'cirrussearch2083.codfw.wmnet', 'cirrussearch2103.codfw.wmnet', 'cirrussearch2113.codfw.wmnet', 'cirrussearch2107.codfw.wmnet', 'cirrussearch2108.codfw.wmnet', 'cirrussearch2068.codfw.wmnet', 'cirrussearch2085.codfw.wmnet', 'cirrussearch2109.codfw.wmnet', 'cirrussearch2115.codfw.wmnet'], 'expected_nodes': 18, 'recover_after_nodes': 11}}
+    java_home               => /usr/lib/jvm/java-11-openjdk-amd64
+    default_instance_params => {}
+    logstash_host           => localhost
+    version                 => 1
+    row                     => codfw-row-a
+    enable_curator          => False
+    logstash_logback_port   => 11514
+    base_data_dir           => /srv/opensearch
+    native_lib_path         => /usr/share/opensearch/plugins/opensearch-knn/lib
+    before                  => ['File[/usr/share/opensearch/plugins]']
+    rack                    => A4

Systemd::Unit[wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer]

Parameters differences:

--- Systemd::Unit[wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer].orig
+++ Systemd::Unit[wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer
+    ensure            => present

File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-shard-size-check-9600.conf]

Parameters differences:

--- File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-shard-size-check-9600.conf].orig
+++ File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-shard-size-check-9600.conf]

+    mode   => 0444
+    notify => Service[rsyslog]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /etc/rsyslog.d/25-nrpe2nodexp-opensearch-shard-size-check-9600.conf.orig
+++ /etc/rsyslog.d/25-nrpe2nodexp-opensearch-shard-size-check-9600.conf
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: Apache-2.0
+if $programname contains "nrpe2nodexp-opensearch_shard_size_check_9600" then {
+    if ($msg contains "\"ecs.version\": \"1.7.0\"") then {
+        # Send logs to kafka
+        set $.log_outputs = "kafka ecs_170 local";
+    } else {
+        # Filter out non-relevant nrpe2nodexp messages
+        stop
+    }
+}

Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9600.timer (nrpe2nodexp-opensearch_shard_size_check_9600.timer)]

Parameters differences:

--- Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9600.timer (nrpe2nodexp-opensearch_shard_size_check_9600.timer)].orig
+++ Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9600.timer (nrpe2nodexp-opensearch_shard_size_check_9600.timer)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

Systemd::Unit[nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer]

Parameters differences:

--- Systemd::Unit[nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer].orig
+++ Systemd::Unit[nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer
+    ensure            => absent

Logrotate::Conf[opensearch-production-search-codfw-gc-log-cleanup]

Parameters differences:

--- Logrotate::Conf[opensearch-production-search-codfw-gc-log-cleanup].orig
+++ Logrotate::Conf[opensearch-production-search-codfw-gc-log-cleanup]

+    ensure => present

Exec[systemd daemon-reload for opensearch-hot-threads-log.timer (opensearch-hot-threads-log.timer)]

Parameters differences:

--- Exec[systemd daemon-reload for opensearch-hot-threads-log.timer (opensearch-hot-threads-log.timer)].orig
+++ Exec[systemd daemon-reload for opensearch-hot-threads-log.timer (opensearch-hot-threads-log.timer)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload
+    before      => ['Service[opensearch-hot-threads-log.timer]']

Systemd::Unit[nrpe2nodexp-opensearch_shard_size_check_9200.timer]

Parameters differences:

--- Systemd::Unit[nrpe2nodexp-opensearch_shard_size_check_9200.timer].orig
+++ Systemd::Unit[nrpe2nodexp-opensearch_shard_size_check_9200.timer]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => nrpe2nodexp-opensearch_shard_size_check_9200.timer
+    ensure            => absent

File[/lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9200.service]

Parameters differences:

--- File[/lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9200.service].orig
+++ File[/lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9200.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9200.service (nrpe2nodexp-opensearch_shard_size_check_9200.service)]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9200.service.orig
+++ /lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9200.service
@@ -0,0 +1,11 @@
+[Unit]
+Description=execution of nrpe2nodexp for the check_opensearch_shard_size_check_9200 command.
+Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+
+[Service]
+Type=oneshot
+User=nagios
+
+Group=prometheus-node-exporter
+SyslogIdentifier=nrpe2nodexp-opensearch_shard_size_check_9200
+ExecStart=-/usr/local/bin/nrpe2nodexp --alert-rule-hash "ce424a305cb5464806cbf788bde4430a" --timeout 10 --check-command "check_opensearch_shard_size_check_9200"

File[/etc/opensearch/production-search-codfw/log4j2.properties]

Parameters differences:

--- File[/etc/opensearch/production-search-codfw/log4j2.properties].orig
+++ File[/etc/opensearch/production-search-codfw/log4j2.properties]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => file

Content differences:

--- /etc/opensearch/production-search-codfw/log4j2.properties.orig
+++ /etc/opensearch/production-search-codfw/log4j2.properties
@@ -0,0 +1,201 @@
+status = error
+
+# log action execution errors for easier debugging
+logger.action.name = org.opensearch.action
+logger.action.level = debug
+
+appender.console.type = Console
+appender.console.name = console
+appender.console.layout.type = PatternLayout
+appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %m%n
+
+######## Server JSON ############################
+appender.rolling.type = RollingFile
+appender.rolling.name = rolling
+appender.rolling.fileName = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}_server.json
+appender.rolling.layout.type = OpenSearchJsonLayout
+appender.rolling.layout.type_name = server
+
+appender.rolling.filePattern = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}-%d{yyyy-MM-dd}-%i.json.gz
+appender.rolling.policies.type = Policies
+appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
+appender.rolling.policies.time.interval = 1
+appender.rolling.policies.time.modulate = true
+appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
+appender.rolling.policies.size.size = 128MB
+appender.rolling.strategy.type = DefaultRolloverStrategy
+appender.rolling.strategy.fileIndex = nomax
+appender.rolling.strategy.action.type = Delete
+appender.rolling.strategy.action.basepath = ${sys:opensearch.logs.base_path}
+appender.rolling.strategy.action.condition.type = IfFileName
+appender.rolling.strategy.action.condition.glob = ${sys:opensearch.logs.cluster_name}-*
+appender.rolling.strategy.action.condition.nested_condition.type = IfAccumulatedFileSize
+appender.rolling.strategy.action.condition.nested_condition.exceeds = 2GB
+################################################
+######## Server -  old style pattern ###########
+appender.rolling_old.type = RollingFile
+appender.rolling_old.name = rolling_old
+appender.rolling_old.fileName = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}.log
+appender.rolling_old.layout.type = PatternLayout
+appender.rolling_old.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %m%n
+
+appender.rolling_old.filePattern = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}-%d{yyyy-MM-dd}-%i.log.gz
+appender.rolling_old.policies.type = Policies
+appender.rolling_old.policies.time.type = TimeBasedTriggeringPolicy
+appender.rolling_old.policies.time.interval = 1
+appender.rolling_old.policies.time.modulate = true
+appender.rolling_old.policies.size.type = SizeBasedTriggeringPolicy
+appender.rolling_old.policies.size.size = 128MB
+appender.rolling_old.strategy.type = DefaultRolloverStrategy
+appender.rolling_old.strategy.fileIndex = nomax
+appender.rolling_old.strategy.action.type = Delete
+appender.rolling_old.strategy.action.basepath = ${sys:opensearch.logs.base_path}
+appender.rolling_old.strategy.action.condition.type = IfFileName
+appender.rolling_old.strategy.action.condition.glob = ${sys:opensearch.logs.cluster_name}-*
+appender.rolling_old.strategy.action.condition.nested_condition.type = IfAccumulatedFileSize
+appender.rolling_old.strategy.action.condition.nested_condition.exceeds = 2GB
+################################################
+
+rootLogger.level = info
+rootLogger.appenderRef.console.ref = console
+rootLogger.appenderRef.rolling.ref = rolling
+rootLogger.appenderRef.rolling_old.ref = rolling_old
+
+######## Deprecation JSON #######################
+appender.deprecation_rolling.type = RollingFile
+appender.deprecation_rolling.name = deprecation_rolling
+appender.deprecation_rolling.fileName = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}_deprecation.json
+appender.deprecation_rolling.layout.type = OpenSearchJsonLayout
+appender.deprecation_rolling.layout.type_name = deprecation
+appender.deprecation_rolling.layout.opensearchmessagefields=x-opaque-id
+
+appender.deprecation_rolling.filePattern = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}_deprecation-%i.json.gz
+appender.deprecation_rolling.policies.type = Policies
+appender.deprecation_rolling.policies.size.type = SizeBasedTriggeringPolicy
+appender.deprecation_rolling.policies.size.size = 1GB
+appender.deprecation_rolling.strategy.type = DefaultRolloverStrategy
+appender.deprecation_rolling.strategy.max = 4
+#################################################
+######## Deprecation -  old style pattern #######
+appender.deprecation_rolling_old.type = RollingFile
+appender.deprecation_rolling_old.name = deprecation_rolling_old
+appender.deprecation_rolling_old.fileName = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}_deprecation.log
+appender.deprecation_rolling_old.layout.type = PatternLayout
+appender.deprecation_rolling_old.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %m%n
+
+appender.deprecation_rolling_old.filePattern = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}\
+  _deprecation-%i.log.gz
+appender.deprecation_rolling_old.policies.type = Policies
+appender.deprecation_rolling_old.policies.size.type = SizeBasedTriggeringPolicy
+appender.deprecation_rolling_old.policies.size.size = 1GB
+appender.deprecation_rolling_old.strategy.type = DefaultRolloverStrategy
+appender.deprecation_rolling_old.strategy.max = 4
+#################################################
+logger.deprecation.name = org.opensearch.deprecation
+logger.deprecation.level = warn
+logger.deprecation.appenderRef.deprecation_rolling.ref = deprecation_rolling
+logger.deprecation.appenderRef.deprecation_rolling_old.ref = deprecation_rolling_old
+logger.deprecation.additivity = false
+
+######## Search slowlog JSON ####################
+appender.index_search_slowlog_rolling.type = RollingFile
+appender.index_search_slowlog_rolling.name = index_search_slowlog_rolling
+appender.index_search_slowlog_rolling.fileName = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs\
+  .cluster_name}_index_search_slowlog.json
+appender.index_search_slowlog_rolling.layout.type = OpenSearchJsonLayout
+appender.index_search_slowlog_rolling.layout.type_name = index_search_slowlog
+appender.index_search_slowlog_rolling.layout.opensearchmessagefields=message,took,took_millis,total_hits,types,stats,search_type,total_shards,source,id
+
+appender.index_search_slowlog_rolling.filePattern = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs\
+  .cluster_name}_index_search_slowlog-%i.json.gz
+appender.index_search_slowlog_rolling.policies.type = Policies
+appender.index_search_slowlog_rolling.policies.size.type = SizeBasedTriggeringPolicy
+appender.index_search_slowlog_rolling.policies.size.size = 1GB
+appender.index_search_slowlog_rolling.strategy.type = DefaultRolloverStrategy
+appender.index_search_slowlog_rolling.strategy.max = 4
+#################################################
+######## Search slowlog -  old style pattern ####
+appender.index_search_slowlog_rolling_old.type = RollingFile
+appender.index_search_slowlog_rolling_old.name = index_search_slowlog_rolling_old
+appender.index_search_slowlog_rolling_old.fileName = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}\
+  _index_search_slowlog.log
+appender.index_search_slowlog_rolling_old.layout.type = PatternLayout
+appender.index_search_slowlog_rolling_old.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %m%n
+
+appender.index_search_slowlog_rolling_old.filePattern = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}\
+  _index_search_slowlog-%i.log.gz
+appender.index_search_slowlog_rolling_old.policies.type = Policies
+appender.index_search_slowlog_rolling_old.policies.size.type = SizeBasedTriggeringPolicy
+appender.index_search_slowlog_rolling_old.policies.size.size = 1GB
+appender.index_search_slowlog_rolling_old.strategy.type = DefaultRolloverStrategy
+appender.index_search_slowlog_rolling_old.strategy.max = 4
+#################################################
+logger.index_search_slowlog_rolling.name = index.search.slowlog
+logger.index_search_slowlog_rolling.level = trace
+logger.index_search_slowlog_rolling.appenderRef.index_search_slowlog_rolling.ref = index_search_slowlog_rolling
+logger.index_search_slowlog_rolling.appenderRef.index_search_slowlog_rolling_old.ref = index_search_slowlog_rolling_old
+logger.index_search_slowlog_rolling.additivity = false
+
+######## Indexing slowlog JSON ##################
+appender.index_indexing_slowlog_rolling.type = RollingFile
+appender.index_indexing_slowlog_rolling.name = index_indexing_slowlog_rolling
+appender.index_indexing_slowlog_rolling.fileName = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}\
+  _index_indexing_slowlog.json
+appender.index_indexing_slowlog_rolling.layout.type = OpenSearchJsonLayout
+appender.index_indexing_slowlog_rolling.layout.type_name = index_indexing_slowlog
+appender.index_indexing_slowlog_rolling.layout.opensearchmessagefields=message,took,took_millis,doc_type,id,routing,source
+
+appender.index_indexing_slowlog_rolling.filePattern = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}\
+  _index_indexing_slowlog-%i.json.gz
+appender.index_indexing_slowlog_rolling.policies.type = Policies
+appender.index_indexing_slowlog_rolling.policies.size.type = SizeBasedTriggeringPolicy
+appender.index_indexing_slowlog_rolling.policies.size.size = 1GB
+appender.index_indexing_slowlog_rolling.strategy.type = DefaultRolloverStrategy
+appender.index_indexing_slowlog_rolling.strategy.max = 4
+#################################################
+######## Indexing slowlog -  old style pattern ##
+appender.index_indexing_slowlog_rolling_old.type = RollingFile
+appender.index_indexing_slowlog_rolling_old.name = index_indexing_slowlog_rolling_old
+appender.index_indexing_slowlog_rolling_old.fileName = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}\
+  _index_indexing_slowlog.log
+appender.index_indexing_slowlog_rolling_old.layout.type = PatternLayout
+appender.index_indexing_slowlog_rolling_old.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %m%n
+
+appender.index_indexing_slowlog_rolling_old.filePattern = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}\
+  _index_indexing_slowlog-%i.log.gz
+appender.index_indexing_slowlog_rolling_old.policies.type = Policies
+appender.index_indexing_slowlog_rolling_old.policies.size.type = SizeBasedTriggeringPolicy
+appender.index_indexing_slowlog_rolling_old.policies.size.size = 1GB
+appender.index_indexing_slowlog_rolling_old.strategy.type = DefaultRolloverStrategy
+appender.index_indexing_slowlog_rolling_old.strategy.max = 4
+#################################################
+
+logger.index_indexing_slowlog.name = index.indexing.slowlog.index
+logger.index_indexing_slowlog.level = trace
+logger.index_indexing_slowlog.appenderRef.index_indexing_slowlog_rolling.ref = index_indexing_slowlog_rolling
+logger.index_indexing_slowlog.appenderRef.index_indexing_slowlog_rolling_old.ref = index_indexing_slowlog_rolling_old
+logger.index_indexing_slowlog.additivity = false
+
+
+### Custom settings
+monitorInterval=600
+
+# Add references to Logstash appender for interesting loggers
+
+rootLogger.appenderRef.ship_to_logstash.ref = ship_to_logstash
+logger.index_indexing_slowlog.appenderRef.ship_to_logstash.ref = ship_to_logstash
+logger.index_search_slowlog_rolling.appenderRef.ship_to_logstash.ref = ship_to_logstash
+logger.index_search_slowlog_rolling.appenderRef.ship_to_logstash.ref = ship_to_logstash
+
+# Define Logstash appender
+appender.ship_to_logstash.name=ship_to_logstash
+appender.ship_to_logstash.type=syslog
+appender.ship_to_logstash.protocol=udp
+appender.ship_to_logstash.facility=LOCAL0
+appender.ship_to_logstash.layout.type=OpenSearchJsonLayout
+appender.ship_to_logstash.layout.compact=true
+appender.ship_to_logstash.layout.complete=true
+appender.ship_to_logstash.host=localhost
+appender.ship_to_logstash.port=11514
+appender.ship_to_logstash.layout.type_name=opensearch
+

File[/etc/update-motd.d/96-cluster-memberships]

Parameters differences:

--- File[/etc/update-motd.d/96-cluster-memberships].orig
+++ File[/etc/update-motd.d/96-cluster-memberships]

+    mode   => 0555
+    source => puppet:///modules/opensearch/opensearch.motd
+    group  => root
+    owner  => root
+    ensure => present

File[/etc/nagios/nrpe.d/check_opensearch_shards_9200.cfg]

Parameters differences:

--- File[/etc/nagios/nrpe.d/check_opensearch_shards_9200.cfg].orig
+++ File[/etc/nagios/nrpe.d/check_opensearch_shards_9200.cfg]

+    mode    => 0444
+    require => Package[nagios-nrpe-server]
+    tag     => nrpe::check
+    notify  => Service[nagios-nrpe-server]
+    group   => root
+    owner   => root
+    ensure  => present

Content differences:

--- /etc/nagios/nrpe.d/check_opensearch_shards_9200.cfg.orig
+++ /etc/nagios/nrpe.d/check_opensearch_shards_9200.cfg
@@ -0,0 +1,2 @@
+# File generated by puppet. DO NOT edit by hand
+command[check_opensearch_shards_9200]=/usr/lib/nagios/plugins/check_elasticsearch.py --ignore-status --url http://localhost:9200 --shards-inactive '>=0.15' --timeout 4

File[/usr/local/bin/jheapdump]

Parameters differences:

--- File[/usr/local/bin/jheapdump].orig
+++ File[/usr/local/bin/jheapdump]

+    mode   => 0555
+    source => puppet:///modules/java/jheapdump
+    group  => root
+    owner  => root
+    ensure => file

Sudo::Group[elasticsearch-roots]

Parameters differences:

--- Sudo::Group[elasticsearch-roots].orig
+++ Sudo::Group[elasticsearch-roots]

+    group      => elasticsearch-roots
+    privileges => ['ALL = (ALL) NOPASSWD: ALL']
+    require    => ['Class[Sudo]']
+    ensure     => present

Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9200.service (nrpe2nodexp-opensearch_shard_size_check_9200.service)]

Parameters differences:

--- Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9200.service (nrpe2nodexp-opensearch_shard_size_check_9200.service)].orig
+++ Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9200.service (nrpe2nodexp-opensearch_shard_size_check_9200.service)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

Monitoring::Service[opensearch_shard_size_check_9600]

Parameters differences:

--- Monitoring::Service[opensearch_shard_size_check_9600].orig
+++ Monitoring::Service[opensearch_shard_size_check_9600]

+    check_command  => nrpe_check!check_opensearch_shard_size_check_9600!10
+    retry_interval => 180
+    description    => OpenSearch shard size check - 9600
+    critical       => False
+    notes_url      => https://wikitech.wikimedia.org/wiki/Search#If_it_has_been_indexed
+    retries        => 3
+    config_dir     => /etc/nagios
+    freshness      => 36000
+    migration_task => T321808
+    ensure         => present
+    check_interval => 1440
+    contact_group  => admins,team-discovery
+    host           => cirrussearch2089
+    passive        => False

Augeas[ipip60_manual]

Parameters differences:

--- Augeas[ipip60_manual].orig
+++ Augeas[ipip60_manual]

+    lens    => Interfaces.lns
+    incl    => /etc/network/interfaces
+    changes => ["set auto[./1 = 'ipip60']/1 'ipip60'", "set iface[. = 'ipip60'] 'ipip60'", "set iface[. = 'ipip60']/family 'inet6'", "set iface[. = 'ipip60']/method 'manual'"]
+    context => /files/etc/network/interfaces

Java::Cacert[wmf:Wikimedia_Internal_Root_CA]

Parameters differences:

--- Java::Cacert[wmf:Wikimedia_Internal_Root_CA].orig
+++ Java::Cacert[wmf:Wikimedia_Internal_Root_CA]

+    require   => Alternatives::Java[11]
+    path      => /usr/share/ca-certificates/wikimedia/Wikimedia_Internal_Root_CA.crt
+    group     => root
+    owner     => root
+    storepass => changeit
+    ensure    => present

Systemd::Timer::Job[nrpe2nodexp-opensearch_unassigned_shard_check_9600]

Parameters differences:

--- Systemd::Timer::Job[nrpe2nodexp-opensearch_unassigned_shard_check_9600].orig
+++ Systemd::Timer::Job[nrpe2nodexp-opensearch_unassigned_shard_check_9600]

+    description               => execution of nrpe2nodexp for the check_opensearch_unassigned_shard_check_9600 command.
+    syslog_match_startswith   => True
+    monitoring_contact_groups => admins
+    logging_enabled           => False
+    success_exit_status       => []
+    fixed_random_delay        => False
+    private_tmp               => False
+    logfile_basedir           => /var/log
+    group                     => prometheus-node-exporter
+    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+    syslog_identifier         => nrpe2nodexp-opensearch_unassigned_shard_check_9600
+    send_mail_only_on_error   => True
+    ignore_errors             => True
+    logfile_group             => root
+    interval                  => [{'start': 'OnUnitInactiveSec', 'interval': '720min'}]
+    send_mail                 => False
+    logfile_name              => syslog.log
+    user                      => nagios
+    ensure                    => absent
+    send_mail_to              => root@cirrussearch2089.codfw.wmnet
+    logfile_perms             => all
+    syslog_force_stop         => True
+    environment               => {}
+    command                   => /usr/local/bin/nrpe2nodexp --alert-rule-hash "8641ad857de0a06c2748b18ea2c135ab" --timeout 10 --check-command "check_opensearch_unassigned_shard_check_9600"
+    monitoring_enabled        => False

File[/etc/ferm/conf.d/10_opensearch-https-9643]

Parameters differences:

--- File[/etc/ferm/conf.d/10_opensearch-https-9643].orig
+++ File[/etc/ferm/conf.d/10_opensearch-https-9643]

+    mode    => 0400
+    require => File[/etc/ferm/conf.d]
+    tag     => ferm
+    notify  => Service[ferm]
+    group   => root
+    owner   => root
+    ensure  => present

Content differences:

--- /etc/ferm/conf.d/10_opensearch-https-9643.orig
+++ /etc/ferm/conf.d/10_opensearch-https-9643
@@ -0,0 +1,6 @@
+# Autogenerated by puppet. DO NOT EDIT BY HAND!
+#
+# 
+&R_SERVICE(tcp, 9643, $DOMAIN_NETWORKS);
+
+

Rsyslog::Conf[opensearch-disable-readahead-production-search-psi-codfw]

Parameters differences:

--- Rsyslog::Conf[opensearch-disable-readahead-production-search-psi-codfw].orig
+++ Rsyslog::Conf[opensearch-disable-readahead-production-search-psi-codfw]

+    mode     => 0444
+    require  => File[/var/log/opensearch-disable-readahead-production-search-psi-codfw]
+    priority => 40
+    ensure   => present

File[/lib/systemd/system/prometheus-elasticsearch-exporter-9600.service]

Parameters differences:

--- File[/lib/systemd/system/prometheus-elasticsearch-exporter-9600.service].orig
+++ File[/lib/systemd/system/prometheus-elasticsearch-exporter-9600.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for prometheus-elasticsearch-exporter-9600.service (prometheus-elasticsearch-exporter-9600)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/prometheus-elasticsearch-exporter-9600.service.orig
+++ /lib/systemd/system/prometheus-elasticsearch-exporter-9600.service
@@ -0,0 +1,11 @@
+[Unit]
+Description=Prometheus exporter for Elasticsearch
+Documentation=https://prometheus.io/docs/introduction/overview/
+
+[Service]
+Restart=always
+User=prometheus
+ExecStart=/usr/bin/prometheus-elasticsearch-exporter --es.uri=http://localhost:9600 --web.listen-address=:9109 
+
+[Install]
+WantedBy=multi-user.target

File[/etc/logstash/conf.d/10-input-gelf-gelf_relay.conf]

Parameters differences:

--- File[/etc/logstash/conf.d/10-input-gelf-gelf_relay.conf].orig
+++ File[/etc/logstash/conf.d/10-input-gelf-gelf_relay.conf]

+    mode         => 0440
+    validate_cmd => /usr/local/bin/logstash-config-test %
+    notify       => Service[logstash]
+    group        => logstash
+    owner        => logstash
+    ensure       => present

Content differences:

--- /etc/logstash/conf.d/10-input-gelf-gelf_relay.conf.orig
+++ /etc/logstash/conf.d/10-input-gelf-gelf_relay.conf
@@ -0,0 +1,11 @@
+input {
+  # Graylog extended logging format
+  gelf {
+    type  => "gelf"
+    port  => 12201
+    remap => false
+    host  => "localhost"
+    id   => "input/gelf/12201"
+    tags => ["input-gelf-relay"]
+  }
+}

Profile::Prometheus::Elasticsearch_exporter[cirrussearch2089:9200]

Parameters differences:

--- Profile::Prometheus::Elasticsearch_exporter[cirrussearch2089:9200].orig
+++ Profile::Prometheus::Elasticsearch_exporter[cirrussearch2089:9200]

+    extra_config       => 
+    prometheus_port    => 9108
+    elasticsearch_port => 9200

File[/etc/logrotate.d/wmf_auto_restart_prometheus-elasticsearch-exporter-9200]

Parameters differences:

--- File[/etc/logrotate.d/wmf_auto_restart_prometheus-elasticsearch-exporter-9200].orig
+++ File[/etc/logrotate.d/wmf_auto_restart_prometheus-elasticsearch-exporter-9200]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/logrotate.d/wmf_auto_restart_prometheus-elasticsearch-exporter-9200.orig
+++ /etc/logrotate.d/wmf_auto_restart_prometheus-elasticsearch-exporter-9200
@@ -0,0 +1,12 @@
+# logrotate(8) config for wmf_auto_restart_prometheus-elasticsearch-exporter-9200
+
+/var/log/wmf_auto_restart_prometheus-elasticsearch-exporter-9200/*.log {
+    daily
+    copytruncate
+    missingok
+    compress
+    delaycompress
+    notifempty
+    rotate 15
+    size 256M
+}

Systemd::Syslog[opensearch-production-search-codfw-gc-log-cleanup]

Parameters differences:

--- Systemd::Syslog[opensearch-production-search-codfw-gc-log-cleanup].orig
+++ Systemd::Syslog[opensearch-production-search-codfw-gc-log-cleanup]

+    programname_comparison => startswith
+    force_stop             => True
+    ensure                 => present
+    base_dir               => /var/log
+    readable_by            => all
+    log_filename           => syslog.log
+    owner                  => root
+    group                  => root

Rsyslog::Conf[nrpe2nodexp-opensearch_shard_size_check_9200]

Parameters differences:

--- Rsyslog::Conf[nrpe2nodexp-opensearch_shard_size_check_9200].orig
+++ Rsyslog::Conf[nrpe2nodexp-opensearch_shard_size_check_9200]

+    mode     => 0444
+    priority => 25
+    ensure   => absent

Exec[systemd daemon-reload for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer (wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer)]

Parameters differences:

--- Exec[systemd daemon-reload for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer (wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer)].orig
+++ Exec[systemd daemon-reload for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer (wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload
+    before      => ['Service[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer]']

Package[nginx-light]

Parameters differences:

--- Package[nginx-light].orig
+++ Package[nginx-light]

+    provider => apt
+    ensure   => installed
+    notify   => ['Systemd::Unmask[nginx.service]']

Systemd::Tmpfile[opensearch-production-search-codfw]

Parameters differences:

--- Systemd::Tmpfile[opensearch-production-search-codfw].orig
+++ Systemd::Tmpfile[opensearch-production-search-codfw]

+    group  => root
+    owner  => root
+    ensure => present

Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9200.timer (nrpe2nodexp-opensearch_shards_9200.timer)]

Parameters differences:

--- Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9200.timer (nrpe2nodexp-opensearch_shards_9200.timer)].orig
+++ Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9200.timer (nrpe2nodexp-opensearch_shards_9200.timer)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

Ferm::Service[opensearch-inter-node-9300]

Parameters differences:

--- Ferm::Service[opensearch-inter-node-9300].orig
+++ Ferm::Service[opensearch-inter-node-9300]

+    proto   => tcp
+    port    => 9300
+    prio    => 10
+    desc    => 
+    notrack => True
+    srange  => @resolve((cirrussearch2061.codfw.wmnet cirrussearch2062.codfw.wmnet cirrussearch2063.codfw.wmnet cirrussearch2064.codfw.wmnet cirrussearch2065.codfw.wmnet cirrussearch2066.codfw.wmnet cirrussearch2067.codfw.wmnet cirrussearch2068.codfw.wmnet cirrussearch2069.codfw.wmnet cirrussearch2070.codfw.wmnet cirrussearch2071.codfw.wmnet cirrussearch2072.codfw.wmnet cirrussearch2073.codfw.wmnet cirrussearch2074.codfw.wmnet cirrussearch2075.codfw.wmnet cirrussearch2076.codfw.wmnet cirrussearch2077.codfw.wmnet cirrussearch2078.codfw.wmnet cirrussearch2079.codfw.wmnet cirrussearch2080.codfw.wmnet cirrussearch2081.codfw.wmnet cirrussearch2082.codfw.wmnet cirrussearch2083.codfw.wmnet cirrussearch2084.codfw.wmnet cirrussearch2085.codfw.wmnet cirrussearch2086.codfw.wmnet cirrussearch2087.codfw.wmnet cirrussearch2088.codfw.wmnet cirrussearch2089.codfw.wmnet cirrussearch2090.codfw.wmnet cirrussearch2091.codfw.wmnet cirrussearch2092.codfw.wmnet cirrussearch2093.codfw.wmnet cirrussearch2094.codfw.wmnet cirrussearch2095.codfw.wmnet cirrussearch2096.codfw.wmnet cirrussearch2097.codfw.wmnet cirrussearch2098.codfw.wmnet cirrussearch2099.codfw.wmnet cirrussearch2100.codfw.wmnet cirrussearch2101.codfw.wmnet cirrussearch2102.codfw.wmnet cirrussearch2103.codfw.wmnet cirrussearch2104.codfw.wmnet cirrussearch2105.codfw.wmnet cirrussearch2106.codfw.wmnet cirrussearch2107.codfw.wmnet cirrussearch2108.codfw.wmnet cirrussearch2109.codfw.wmnet cirrussearch2110.codfw.wmnet cirrussearch2111.codfw.wmnet cirrussearch2112.codfw.wmnet cirrussearch2113.codfw.wmnet cirrussearch2114.codfw.wmnet cirrussearch2115.codfw.wmnet))
+    ensure  => present

Exec[unmask_nginx.service]

Parameters differences:

--- Exec[unmask_nginx.service].orig
+++ Exec[unmask_nginx.service]

+    refreshonly => True
+    onlyif      => /bin/readlink -f /etc/systemd/system/nginx.service | grep -q /dev/null
+    command     => /bin/systemctl unmask nginx.service

Augeas[ipip0_add_up]

Parameters differences:

--- Augeas[ipip0_add_up].orig
+++ Augeas[ipip0_add_up]

+    onlyif  => match up[. = 'ip link add name ipip0 type ipip external'] size == 0
+    context => /files/etc/network/interfaces/*[. = 'ipip0' and ./family = 'inet']
+    incl    => /etc/network/interfaces
+    require => Interface::Manual[ipip_ipv4]
+    lens    => Interfaces.lns
+    changes => set up[last()+1] 'ip link add name ipip0 type ipip external'

File[/usr/share/opensearch/plugins]

Parameters differences:

--- File[/usr/share/opensearch/plugins].orig
+++ File[/usr/share/opensearch/plugins]

+    mode   => 0755
+    force  => True
+    owner  => root
+    ensure => directory
+    group  => root

Class[Poolcounter::Client]

Parameters differences:

--- Class[Poolcounter::Client].orig
+++ Class[Poolcounter::Client]

+    backends => [{'label': 'pc1', 'fqdn': 'poolcounter2005.codfw.wmnet'}, {'label': 'pc2', 'fqdn': 'poolcounter2006.codfw.wmnet'}]
+    ensure   => present

File[/usr/local/sbin/restart-nginx]

Parameters differences:

--- File[/usr/local/sbin/restart-nginx].orig
+++ File[/usr/local/sbin/restart-nginx]

+    mode   => 0555
+    group  => root
+    before => ['Service[nginx]']
+    owner  => root
+    ensure => present

Content differences:

--- /usr/local/sbin/restart-nginx.orig
+++ /usr/local/sbin/restart-nginx
@@ -0,0 +1,2 @@
+#!/bin/bash
+/usr/local/bin/safe-service-restart --pools search-https search-psi-https --services nginx --retries 10 --wait 5 --max-concurrency 5 $@

File[/lib/systemd/system/prometheus-elasticsearch-exporter-9200.service]

Parameters differences:

--- File[/lib/systemd/system/prometheus-elasticsearch-exporter-9200.service].orig
+++ File[/lib/systemd/system/prometheus-elasticsearch-exporter-9200.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for prometheus-elasticsearch-exporter-9200.service (prometheus-elasticsearch-exporter-9200)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/prometheus-elasticsearch-exporter-9200.service.orig
+++ /lib/systemd/system/prometheus-elasticsearch-exporter-9200.service
@@ -0,0 +1,11 @@
+[Unit]
+Description=Prometheus exporter for Elasticsearch
+Documentation=https://prometheus.io/docs/introduction/overview/
+
+[Service]
+Restart=always
+User=prometheus
+ExecStart=/usr/bin/prometheus-elasticsearch-exporter --es.uri=http://localhost:9200 --web.listen-address=:9108 
+
+[Install]
+WantedBy=multi-user.target

File[/etc/logrotate.d/opensearch-production-search-codfw-gc-log-cleanup]

Parameters differences:

--- File[/etc/logrotate.d/opensearch-production-search-codfw-gc-log-cleanup].orig
+++ File[/etc/logrotate.d/opensearch-production-search-codfw-gc-log-cleanup]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/logrotate.d/opensearch-production-search-codfw-gc-log-cleanup.orig
+++ /etc/logrotate.d/opensearch-production-search-codfw-gc-log-cleanup
@@ -0,0 +1,12 @@
+# logrotate(8) config for opensearch-production-search-codfw-gc-log-cleanup
+
+/var/log/opensearch-production-search-codfw-gc-log-cleanup/*.log {
+    daily
+    copytruncate
+    missingok
+    compress
+    delaycompress
+    notifempty
+    rotate 15
+    size 256M
+}

Exec[systemd daemon-reload for wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer (wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer)]

Parameters differences:

--- Exec[systemd daemon-reload for wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer (wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer)].orig
+++ Exec[systemd daemon-reload for wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer (wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload
+    before      => ['Service[wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer]']

Systemd::Unit[wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service]

Parameters differences:

--- Systemd::Unit[wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service].orig
+++ Systemd::Unit[wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service
+    ensure            => present

Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9600.service (nrpe2nodexp-opensearch_shards_9600.service)]

Parameters differences:

--- Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9600.service (nrpe2nodexp-opensearch_shards_9600.service)].orig
+++ Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9600.service (nrpe2nodexp-opensearch_shards_9600.service)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

Exec[java__cacert_wmf:Wikimedia_Internal_Root_CA]

Parameters differences:

--- Exec[java__cacert_wmf:Wikimedia_Internal_Root_CA].orig
+++ Exec[java__cacert_wmf:Wikimedia_Internal_Root_CA]

+    unless  => /usr/bin/keytool -list -cacerts -noprompt -storepass changeit -alias wmf:Wikimedia_Internal_Root_CA
+    command => /usr/bin/keytool -import -trustcacerts -noprompt -cacerts     -file /usr/share/ca-certificates/wikimedia/Wikimedia_Internal_Root_CA.crt -storepass changeit -alias wmf:Wikimedia_Internal_Root_CA

+    user    => root
+    group   => root

Nrpe::Monitor_service[raid_md]

Parameters differences:

--- Nrpe::Monitor_service[raid_md].orig
+++ Nrpe::Monitor_service[raid_md]

@@
-    contact_group => admins
+    contact_group => team-data-platform,team-discovery

File[/etc/sysctl.d/70-TCP-Fast-Open.conf]

Parameters differences:

--- File[/etc/sysctl.d/70-TCP-Fast-Open.conf].orig
+++ File[/etc/sysctl.d/70-TCP-Fast-Open.conf]

+    notify => Exec[update_sysctl]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/sysctl.d/70-TCP-Fast-Open.conf.orig
+++ /etc/sysctl.d/70-TCP-Fast-Open.conf
@@ -0,0 +1,2 @@
+# sysctl parameters managed by Puppet.
+net.ipv4.tcp_fastopen = 3

Rsyslog::Conf[opensearch-production-search-psi-codfw-gc-log-cleanup]

Parameters differences:

--- Rsyslog::Conf[opensearch-production-search-psi-codfw-gc-log-cleanup].orig
+++ Rsyslog::Conf[opensearch-production-search-psi-codfw-gc-log-cleanup]

+    mode     => 0444
+    require  => File[/var/log/opensearch-production-search-psi-codfw-gc-log-cleanup]
+    priority => 40
+    ensure   => present

File[/etc/ferm/conf.d/10_ip6ip6]

Parameters differences:

--- File[/etc/ferm/conf.d/10_ip6ip6].orig
+++ File[/etc/ferm/conf.d/10_ip6ip6]

+    mode    => 0400
+    require => File[/etc/ferm/conf.d]
+    tag     => ferm
+    notify  => Service[ferm]
+    group   => root
+    owner   => root
+    ensure  => present

Content differences:

--- /etc/ferm/conf.d/10_ip6ip6.orig
+++ /etc/ferm/conf.d/10_ip6ip6
@@ -0,0 +1,11 @@
+# Autogenerated by puppet. DO NOT EDIT BY HAND!
+#
+# 10_ip6ip6: 
+
+domain (ip6) {
+	table filter {
+		chain INPUT {
+			saddr 0100::/64 proto ipv6 ACCEPT;
+		}
+	}
+}

Systemd::Unit[nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer]

Parameters differences:

--- Systemd::Unit[nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer].orig
+++ Systemd::Unit[nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer
+    ensure            => absent

Service[prometheus_ferm_mss.timer]

Parameters differences:

--- Service[prometheus_ferm_mss.timer].orig
+++ Service[prometheus_ferm_mss.timer]

+    provider => systemd
+    enable   => True
+    ensure   => running

File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-unassigned-shard-check-9200.conf]

Parameters differences:

--- File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-unassigned-shard-check-9200.conf].orig
+++ File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-unassigned-shard-check-9200.conf]

+    mode   => 0444
+    notify => Service[rsyslog]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /etc/rsyslog.d/25-nrpe2nodexp-opensearch-unassigned-shard-check-9200.conf.orig
+++ /etc/rsyslog.d/25-nrpe2nodexp-opensearch-unassigned-shard-check-9200.conf
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: Apache-2.0
+if $programname contains "nrpe2nodexp-opensearch_unassigned_shard_check_9200" then {
+    if ($msg contains "\"ecs.version\": \"1.7.0\"") then {
+        # Send logs to kafka
+        set $.log_outputs = "kafka ecs_170 local";
+    } else {
+        # Filter out non-relevant nrpe2nodexp messages
+        stop
+    }
+}

Augeas[ipip0_set_up]

Parameters differences:

--- Augeas[ipip0_set_up].orig
+++ Augeas[ipip0_set_up]

+    onlyif  => match up[. = 'ip link set up dev ipip0'] size == 0
+    context => /files/etc/network/interfaces/*[. = 'ipip0' and ./family = 'inet']
+    incl    => /etc/network/interfaces
+    require => Augeas[ipip0_add_up]
+    lens    => Interfaces.lns
+    changes => set up[last()+1] 'ip link set up dev ipip0'

Sudo::User[nrpe-check_opensearch_unassigned_shard_check_9200]

Parameters differences:

--- Sudo::User[nrpe-check_opensearch_unassigned_shard_check_9200].orig
+++ Sudo::User[nrpe-check_opensearch_unassigned_shard_check_9200]

+    privileges => []
+    tag        => nrpe::check
+    require    => ['Class[Sudo]']
+    user       => nagios
+    ensure     => absent

Package[openjdk-11-jdk]

Parameters differences:

--- Package[openjdk-11-jdk].orig
+++ Package[openjdk-11-jdk]

+    provider => apt
+    ensure   => installed

File[/etc/opensearch/production-search-psi-codfw/opensearch.yml]

Parameters differences:

--- File[/etc/opensearch/production-search-psi-codfw/opensearch.yml].orig
+++ File[/etc/opensearch/production-search-psi-codfw/opensearch.yml]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => file

Content differences:

--- /etc/opensearch/production-search-psi-codfw/opensearch.yml.orig
+++ /etc/opensearch/production-search-psi-codfw/opensearch.yml
@@ -0,0 +1,195 @@
+# ======================== OpenSearch Configuration =========================
+#
+# NOTE: OpenSearch comes with reasonable defaults for most settings.
+#       Before you set out to tweak and tune the configuration, make sure you
+#       understand what are you trying to accomplish and the consequences.
+#
+# The primary way of configuring a node is via this file. This template lists
+# the most important settings you may want to configure for a production cluster.
+#
+# Please consult the documentation for further information on configuration options:
+# https://opensearch.org/docs/opensearch/configuration/
+#
+# ---------------------------------- Cluster -----------------------------------
+#
+# Use a descriptive name for your cluster:
+#
+cluster.name: production-search-psi-codfw
+#
+# ------------------------------------ Node ------------------------------------
+#
+# Use a descriptive name for the node:
+#
+node.name: cirrussearch2089-production-search-psi-codfw
+#
+# Add custom attributes to the node:
+#
+#node.attr.rack: r1
+#
+# Row/rack awareness attributes
+#
+cluster.routing.allocation.awareness.attributes: row
+node.attr.row: codfw-row-a
+node.attr.rack: A4
+#
+# Other attributes
+#
+node.attr.hostname: cirrussearch2089
+node.attr.fqdn: cirrussearch2089.codfw.wmnet
+#
+# Allow this node to be eligible as a master node (enabled by default):
+node.master: false
+# Allow this node to store data (enabled by default):
+node.data: true
+#
+# ----------------------------------- Paths ------------------------------------
+#
+# Path to directory where to store the data (separate multiple locations by comma):
+#
+path.data: /srv/opensearch/production-search-psi-codfw
+#
+# Path to log files:
+#
+path.logs: /var/log/opensearch
+#
+# ----------------------------------- Memory -----------------------------------
+#
+# Lock the memory on startup:
+#
+bootstrap.memory_lock: false # we don't have swap anyway
+#
+# Make sure that the heap size is set to about half the memory available
+# on the system and that the owner of the process is allowed to use this
+# limit.
+#
+# OpenSearch performs poorly when the system is swapping the memory.
+#
+# ---------------------------------- Network -----------------------------------
+#
+# Set the bind address to a specific IP (IPv4 or IPv6):
+#
+#network.host: 192.168.0.1
+#
+# Set a custom port for HTTP:
+#
+http.port: 9600
+#
+# For more information, consult the network module documentation.
+#
+# Set the address other nodes will use to communicate with this node. If not
+# set, it is automatically derived. It must point to an actual IP address.
+#
+network.publish_host: 10.192.6.18
+
+# Set both 'bind_host' and 'publish_host':
+#
+network.host: [ _local_,_site_ ]
+
+# Set a custom port for the node to node communication (9300 by default):
+#
+transport.tcp.port: 9700
+#
+# --------------------------------- Discovery ----------------------------------
+#
+# Pass an initial list of hosts to perform discovery when this node is started:
+# The default list of hosts is ["127.0.0.1", "[::1]"]
+#
+#discovery.seed_hosts: ["host1", "host2"]
+#
+# Bootstrap the cluster using an initial set of master-eligible nodes:
+#
+#cluster.initial_master_nodes: ["node-1", "node-2"]
+#
+discovery.zen.ping.unicast.hosts: ["cirrussearch2108.codfw.wmnet", "cirrussearch2076.codfw.wmnet", "cirrussearch2080.codfw.wmnet", "cirrussearch2083.codfw.wmnet", "cirrussearch2115.codfw.wmnet"]
+#
+# For more information, consult the discovery and cluster formation module documentation.
+#
+# ---------------------------------- Gateway -----------------------------------
+#
+# Block initial recovery after a full cluster restart until N nodes are started:
+#
+gateway.recover_after_nodes: 11
+#
+# For more information, consult the gateway module documentation.
+#
+#
+gateway:
+    recover_after_time: 5m
+    expected_nodes: 18
+#
+# ---------------------------------- Various -----------------------------------
+#
+# Require explicit names when deleting indices:
+#
+#action.destructive_requires_name: true
+#
+# ---------------------------------- Custom settings ---------------------------
+#
+#path.plugins: /usr/share/opensearch/plugins
+#
+# If a plugin listed here is not installed for current node, the node will not start.
+#
+plugin.mandatory: analysis-hebrew,analysis-icu,analysis-smartcn,analysis-stempel,analysis-ukrainian,cirrus-highlighter,opensearch-analysis-stconvert,opensearch-extra,opensearch-extra-analysis-esperanto,opensearch-extra-analysis-serbian,opensearch-extra-analysis-slovak,ltr
+
+# Specify the pattern of index names allowed to be created automatically
+action.auto_create_index: +apifeatureusage-*,+glent_*,-*
+
+# Protect against accidental close/delete operations on all indices. You can
+# still close/delete individual indices.
+action.destructive_requires_name: true
+
+# Allow up to 5000 shards to be queried at a time. The default
+# 1k is too low to allow mwgrep to operate.
+action.search.shard_count.limit: 5000
+
+# Enable the disk space aware shard allocator
+cluster.routing.allocation.disk.threshold_enabled: true
+# Stop allocating new shards to nodes with 75% disk utilization
+cluster.routing.allocation.disk.watermark.low: 0.75
+# Move shards off of nodes with 80% disk utilization
+cluster.routing.allocation.disk.watermark.high: 0.80
+# Check disk utilization every 60 seconds
+cluster.info.update.interval: 60s
+
+# Set the number of concurrent recoveries happening on a node:
+#
+# 1. During the initial recovery
+cluster.routing.allocation.node_initial_primaries_recoveries: 3
+
+# 2. During adding/removing nodes, rebalancing, etc
+cluster.routing.allocation.node_concurrent_recoveries: 2
+
+# Set to throttle throughput when recovering (eg. 100mb, by default 20mb):
+indices.recovery.max_bytes_per_sec: 40mb
+
+##
+# OpenSearch cluster balance configuration
+##
+# This puts much more weight on distributing the shards per index then the defaults.  This will make sure that,
+# for example, each live node gets a copy of enwiki.
+cluster.routing.allocation.balance:
+    shard: 0.195
+    index: 0.8
+
+##
+# Filter cache size
+##
+indices.queries.cache.size: 20%
+
+##
+# Thread pool settings
+##
+thread_pool:
+    write:
+        size: 6
+        queue_size: 1000
+
+
+
+
+
+##
+# Size the LTR cache to prevent churn under normal and AB testing scenarios
+ltr.caches.max_mem: 100mb
+
+

Package[edac-utils]

Parameters differences:

--- Package[edac-utils].orig
+++ Package[edac-utils]

+    provider => apt
+    ensure   => latest

Admin::Hashuser[dr0ptp4kt]

Parameters differences:

--- Admin::Hashuser[dr0ptp4kt].orig
+++ Admin::Hashuser[dr0ptp4kt]

+    ensure_ssh_key => True

Prometheus::Alert::Rule[check_opensearch_shards_9200_8cb398d9f4c08b77d96bfdab16485563]

Parameters differences:

--- Prometheus::Alert::Rule[check_opensearch_shards_9200_8cb398d9f4c08b77d96bfdab16485563].orig
+++ Prometheus::Alert::Rule[check_opensearch_shards_9200_8cb398d9f4c08b77d96bfdab16485563]

+    dashboard          => TODO
+    expr               => (nagios_nrpe_check_result{alert_rule_hash="8cb398d9f4c08b77d96bfdab16485563",check_name="check_opensearch_shards_9200", status=~"(WARNING|CRITICAL)", severity=~"(warning|critical)"} > 0) * on (instance) group_left (team) role_owner
+    team               => observability
+    description        => OpenSearch health check for shards on 9200
+    runbook            => https://wikitech.wikimedia.org/wiki/Search#Administration
+    def_label_whitelst => ['team', 'severity']
+    ensure             => absent
+    logs               => https://logstash.wikimedia.org/app/dashboards#/view/2d343ac0-6df8-11f0-8e08-7fab0da52b33?_g=(filters:!((query:(match_phrase:(event.module:check_opensearch_shards_9200))),(query:(match_phrase:(host.name:{{$labels.instance|stripPort}})))))
+    severity           => info
+    summary            => OpenSearch health check for shards on 9200
+    instance           => ops
+    group              => nrpechecks
+    alert_name         => check_opensearch_shards_9200
+    for                => 1m
+    site               => codfw

Udev::Rule[opensearch-readahead]

Parameters differences:

--- Udev::Rule[opensearch-readahead].orig
+++ Udev::Rule[opensearch-readahead]

+    priority => 40
+    ensure   => present

File[/etc/logrotate.d/prometheus_ferm_mss]

Parameters differences:

--- File[/etc/logrotate.d/prometheus_ferm_mss].orig
+++ File[/etc/logrotate.d/prometheus_ferm_mss]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/logrotate.d/prometheus_ferm_mss.orig
+++ /etc/logrotate.d/prometheus_ferm_mss
@@ -0,0 +1,12 @@
+# logrotate(8) config for prometheus_ferm_mss
+
+/var/log/prometheus_ferm_mss/*.log {
+    daily
+    copytruncate
+    missingok
+    compress
+    delaycompress
+    notifempty
+    rotate 15
+    size 256M
+}

Systemd::Unit[nrpe2nodexp-opensearch_shards_9600.service]

Parameters differences:

--- Systemd::Unit[nrpe2nodexp-opensearch_shards_9600.service].orig
+++ Systemd::Unit[nrpe2nodexp-opensearch_shards_9600.service]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => nrpe2nodexp-opensearch_shards_9600.service
+    ensure            => absent

Class[Profile::Contacts]

Parameters differences:

--- Class[Profile::Contacts].orig
+++ Class[Profile::Contacts]

@@
-    cluster       => insetup
+    cluster       => elasticsearch
@@
-    role_contacts => ['Data Platform']
+    role_contacts => ['Search Platform']

Systemd::Service[nrpe2nodexp-opensearch_shards_9200]

Parameters differences:

--- Systemd::Service[nrpe2nodexp-opensearch_shards_9200].orig
+++ Systemd::Service[nrpe2nodexp-opensearch_shards_9200]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => absent
+    monitoring_critical      => False
+    require                  => Systemd::Unit[nrpe2nodexp-opensearch_shards_9200.service]
+    monitoring_enabled       => False
+    unit_type                => timer
+    restart                  => False

Service[prometheus_lvs_realserver_mss.timer]

Parameters differences:

--- Service[prometheus_lvs_realserver_mss.timer].orig
+++ Service[prometheus_lvs_realserver_mss.timer]

+    provider => systemd
+    enable   => True
+    ensure   => running

Service[opensearch-production-search-psi-codfw-gc-log-cleanup.timer]

Parameters differences:

--- Service[opensearch-production-search-psi-codfw-gc-log-cleanup.timer].orig
+++ Service[opensearch-production-search-psi-codfw-gc-log-cleanup.timer]

+    provider => systemd
+    enable   => True
+    ensure   => running

Nrpe::Monitor_service[check_tcp-mss-clamper_status]

Parameters differences:

--- Nrpe::Monitor_service[check_tcp-mss-clamper_status].orig
+++ Nrpe::Monitor_service[check_tcp-mss-clamper_status]

+    timeout                     => 10
+    nrpe2nodexp_parse_perf_data => False
+    retry_interval              => 1
+    description                 => Check unit status of tcp-mss-clamper
+    notes_url                   => https://wikitech.wikimedia.org/wiki/LVS#IPIP_encapsulation_experiments
+    critical                    => False
+    retries                     => 2
+    migration_task              => T321808
+    ensure                      => absent
+    enable_nrpe2nodexp          => False
+    enable_icinga_check         => True
+    check_interval              => 10
+    contact_group               => admins
+    nrpe_command                => /usr/local/lib/nagios/plugins/check_systemd_unit_status tcp-mss-clamper

Exec[/usr/sbin/dpkg-reconfigure -p critical -f noninteractive wikimedia-lvs-realserver]

Parameters differences:

--- Exec[/usr/sbin/dpkg-reconfigure -p critical -f noninteractive wikimedia-lvs-realserver].orig
+++ Exec[/usr/sbin/dpkg-reconfigure -p critical -f noninteractive wikimedia-lvs-realserver]

+    refreshonly => True
+    subscribe   => File[/etc/default/wikimedia-lvs-realserver]
+    require     => Package[wikimedia-lvs-realserver]
+    path        => /bin:/sbin:/usr/bin:/usr/sbin

File[/var/log/prometheus_ferm_mss]

Parameters differences:

--- File[/var/log/prometheus_ferm_mss].orig
+++ File[/var/log/prometheus_ferm_mss]

+    mode   => 0755
+    backup => False
+    force  => True
+    owner  => root
+    ensure => directory
+    group  => root

File[/var/log/opensearch-disable-readahead-production-search-psi-codfw]

Parameters differences:

--- File[/var/log/opensearch-disable-readahead-production-search-psi-codfw].orig
+++ File[/var/log/opensearch-disable-readahead-production-search-psi-codfw]

+    mode   => 0755
+    backup => False
+    force  => True
+    owner  => root
+    ensure => directory
+    group  => root

File[/etc/ferm/conf.d/10_clamp-mss-ipv6]

Parameters differences:

--- File[/etc/ferm/conf.d/10_clamp-mss-ipv6].orig
+++ File[/etc/ferm/conf.d/10_clamp-mss-ipv6]

+    mode    => 0400
+    require => File[/etc/ferm/conf.d]
+    tag     => ferm
+    notify  => Service[ferm]
+    group   => root
+    owner   => root
+    ensure  => present

Content differences:

--- /etc/ferm/conf.d/10_clamp-mss-ipv6.orig
+++ /etc/ferm/conf.d/10_clamp-mss-ipv6
@@ -0,0 +1,11 @@
+# Autogenerated by puppet. DO NOT EDIT BY HAND!
+#
+# 10_clamp-mss-ipv6: 
+
+domain (ip6) {
+	table filter {
+		chain OUTPUT {
+			outerface (eno12399np0 lo) saddr @ipfilter(10.2.1.30) proto tcp sport (9200 9243 9643) tcp-flags (SYN) SYN TCPMSS set-mss 1400;
+		}
+	}
+}

File[/etc/conftool]

Parameters differences:

--- File[/etc/conftool].orig
+++ File[/etc/conftool]

+    mode   => 0755
+    group  => root
+    owner  => root
+    ensure => directory

Exec[Refresh tmpfile opensearch-production-search-psi-codfw]

Parameters differences:

--- Exec[Refresh tmpfile opensearch-production-search-psi-codfw].orig
+++ Exec[Refresh tmpfile opensearch-production-search-psi-codfw]

+    refreshonly => True
+    subscribe   => File[/etc/tmpfiles.d/opensearch-production-search-psi-codfw.conf]
+    command     => /bin/systemd-tmpfiles --create --remove '/etc/tmpfiles.d/opensearch-production-search-psi-codfw.conf'
+    user        => root

File[/usr/local/bin/pooler-loop]

Parameters differences:

--- File[/usr/local/bin/pooler-loop].orig
+++ File[/usr/local/bin/pooler-loop]

+    mode   => 0555
+    source => puppet:///modules/conftool/pooler_loop.rb
+    group  => root
+    owner  => root
+    ensure => present

File[/etc/logrotate.d/opensearch-disable-readahead-production-search-psi-codfw]

Parameters differences:

--- File[/etc/logrotate.d/opensearch-disable-readahead-production-search-psi-codfw].orig
+++ File[/etc/logrotate.d/opensearch-disable-readahead-production-search-psi-codfw]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/logrotate.d/opensearch-disable-readahead-production-search-psi-codfw.orig
+++ /etc/logrotate.d/opensearch-disable-readahead-production-search-psi-codfw
@@ -0,0 +1,12 @@
+# logrotate(8) config for opensearch-disable-readahead-production-search-psi-codfw
+
+/var/log/opensearch-disable-readahead-production-search-psi-codfw/*.log {
+    daily
+    copytruncate
+    missingok
+    compress
+    delaycompress
+    notifempty
+    rotate 15
+    size 256M
+}

Rsyslog::Conf[opensearch-production-search-codfw-gc-log-cleanup]

Parameters differences:

--- Rsyslog::Conf[opensearch-production-search-codfw-gc-log-cleanup].orig
+++ Rsyslog::Conf[opensearch-production-search-codfw-gc-log-cleanup]

+    mode     => 0444
+    require  => File[/var/log/opensearch-production-search-codfw-gc-log-cleanup]
+    priority => 40
+    ensure   => present

File[/lib/systemd/system/opensearch-production-search-codfw-gc-log-cleanup.timer]

Parameters differences:

--- File[/lib/systemd/system/opensearch-production-search-codfw-gc-log-cleanup.timer].orig
+++ File[/lib/systemd/system/opensearch-production-search-codfw-gc-log-cleanup.timer]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for opensearch-production-search-codfw-gc-log-cleanup.timer (opensearch-production-search-codfw-gc-log-cleanup.timer)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/opensearch-production-search-codfw-gc-log-cleanup.timer.orig
+++ /lib/systemd/system/opensearch-production-search-codfw-gc-log-cleanup.timer
@@ -0,0 +1,12 @@
+[Unit]
+Description=Periodic execution of opensearch-production-search-codfw-gc-log-cleanup.service
+
+[Timer]
+Unit=opensearch-production-search-codfw-gc-log-cleanup.service
+# Accuracy sets the maximum time interval around the execution time we want to allow
+AccuracySec=15sec
+OnCalendar=*-*-* 02:12:00
+RandomizedDelaySec=0
+
+[Install]
+WantedBy=multi-user.target

File[/etc/sudoers.d/nrpe-check_opensearch_unassigned_shard_check_9200]

Parameters differences:

--- File[/etc/sudoers.d/nrpe-check_opensearch_unassigned_shard_check_9200].orig
+++ File[/etc/sudoers.d/nrpe-check_opensearch_unassigned_shard_check_9200]

+    group   => root
+    require => Package[nagios-nrpe-server]
+    owner   => root
+    ensure  => absent

Exec[s3-credentials-user-production-search-codfw]

Parameters differences:

--- Exec[s3-credentials-user-production-search-codfw].orig
+++ Exec[s3-credentials-user-production-search-codfw]

+    require     => File[/etc/opensearch/production-search-codfw/opensearch.keystore]
+    path        => /bin:/usr/bin
+    environment => ['OPENSEARCH_PATH_CONF=/etc/opensearch/production-search-codfw']
+    unless      => /usr/share/opensearch/bin/opensearch-keystore list | grep s3.client.default.access_key
+    command     => echo 'search:platform' | /usr/share/opensearch/bin/opensearch-keystore add s3.client.default.access_key
+    group       => opensearch

Systemd::Unit[prometheus-elasticsearch-exporter-9600]

Parameters differences:

--- Systemd::Unit[prometheus-elasticsearch-exporter-9600].orig
+++ Systemd::Unit[prometheus-elasticsearch-exporter-9600]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => prometheus-elasticsearch-exporter-9600
+    ensure            => present

File[/etc/logrotate.d/prometheus_lvs_realserver_mss]

Parameters differences:

--- File[/etc/logrotate.d/prometheus_lvs_realserver_mss].orig
+++ File[/etc/logrotate.d/prometheus_lvs_realserver_mss]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/logrotate.d/prometheus_lvs_realserver_mss.orig
+++ /etc/logrotate.d/prometheus_lvs_realserver_mss
@@ -0,0 +1,12 @@
+# logrotate(8) config for prometheus_lvs_realserver_mss
+
+/var/log/prometheus_lvs_realserver_mss/*.log {
+    daily
+    copytruncate
+    missingok
+    compress
+    delaycompress
+    notifempty
+    rotate 15
+    size 256M
+}

File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-shards-9200.conf]

Parameters differences:

--- File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-shards-9200.conf].orig
+++ File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-shards-9200.conf]

+    mode   => 0444
+    notify => Service[rsyslog]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /etc/rsyslog.d/25-nrpe2nodexp-opensearch-shards-9200.conf.orig
+++ /etc/rsyslog.d/25-nrpe2nodexp-opensearch-shards-9200.conf
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: Apache-2.0
+if $programname contains "nrpe2nodexp-opensearch_shards_9200" then {
+    if ($msg contains "\"ecs.version\": \"1.7.0\"") then {
+        # Send logs to kafka
+        set $.log_outputs = "kafka ecs_170 local";
+    } else {
+        # Filter out non-relevant nrpe2nodexp messages
+        stop
+    }
+}

Conftool::Scripts::Safe_service_restart[opensearch_1@production-search-codfw]

Parameters differences:

--- Conftool::Scripts::Safe_service_restart[opensearch_1@production-search-codfw].orig
+++ Conftool::Scripts::Safe_service_restart[opensearch_1@production-search-codfw]

+    require         => ['Class[Conftool::Scripts]']
+    lvs_pools       => ['search', 'search-https']
+    max_concurrency => 5

Augeas[ipip0_manual]

Parameters differences:

--- Augeas[ipip0_manual].orig
+++ Augeas[ipip0_manual]

+    lens    => Interfaces.lns
+    incl    => /etc/network/interfaces
+    changes => ["set auto[./1 = 'ipip0']/1 'ipip0'", "set iface[. = 'ipip0'] 'ipip0'", "set iface[. = 'ipip0']/family 'inet'", "set iface[. = 'ipip0']/method 'manual'"]
+    context => /files/etc/network/interfaces

Motd::Message[cirrus::opensearch]

Parameters differences:

--- Motd::Message[cirrus::opensearch].orig
+++ Motd::Message[cirrus::opensearch]

+    message  => cirrussearch2089 is a OpenSearch cirrus (cirrus::opensearch)
+    priority => 5
+    ensure   => present

File[/usr/local/bin/prometheus-ferm-mss]

Parameters differences:

--- File[/usr/local/bin/prometheus-ferm-mss].orig
+++ File[/usr/local/bin/prometheus-ferm-mss]

+    mode   => 0555
+    source => puppet:///modules/prometheus/usr/local/bin/prometheus-ferm-mss.py
+    group  => root
+    owner  => root
+    ensure => file

Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer (nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer)]

Parameters differences:

--- Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer (nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer)].orig
+++ Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer (nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

Opensearch::Log::Hot_threads_cluster[production-search-psi-codfw]

Parameters differences:

--- Opensearch::Log::Hot_threads_cluster[production-search-psi-codfw].orig
+++ Opensearch::Log::Hot_threads_cluster[production-search-psi-codfw]

+    cluster_name => production-search-psi-codfw
+    http_port    => 9600

File[/etc/opensearch_hot_threads.d/production-search-codfw.yml]

Parameters differences:

--- File[/etc/opensearch_hot_threads.d/production-search-codfw.yml].orig
+++ File[/etc/opensearch_hot_threads.d/production-search-codfw.yml]

+    mode   => 0755
+    group  => root
+    owner  => root
+    ensure => file

Content differences:

--- /etc/opensearch_hot_threads.d/production-search-codfw.yml.orig
+++ /etc/opensearch_hot_threads.d/production-search-codfw.yml
@@ -0,0 +1,3 @@
+---
+port: 9200
+log_file: "/var/log/opensearch/opensearch_hot_threads-production-search-codfw.log"

Class[Profile::Admin]

Parameters differences:

--- Class[Profile::Admin].orig
+++ Class[Profile::Admin]

@@
-    groups => []
+    groups => ['elasticsearch-roots']

Nrpe::Check[check_disk_space]

Parameters differences:

--- Nrpe::Check[check_disk_space].orig
+++ Nrpe::Check[check_disk_space]

@@
-    command => /usr/lib/nagios/plugins/check_disk -w 6% -c 3% -W 6% -K 3% -l -e -A -i "/srv/sd[a-b][1-3]" -i "/srv/nvme[0-9]n[0-9]p[0-9]" --exclude-type=fuse --exclude-type=fuse.fuse_dfs --exclude-type=tracefs
+    command => /usr/lib/nagios/plugins/check_disk -w 15% -c 10% -W 6% -K 3% -l -e -A -i "/srv/sd[a-b][1-3]" --exclude-type=tracefs

Rsyslog::Conf[nrpe2nodexp-opensearch_shard_size_check_9600]

Parameters differences:

--- Rsyslog::Conf[nrpe2nodexp-opensearch_shard_size_check_9600].orig
+++ Rsyslog::Conf[nrpe2nodexp-opensearch_shard_size_check_9600]

+    mode     => 0444
+    priority => 25
+    ensure   => absent

File[/usr/local/bin/opensearch-disable-readahead.sh]

Parameters differences:

--- File[/usr/local/bin/opensearch-disable-readahead.sh].orig
+++ File[/usr/local/bin/opensearch-disable-readahead.sh]

+    mode   => 0555
+    source => puppet:///modules/profile/opensearch/cirrus/opensearch-disable-readahead.sh
+    group  => root
+    owner  => root
+    ensure => file

Logrotate::Conf[wmf_auto_restart_prometheus-elasticsearch-exporter-9600]

Parameters differences:

--- Logrotate::Conf[wmf_auto_restart_prometheus-elasticsearch-exporter-9600].orig
+++ Logrotate::Conf[wmf_auto_restart_prometheus-elasticsearch-exporter-9600]

+    ensure => present

Monitoring::Exported_nagios_service[cirrussearch2089 ferm_active]

Parameters differences:

--- Monitoring::Exported_nagios_service[cirrussearch2089 ferm_active].orig
+++ Monitoring::Exported_nagios_service[cirrussearch2089 ferm_active]

@@
-    notifications_enabled => 0
+    notifications_enabled => 1
@@
-    servicegroups         => insetup_codfw
+    servicegroups         => elasticsearch_codfw

Systemd::Timer::Job[nrpe2nodexp-check_tcp-mss-clamper_status]

Parameters differences:

--- Systemd::Timer::Job[nrpe2nodexp-check_tcp-mss-clamper_status].orig
+++ Systemd::Timer::Job[nrpe2nodexp-check_tcp-mss-clamper_status]

+    description               => execution of nrpe2nodexp for the check_check_tcp-mss-clamper_status command.
+    syslog_match_startswith   => True
+    monitoring_contact_groups => admins
+    logging_enabled           => False
+    success_exit_status       => []
+    fixed_random_delay        => False
+    private_tmp               => False
+    logfile_basedir           => /var/log
+    group                     => prometheus-node-exporter
+    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+    syslog_identifier         => nrpe2nodexp-check_tcp-mss-clamper_status
+    send_mail_only_on_error   => True
+    ignore_errors             => True
+    logfile_group             => root
+    interval                  => [{'start': 'OnUnitInactiveSec', 'interval': '10min'}]
+    send_mail                 => False
+    logfile_name              => syslog.log
+    user                      => nagios
+    ensure                    => absent
+    send_mail_to              => root@cirrussearch2089.codfw.wmnet
+    logfile_perms             => all
+    syslog_force_stop         => True
+    environment               => {}
+    command                   => /usr/local/bin/nrpe2nodexp --alert-rule-hash "15c652bc7390794191321c22a2885d1f" --timeout 10 --check-command "check_check_tcp-mss-clamper_status"
+    monitoring_enabled        => False

File[/lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9600.service]

Parameters differences:

--- File[/lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9600.service].orig
+++ File[/lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9600.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9600.service (nrpe2nodexp-opensearch_unassigned_shard_check_9600.service)]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9600.service.orig
+++ /lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9600.service
@@ -0,0 +1,11 @@
+[Unit]
+Description=execution of nrpe2nodexp for the check_opensearch_unassigned_shard_check_9600 command.
+Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+
+[Service]
+Type=oneshot
+User=nagios
+
+Group=prometheus-node-exporter
+SyslogIdentifier=nrpe2nodexp-opensearch_unassigned_shard_check_9600
+ExecStart=-/usr/local/bin/nrpe2nodexp --alert-rule-hash "8641ad857de0a06c2748b18ea2c135ab" --timeout 10 --check-command "check_opensearch_unassigned_shard_check_9600"

File[/etc/ssh/userkeys/ebernhardson]

Parameters differences:

--- File[/etc/ssh/userkeys/ebernhardson].orig
+++ File[/etc/ssh/userkeys/ebernhardson]

+    mode      => 0444
+    show_diff => False
+    force     => True
+    owner     => root
+    ensure    => file
+    group     => root

Content differences:

--- /etc/ssh/userkeys/ebernhardson.orig
+++ /etc/ssh/userkeys/ebernhardson
@@ -0,0 +1 @@
+ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCq5sClypKI1N1T1ZfR2ZZPbWH6gXPrQ9a3HRiFgWpcp+W4YIYT4taNNqd+gFO6aF7UyAS9148ROxivCksSK+MmfULsIz0cD4l8R6haR939kfK5MRC2yONusy8IUkQgPvkBWkIvOhrURNs2sPkxYVwEyHIOUX3Izf1cLjVZenz4lhRqPtstHMZQaa3TGAoCFcymOln/64j5vq9W8MljLBsEWpPdeUwmf6mD0EHb5LUA+2aVB1JwfmViY5Dm77FBh5CDHqdvMWa8ZF0ZybtUAuL0wpcO0h/H1WDhqS5j1Q4AemxtOeRnBJoCsh5Wlf4J4sZptMM2hvdNYnqWXb+3Dto7 ebernhardson@wikimedia.org

Service[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer]

Parameters differences:

--- Service[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer].orig
+++ Service[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer]

+    provider => systemd
+    enable   => True
+    ensure   => running

File[/var/log/opensearch/opensearch_index_indexing_slowlog.log]

Parameters differences:

--- File[/var/log/opensearch/opensearch_index_indexing_slowlog.log].orig
+++ File[/var/log/opensearch/opensearch_index_indexing_slowlog.log]

+    group  => root
+    owner  => root
+    ensure => absent

File[/etc/ssh/userkeys/pfischer]

Parameters differences:

--- File[/etc/ssh/userkeys/pfischer].orig
+++ File[/etc/ssh/userkeys/pfischer]

+    mode      => 0444
+    show_diff => False
+    force     => True
+    owner     => root
+    ensure    => file
+    group     => root

Content differences:

--- /etc/ssh/userkeys/pfischer.orig
+++ /etc/ssh/userkeys/pfischer
@@ -0,0 +1 @@
+ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFjit1wuGOgmJNp2JMHRcy6LyfJWXwjacns04JXyXyLA pfischer production

File[/usr/local/bin/prometheus-lvs-realserver-mss]

Parameters differences:

--- File[/usr/local/bin/prometheus-lvs-realserver-mss].orig
+++ File[/usr/local/bin/prometheus-lvs-realserver-mss]

+    mode   => 0555
+    source => puppet:///modules/prometheus/usr/local/bin/prometheus-lvs-realserver-mss.py
+    group  => root
+    owner  => root
+    ensure => file

File[/lib/systemd/system/clean_up_dlq.timer]

Parameters differences:

--- File[/lib/systemd/system/clean_up_dlq.timer].orig
+++ File[/lib/systemd/system/clean_up_dlq.timer]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for clean_up_dlq.timer (clean_up_dlq.timer)]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /lib/systemd/system/clean_up_dlq.timer.orig
+++ /lib/systemd/system/clean_up_dlq.timer
@@ -0,0 +1,12 @@
+[Unit]
+Description=Periodic execution of clean_up_dlq.service
+
+[Timer]
+Unit=clean_up_dlq.service
+# Accuracy sets the maximum time interval around the execution time we want to allow
+AccuracySec=15sec
+OnCalendar=1h
+RandomizedDelaySec=0
+
+[Install]
+WantedBy=multi-user.target

Admin::User[dr0ptp4kt]

Parameters differences:

--- Admin::User[dr0ptp4kt].orig
+++ Admin::User[dr0ptp4kt]

+    uid      => 2962
+    groups   => []
+    gid      => 500
+    ensure   => present
+    ssh_keys => ['ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoKHi5isY9FixH31qz/81V7fOHsorLZI/NLKr9Z6Xawl2a2Ih0ZV/pJtD+BTu1ufK2QOdgobeRSrnybzf2/1aCqi3Z9H2XxJhMCfnLb/9AIcKJ9tN63T4nRnjLoPsmRgDQrOSIqY5NfLKzXBsQOqc3chZ5SaDf8f09OdBk+Obn5vhr6yWh4GhrfTzoZUfp6+JRiueZZYuGMIKdBAH82s9TyuhuGWvHJmO9WC1MJOV/3hIcim+X0xR+BNLEU/Uj4OPEXC0/EiXh2CJDLugBpLU28RF+Y16TRj/WmO2H0H6qVdmkiK7Ez9PCbsy4RFPq4hdART9QiQbQJzZzaYSAkSFV abaso@wikimedia.org']
+    home_dir => /home/dr0ptp4kt
+    comment  => Adam Baso
+    shell    => /bin/bash

Systemd::Service[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]

Parameters differences:

--- Systemd::Service[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200].orig
+++ Systemd::Service[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => present
+    monitoring_critical      => False
+    require                  => Systemd::Unit[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service]
+    monitoring_enabled       => False
+    unit_type                => timer
+    restart                  => False

Logstash::Output::Udp[gelf_relay]

Parameters differences:

--- Logstash::Output::Udp[gelf_relay].orig
+++ Logstash::Output::Udp[gelf_relay]

+    host      => localhost
+    codec     => json
+    port      => 11514
+    plugin_id => output/udp/gelf_relay
+    priority  => 90
+    ensure    => present

Systemd::Syslog[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]

Parameters differences:

--- Systemd::Syslog[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200].orig
+++ Systemd::Syslog[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]

+    programname_comparison => startswith
+    force_stop             => True
+    ensure                 => present
+    base_dir               => /var/log
+    readable_by            => all
+    log_filename           => syslog.log
+    owner                  => root
+    group                  => root

File[/usr/local/bin/ispooled]

Parameters differences:

--- File[/usr/local/bin/ispooled].orig
+++ File[/usr/local/bin/ispooled]

+    mode   => 0555
+    source => puppet:///modules/conftool/ispooled.sh
+    group  => root
+    owner  => root
+    ensure => present

File[/lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer]

Parameters differences:

--- File[/lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer].orig
+++ File[/lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer (wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer.orig
+++ /lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer
@@ -0,0 +1,12 @@
+[Unit]
+Description=Periodic execution of wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service
+
+[Timer]
+Unit=wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service
+# Accuracy sets the maximum time interval around the execution time we want to allow
+AccuracySec=15sec
+OnCalendar=Mon,Tue,Wed,Thu,Fri *-*-* 5:3:00
+RandomizedDelaySec=0
+
+[Install]
+WantedBy=multi-user.target

Systemd::Timer[nrpe2nodexp-opensearch_shards_9200]

Parameters differences:

--- Systemd::Timer[nrpe2nodexp-opensearch_shards_9200].orig
+++ Systemd::Timer[nrpe2nodexp-opensearch_shards_9200]

+    fixed_random_delay => False
+    unit_name          => nrpe2nodexp-opensearch_shards_9200.service
+    timer_intervals    => [{'start': 'OnUnitInactiveSec', 'interval': '1min'}, {'interval': '1s', 'start': 'OnActiveSec'}]
+    splay              => 0
+    accuracy           => 15sec
+    ensure             => absent

Nrpe::Check[check_opensearch_shard_size_check_9600]

Parameters differences:

--- Nrpe::Check[check_opensearch_shard_size_check_9600].orig
+++ Nrpe::Check[check_opensearch_shard_size_check_9600]

+    command => /usr/lib/nagios/plugins/check_elasticsearch_shard_size.py --url http://localhost:9600 --shard-size-warning 110 --shard-size-critical 140 --timeout 4
+    before  => Monitoring::Service[opensearch_shard_size_check_9600]
+    ensure  => present

Class[Java]

Parameters differences:

--- Class[Java].orig
+++ Class[Java]

+    egd_source    => /dev/random
+    require       => Package[wmf-certificates]
+    enable_dbg    => False
+    java_packages => [{'version': '11', 'variant': 'jdk'}]
+    before        => ['Java::Cacert[wmf:puppetca.pem]', 'Java::Cacert[wmf:Wikimedia_Internal_Root_CA]', 'Java::Cacert[wmf:puppetca.pem]', 'Java::Cacert[wmf:Wikimedia_Internal_Root_CA]']
+    hardened_tls  => False

Systemd::Syslog[clean_up_dlq]

Parameters differences:

--- Systemd::Syslog[clean_up_dlq].orig
+++ Systemd::Syslog[clean_up_dlq]

+    programname_comparison => startswith
+    force_stop             => True
+    ensure                 => absent
+    base_dir               => /var/log
+    readable_by            => all
+    log_filename           => syslog.log
+    owner                  => root
+    group                  => root

Service[opensearch_1@production-search-psi-codfw]

Parameters differences:

--- Service[opensearch_1@production-search-psi-codfw].orig
+++ Service[opensearch_1@production-search-psi-codfw]

+    require  => ['Systemd::Unit[opensearch_1@.service]', 'File[/etc/opensearch/production-search-psi-codfw/opensearch.yml]', 'File[/etc/opensearch/production-search-psi-codfw/logging.yml]', 'File[/etc/opensearch/production-search-psi-codfw/log4j2.properties]', 'File[/etc/opensearch/production-search-psi-codfw/jvm.options]', 'File[/srv/opensearch/production-search-psi-codfw]', 'Systemd::Tmpfile[opensearch-production-search-psi-codfw]']
+    tag      => opensearch_services
+    enable   => True
+    provider => systemd
+    ensure   => running

Class[Base::Sysctl]

Parameters differences:

--- Class[Base::Sysctl].orig
+++ Class[Base::Sysctl]

@@
-    enable_rp_filter => True
+    enable_rp_filter => False

Exec[elasticsearch-roots_ensure_members]

Parameters differences:

--- Exec[elasticsearch-roots_ensure_members].orig
+++ Exec[elasticsearch-roots_ensure_members]

+    logoutput => True
+    require   => ['User[ebernhardson]', 'User[dcausse]', 'User[pfischer]', 'User[dr0ptp4kt]']
+    path      => /usr/bin:/bin
+    unless    => getent group elasticsearch-roots | xargs test -z || getent group elasticsearch-roots | cut -d ':' -f 4 | grep -E ^ebernhardson,dcausse,pfischer,dr0ptp4kt$
+    command   => /usr/bin/gpasswd elasticsearch-roots -M ebernhardson,dcausse,pfischer,dr0ptp4kt

File[/etc/nginx/sites-enabled/production-search-psi-codfw]

Parameters differences:

--- File[/etc/nginx/sites-enabled/production-search-psi-codfw].orig
+++ File[/etc/nginx/sites-enabled/production-search-psi-codfw]

+    target => /etc/nginx/sites-available/production-search-psi-codfw
+    group  => root
+    before => ['Service[nginx]']
+    owner  => root
+    ensure => link

Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9200.service (nrpe2nodexp-opensearch_unassigned_shard_check_9200.service)]

Parameters differences:

--- Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9200.service (nrpe2nodexp-opensearch_unassigned_shard_check_9200.service)].orig
+++ Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9200.service (nrpe2nodexp-opensearch_unassigned_shard_check_9200.service)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

Systemd::Unit[prometheus-wmf-elasticsearch-exporter-9600]

Parameters differences:

--- Systemd::Unit[prometheus-wmf-elasticsearch-exporter-9600].orig
+++ Systemd::Unit[prometheus-wmf-elasticsearch-exporter-9600]

+    restart           => True
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => prometheus-wmf-elasticsearch-exporter-9600
+    ensure            => present

File[/lib/systemd/system/logstash.service]

Parameters differences:

--- File[/lib/systemd/system/logstash.service].orig
+++ File[/lib/systemd/system/logstash.service]

+    group  => root
+    owner  => root
+    ensure => absent

Exec[systemd daemon-reload for opensearch-disable-readahead.service (opensearch-disable-readahead.service)]

Parameters differences:

--- Exec[systemd daemon-reload for opensearch-disable-readahead.service (opensearch-disable-readahead.service)].orig
+++ Exec[systemd daemon-reload for opensearch-disable-readahead.service (opensearch-disable-readahead.service)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

Exec[systemd daemon-reload for opensearch-disable-readahead-production-search-psi-codfw.service (opensearch-disable-readahead-production-search-psi-codfw.service)]

Parameters differences:

--- Exec[systemd daemon-reload for opensearch-disable-readahead-production-search-psi-codfw.service (opensearch-disable-readahead-production-search-psi-codfw.service)].orig
+++ Exec[systemd daemon-reload for opensearch-disable-readahead-production-search-psi-codfw.service (opensearch-disable-readahead-production-search-psi-codfw.service)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

Conftool::Scripts::Safe_service_restart[nginx]

Parameters differences:

--- Conftool::Scripts::Safe_service_restart[nginx].orig
+++ Conftool::Scripts::Safe_service_restart[nginx]

+    require         => ['Class[Conftool::Scripts]']
+    lvs_pools       => ['search-https', 'search-psi-https']
+    max_concurrency => 5

Systemd::Service[nrpe2nodexp-opensearch_unassigned_shard_check_9600]

Parameters differences:

--- Systemd::Service[nrpe2nodexp-opensearch_unassigned_shard_check_9600].orig
+++ Systemd::Service[nrpe2nodexp-opensearch_unassigned_shard_check_9600]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => absent
+    monitoring_critical      => False
+    require                  => Systemd::Unit[nrpe2nodexp-opensearch_unassigned_shard_check_9600.service]
+    monitoring_enabled       => False
+    unit_type                => timer
+    restart                  => False

Systemd::Mask[nginx.service]

Parameters differences:

--- Systemd::Mask[nginx.service].orig
+++ Systemd::Mask[nginx.service]

+    unless => /usr/bin/dpkg -s nginx-light | /bin/grep -q '^Status: install ok installed$'
+    unit   => nginx.service
+    before => ['Package[nginx-light]']

Systemd::Unit[wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service]

Parameters differences:

--- Systemd::Unit[wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service].orig
+++ Systemd::Unit[wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service
+    ensure            => present

Rsyslog::Conf[opensearch-disable-readahead-production-search-codfw]

Parameters differences:

--- Rsyslog::Conf[opensearch-disable-readahead-production-search-codfw].orig
+++ Rsyslog::Conf[opensearch-disable-readahead-production-search-codfw]

+    mode     => 0444
+    require  => File[/var/log/opensearch-disable-readahead-production-search-codfw]
+    priority => 40
+    ensure   => present

Admin::Group[elasticsearch-roots]

Parameters differences:

--- Admin::Group[elasticsearch-roots].orig
+++ Admin::Group[elasticsearch-roots]

+    privileges => ['ALL = (ALL) NOPASSWD: ALL']
+    ensure     => present
+    gid        => 709

File[/usr/local/bin/cleanup-dlq]

Parameters differences:

--- File[/usr/local/bin/cleanup-dlq].orig
+++ File[/usr/local/bin/cleanup-dlq]

+    group  => root
+    owner  => root
+    ensure => absent

Exec[symlink /etc/opensearch/production-search-codfw/sudachi]

Parameters differences:

--- Exec[symlink /etc/opensearch/production-search-codfw/sudachi].orig
+++ Exec[symlink /etc/opensearch/production-search-codfw/sudachi]

+    creates => /etc/opensearch/production-search-codfw/sudachi
+    onlyif  => /usr/bin/test -d '/usr/share/opensearch/config/sudachi'
+    command => /usr/bin/ln -s '/usr/share/opensearch/config/sudachi' '/etc/opensearch/production-search-codfw/sudachi'

File[/usr/local/bin/pool-opensearch_1@production-search-psi-codfw]

Parameters differences:

--- File[/usr/local/bin/pool-opensearch_1@production-search-psi-codfw].orig
+++ File[/usr/local/bin/pool-opensearch_1@production-search-psi-codfw]

+    mode   => 0555
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /usr/local/bin/pool-opensearch_1@production-search-psi-codfw.orig
+++ /usr/local/bin/pool-opensearch_1@production-search-psi-codfw
@@ -0,0 +1,2 @@
+#!/bin/bash
+/usr/local/bin/safe-service-restart --pools search-psi-https --pool --retries 10 --wait 5

Systemd::Service[nrpe2nodexp-opensearch_shard_size_check_9600]

Parameters differences:

--- Systemd::Service[nrpe2nodexp-opensearch_shard_size_check_9600].orig
+++ Systemd::Service[nrpe2nodexp-opensearch_shard_size_check_9600]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => absent
+    monitoring_critical      => False
+    require                  => Systemd::Unit[nrpe2nodexp-opensearch_shard_size_check_9600.service]
+    monitoring_enabled       => False
+    unit_type                => timer
+    restart                  => False

Nrpe::Monitor_service[opensearch_shards_9200]

Parameters differences:

--- Nrpe::Monitor_service[opensearch_shards_9200].orig
+++ Nrpe::Monitor_service[opensearch_shards_9200]

+    timeout                     => 10
+    nrpe2nodexp_parse_perf_data => False
+    retry_interval              => 1
+    description                 => OpenSearch health check for shards on 9200
+    notes_url                   => https://wikitech.wikimedia.org/wiki/Search#Administration
+    critical                    => False
+    migration_task              => T371083
+    retries                     => 3
+    ensure                      => present
+    enable_nrpe2nodexp          => False
+    enable_icinga_check         => True
+    check_interval              => 1
+    contact_group               => admins,team-discovery
+    nrpe_command                => /usr/lib/nagios/plugins/check_elasticsearch.py --ignore-status --url http://localhost:9200 --shards-inactive '>=0.15' --timeout 4

Service[clean_up_dlq.timer]

Parameters differences:

--- Service[clean_up_dlq.timer].orig
+++ Service[clean_up_dlq.timer]

+    before   => ['Exec[systemd daemon-reload for clean_up_dlq.timer (clean_up_dlq.timer)]']
+    provider => systemd
+    enable   => False
+    ensure   => stopped

Systemd::Service[nrpe2nodexp-opensearch_shards_9600]

Parameters differences:

--- Systemd::Service[nrpe2nodexp-opensearch_shards_9600].orig
+++ Systemd::Service[nrpe2nodexp-opensearch_shards_9600]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => absent
+    monitoring_critical      => False
+    require                  => Systemd::Unit[nrpe2nodexp-opensearch_shards_9600.service]
+    monitoring_enabled       => False
+    unit_type                => timer
+    restart                  => False

Prometheus::Node_lvs_realserver_mss[lvs_clamped_ipport]

Parameters differences:

--- Prometheus::Node_lvs_realserver_mss[lvs_clamped_ipport].orig
+++ Prometheus::Node_lvs_realserver_mss[lvs_clamped_ipport]

+    clamped_ipport => ['10.2.1.30:9200', '10.2.1.30:9243', '10.2.1.30:9643']
+    ensure         => present
+    outfile        => /var/lib/prometheus/node.d/lvs-realserver-mss.prom

Systemd::Unit[nrpe2nodexp-opensearch_shard_size_check_9200.service]

Parameters differences:

--- Systemd::Unit[nrpe2nodexp-opensearch_shard_size_check_9200.service].orig
+++ Systemd::Unit[nrpe2nodexp-opensearch_shard_size_check_9200.service]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => nrpe2nodexp-opensearch_shard_size_check_9200.service
+    ensure            => absent

Service[nrpe2nodexp-opensearch_shards_9200.timer]

Parameters differences:

--- Service[nrpe2nodexp-opensearch_shards_9200.timer].orig
+++ Service[nrpe2nodexp-opensearch_shards_9200.timer]

+    before   => ['Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9200.timer (nrpe2nodexp-opensearch_shards_9200.timer)]']
+    provider => systemd
+    enable   => False
+    ensure   => stopped

File[/etc/nginx/sites-available/production-search-codfw]

Parameters differences:

--- File[/etc/nginx/sites-available/production-search-codfw].orig
+++ File[/etc/nginx/sites-available/production-search-codfw]

+    notify => Exec[nginx-reload]
+    group  => root
+    before => ['Service[nginx]']
+    owner  => root
+    ensure => present

Content differences:

--- /etc/nginx/sites-available/production-search-codfw.orig
+++ /etc/nginx/sites-available/production-search-codfw
@@ -0,0 +1,60 @@
+# Proxy site configuration file for production-search-codfw
+# This file is managed by Puppet!
+upstream local_fe_production-search-codfw {
+    server 10.192.6.18:9200 max_fails=0;
+}
+
+# SSL proxying
+server {
+    # Enabling TCP Fast Open is safe for HTTP over TLS. There is no idempotency
+    # concern replaying TLS Client Hello.
+    # https://tools.ietf.org/html/rfc7413#section-6.3.2
+    listen [::]:9243 default_server deferred backlog=16384 reuseport ipv6only=on fastopen=150 ssl ;
+    listen 9243 default_server deferred backlog=16384 reuseport fastopen=150 ssl ;
+    ssl on;
+    server_name cirrussearch2089.codfw.wmnet search.discovery.wmnet search-chi.discovery.wmnet search-omega.discovery.wmnet search-psi.discovery.wmnet search.svc.codfw.wmnet search-chi.svc.codfw.wmnet search-omega.svc.codfw.wmnet search-psi.svc.codfw.wmnet;
+
+    error_log   /var/log/nginx/production-search-codfw.error.log;
+    access_log   off;
+
+    ssl_certificate /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chained.pem;
+    ssl_certificate_key /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet-key.pem;
+
+    keepalive_timeout 60;
+    keepalive_requests 100;
+
+    location / {
+        proxy_pass http://local_fe_production-search-codfw;
+        proxy_http_version 1.1;
+        proxy_set_header Connection close; # should be default
+
+        # this should be in sync with Varnish's first_byte_timeout
+        # and PHP's max_execution_time
+        proxy_read_timeout 180s;
+
+        proxy_set_header Host $host;
+        proxy_set_header X-Client-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto https;
+        proxy_set_header X-Connection-Properties "H2=$h2; SSR=$session_reused; SSL=$ssl_protocol; C=$ssl_cipher;";
+
+        # Don't clobber the Server header from the backend.
+        proxy_pass_header Server;
+
+        proxy_redirect off;
+        proxy_buffering off;
+        proxy_request_buffering on;
+
+        # With proxy buffering off, proxy_buffer_size is still used to
+        # define the singular chunk of buffer the content passes
+        # through.  Apparently if nginx can't read all the response
+        # headers in the first such chunk, it fails with 502.  Upping
+        # this to 8k (from def 4k) solves the one reported case in
+        # https://phabricator.wikimedia.org/T161819
+        # --
+        # Raising this further from 8K to 32K, at least temporarily, due to T226840
+        # Note 32KB is also the similar default limit in our varnishes (http_resp_size)
+        proxy_buffer_size       32k;
+        proxy_buffers         4 32k;
+    }
+}

Exec[systemd daemon-reload for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service (wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service)]

Parameters differences:

--- Exec[systemd daemon-reload for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service (wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service)].orig
+++ Exec[systemd daemon-reload for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service (wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

File[/etc/cfssl/csr/discovery__cirrussearch2089_codfw_wmnet.csr]

Parameters differences:

--- File[/etc/cfssl/csr/discovery__cirrussearch2089_codfw_wmnet.csr].orig
+++ File[/etc/cfssl/csr/discovery__cirrussearch2089_codfw_wmnet.csr]

+    mode   => 0400
+    group  => root
+    owner  => root
+    ensure => file

Content differences:

--- /etc/cfssl/csr/discovery__cirrussearch2089_codfw_wmnet.csr.orig
+++ /etc/cfssl/csr/discovery__cirrussearch2089_codfw_wmnet.csr
@@ -0,0 +1,21 @@
+{
+  "CN": "cirrussearch2089.codfw.wmnet",
+  "hosts": [
+    "search.discovery.wmnet",
+    "search-chi.discovery.wmnet",
+    "search-omega.discovery.wmnet",
+    "search-psi.discovery.wmnet",
+    "search.svc.codfw.wmnet",
+    "search-chi.svc.codfw.wmnet",
+    "search-omega.svc.codfw.wmnet",
+    "search-psi.svc.codfw.wmnet",
+    "cirrussearch2089.codfw.wmnet"
+  ],
+  "key": {
+    "algo": "ecdsa",
+    "size": 256
+  },
+  "names": [
+
+  ]
+}

Systemd::Service[prometheus_ferm_mss]

Parameters differences:

--- Systemd::Service[prometheus_ferm_mss].orig
+++ Systemd::Service[prometheus_ferm_mss]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => present
+    monitoring_critical      => False
+    require                  => Systemd::Unit[prometheus_ferm_mss.service]
+    monitoring_enabled       => False
+    unit_type                => timer
+    restart                  => False

Systemd::Service[opensearch-disable-readahead-production-search-codfw]

Parameters differences:

--- Systemd::Service[opensearch-disable-readahead-production-search-codfw].orig
+++ Systemd::Service[opensearch-disable-readahead-production-search-codfw]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => present
+    monitoring_critical      => False
+    require                  => Systemd::Unit[opensearch-disable-readahead-production-search-codfw.service]
+    monitoring_enabled       => False
+    unit_type                => timer
+    restart                  => False

File[/etc/elasticsearch]

Parameters differences:

--- File[/etc/elasticsearch].orig
+++ File[/etc/elasticsearch]

+    target  => /etc/opensearch
+    require => File[/etc/opensearch/instances]
+    group   => root
+    owner   => root
+    ensure  => link

File[/etc/ferm/conf.d/10_ipip]

Parameters differences:

--- File[/etc/ferm/conf.d/10_ipip].orig
+++ File[/etc/ferm/conf.d/10_ipip]

+    mode    => 0400
+    require => File[/etc/ferm/conf.d]
+    tag     => ferm
+    notify  => Service[ferm]
+    group   => root
+    owner   => root
+    ensure  => present

Content differences:

--- /etc/ferm/conf.d/10_ipip.orig
+++ /etc/ferm/conf.d/10_ipip
@@ -0,0 +1,11 @@
+# Autogenerated by puppet. DO NOT EDIT BY HAND!
+#
+# 10_ipip: 
+
+domain (ip) {
+	table filter {
+		chain INPUT {
+			saddr 172.16.0.0/12 proto ipencap ACCEPT;
+		}
+	}
+}

File[/etc/opensearch/production-search-codfw/jvm.options]

Parameters differences:

--- File[/etc/opensearch/production-search-codfw/jvm.options].orig
+++ File[/etc/opensearch/production-search-codfw/jvm.options]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => file

Content differences:

--- /etc/opensearch/production-search-codfw/jvm.options.orig
+++ /etc/opensearch/production-search-codfw/jvm.options
@@ -0,0 +1,109 @@
+## JVM configuration
+
+################################################################
+## IMPORTANT: JVM heap size
+################################################################
+##
+## You should always set the min and max JVM heap
+## size to the same value. For example, to set
+## the heap to 4 GB, set:
+##
+## -Xms4g
+## -Xmx4g
+##
+## See https://opensearch.org/docs/opensearch/install/important-settings/
+## for more information
+##
+################################################################
+
+# Xms represents the initial size of total heap space
+# Xmx represents the maximum size of total heap space
+
+-Xms30G
+-Xmx30G
+
+################################################################
+## Numa Awareness
+################################################################
+-XX:+UseNUMA
+
+################################################################
+## Expert settings
+################################################################
+##
+## All settings below this section are considered
+## expert settings. Don't tamper with them unless
+## you understand what you are doing
+##
+################################################################
+
+
+## optimizations
+
+# disable calls to System#gc
+-XX:+DisableExplicitGC
+
+# pre-touch memory pages used by the JVM during initialization
+-XX:+AlwaysPreTouch
+
+## basic
+
+# force the server VM (remove on 32-bit client JVMs)
+-server
+
+# explicitly set the stack size (reduce to 320k on 32-bit client JVMs)
+-Xss1m
+
+# log4j 2
+-Dlog4j.skipJansi=true
+-Dlog4j2.formatMsgNoLookups=true
+
+# security policy, needed for OpenSearch to access non-default files/sockets/ports T395571
+
+-Djava.security.policy=/etc/opensearch/production-search-codfw/java-security.policy
+
+## heap dumps
+
+# generate a heap dump when an allocation from the Java heap fails
+# heap dumps are created in the working directory of the JVM
+-XX:+HeapDumpOnOutOfMemoryError
+
+# specify an alternative path for heap dumps
+# ensure the directory exists and has sufficient space
+-XX:HeapDumpPath=/srv/opensearch/production-search-codfw
+
+## GC logging
+
+#-XX:+PrintGCDetails
+#-XX:+PrintGCTimeStamps
+#-XX:+PrintGCDateStamps
+#-XX:+PrintClassHistogram
+#-XX:+PrintTenuringDistribution
+#-XX:+PrintGCApplicationStoppedTime
+
+# log GC status to a file with time stamps
+# ensure the directory exists
+#-Xloggc:${loggc}
+
+## Custom Wikimedia configuration
+
+# Defines a limit of proportion of time spent in GC before
+# an OutOfMemory error is thrown.  The goal is to bail and
+# restart the server if throughput is suffering due to
+# GC stalls.
+# TODO: Why 70? why not 60 or 80?
+-XX:GCTimeLimit=70
+
+# Defines minimum percentage of free space after a full GC
+# before an OutOfMemoryError is thrown. Typically a full GC
+# should leave > 50% of heap free. If we are down to 10%
+# something is wrong and bailing is a reasonable solution.
+-XX:GCHeapFreeLimit=10
+
+# Exit on OutOfMemoryError, since in most cases the JVM does not recover
+-XX:+ExitOnOutOfMemoryError
+
+# Custom WMF GC flags
+-Xlog:gc*:file=/var/log/opensearch/production-search-codfw_jvm_gc.%p.log::filecount=10,filesize=20000
+-Xlog:gc+age=trace
+-Xlog:safepoint

Opensearch::Instance[production-search-codfw]

Parameters differences:

--- Opensearch::Instance[production-search-codfw].orig
+++ Opensearch::Instance[production-search-codfw]

+    script_max_compilations_per_minute => 10000
+    heap_memory                        => 30G
+    indices_to_monitor                 => ['commonswiki_file', 'wikidatawiki_content', 'enwiki_general', 'enwiki_content', 'viwiki_general', 'commonswiki_general', 'cebwiki_content', 'metawiki_general', 'dewiki_content', 'frwiki_content']
+    ltr_cache_size                     => 100mb
+    plugins_dir                        => /usr/share/opensearch/plugins
+    compatibility_mode                 => False
+    gc_log                             => True
+    require                            => Package[opensearch]
+    logstash_logback_port              => 11514
+    bind_networks                      => ['_local_', '_site_']
+    cluster_name                       => production-search-codfw
+    awareness_attributes               => row
+    certificate_name                   => search.discovery.wmnet
+    tls_port                           => 9243
+    filter_cache_size                  => 20%
+    rack                               => A4
+    node_name                          => cirrussearch2089-production-search-codfw
+    transport_tcp_port                 => 9300
+    watermarks                         => {}
+    http_port                          => 9200
+    bulk_thread_pool_capacity          => 1000
+    logstash_host                      => localhost
+    short_cluster_name                 => chi
+    bulk_thread_pool_executors         => 6
+    disable_security_plugin            => False
+    use_cms_gc                         => False
+    auto_create_index                  => +apifeatureusage-*,+glent_*,-*
+    send_logs_to_logstash              => True
+    cms_gc_init_occupancy_fraction     => 75
+    plugins_mandatory                  => ['analysis-hebrew', 'analysis-icu', 'analysis-smartcn', 'analysis-stempel', 'analysis-ukrainian', 'cirrus-highlighter', 'opensearch-analysis-stconvert', 'opensearch-extra', 'opensearch-extra-analysis-esperanto', 'opensearch-extra-analysis-serbian', 'opensearch-extra-analysis-slovak', 'ltr']
+    holds_data                         => True
+    expected_nodes                     => 36
+    recover_after_time                 => 5m
+    search_shard_count_limit           => 5000
+    cluster_hosts                      => ['cirrussearch2061.codfw.wmnet', 'cirrussearch2062.codfw.wmnet', 'cirrussearch2063.codfw.wmnet', 'cirrussearch2064.codfw.wmnet', 'cirrussearch2065.codfw.wmnet', 'cirrussearch2066.codfw.wmnet', 'cirrussearch2067.codfw.wmnet', 'cirrussearch2068.codfw.wmnet', 'cirrussearch2069.codfw.wmnet', 'cirrussearch2070.codfw.wmnet', 'cirrussearch2071.codfw.wmnet', 'cirrussearch2072.codfw.wmnet', 'cirrussearch2073.codfw.wmnet', 'cirrussearch2074.codfw.wmnet', 'cirrussearch2075.codfw.wmnet', 'cirrussearch2076.codfw.wmnet', 'cirrussearch2077.codfw.wmnet', 'cirrussearch2078.codfw.wmnet', 'cirrussearch2079.codfw.wmnet', 'cirrussearch2080.codfw.wmnet', 'cirrussearch2081.codfw.wmnet', 'cirrussearch2082.codfw.wmnet', 'cirrussearch2083.codfw.wmnet', 'cirrussearch2084.codfw.wmnet', 'cirrussearch2085.codfw.wmnet', 'cirrussearch2086.codfw.wmnet', 'cirrussearch2087.codfw.wmnet', 'cirrussearch2088.codfw.wmnet', 'cirrussearch2089.codfw.wmnet', 'cirrussearch2090.codfw.wmnet', 'cirrussearch2091.codfw.wmnet', 'cirrussearch2092.codfw.wmnet', 'cirrussearch2093.codfw.wmnet', 'cirrussearch2094.codfw.wmnet', 'cirrussearch2095.codfw.wmnet', 'cirrussearch2096.codfw.wmnet', 'cirrussearch2097.codfw.wmnet', 'cirrussearch2098.codfw.wmnet', 'cirrussearch2099.codfw.wmnet', 'cirrussearch2100.codfw.wmnet', 'cirrussearch2101.codfw.wmnet', 'cirrussearch2102.codfw.wmnet', 'cirrussearch2103.codfw.wmnet', 'cirrussearch2104.codfw.wmnet', 'cirrussearch2105.codfw.wmnet', 'cirrussearch2106.codfw.wmnet', 'cirrussearch2107.codfw.wmnet', 'cirrussearch2108.codfw.wmnet', 'cirrussearch2109.codfw.wmnet', 'cirrussearch2110.codfw.wmnet', 'cirrussearch2111.codfw.wmnet', 'cirrussearch2112.codfw.wmnet', 'cirrussearch2113.codfw.wmnet', 'cirrussearch2114.codfw.wmnet', 'cirrussearch2115.codfw.wmnet']
+    recover_after_nodes                => 24
+    curator_uses_unicast_hosts         => True
+    publish_host                       => 10.192.6.18
+    version                            => 1
+    row                                => codfw-row-a
+    configure_curator                  => False
+    load_fixed_bitset_filters_eagerly  => True
+    unicast_hosts                      => ['cirrussearch2093.codfw.wmnet', 'cirrussearch2061.codfw.wmnet', 'cirrussearch2081.codfw.wmnet', 'cirrussearch2084.codfw.wmnet', 'cirrussearch2111.codfw.wmnet']
+    base_data_dir                      => /srv/opensearch

Systemd::Timer[opensearch-disable-readahead-production-search-codfw]

Parameters differences:

--- Systemd::Timer[opensearch-disable-readahead-production-search-codfw].orig
+++ Systemd::Timer[opensearch-disable-readahead-production-search-codfw]

+    fixed_random_delay => False
+    unit_name          => opensearch-disable-readahead-production-search-codfw.service
+    timer_intervals    => [{'start': 'OnUnitActiveSec', 'interval': '30min'}, {'start': 'OnBootSec', 'interval': '1min'}]
+    splay              => 0
+    accuracy           => 15sec
+    ensure             => present

Exec[systemd daemon-reload for nrpe2nodexp-check_tcp-mss-clamper_status.service (nrpe2nodexp-check_tcp-mss-clamper_status.service)]

Parameters differences:

--- Exec[systemd daemon-reload for nrpe2nodexp-check_tcp-mss-clamper_status.service (nrpe2nodexp-check_tcp-mss-clamper_status.service)].orig
+++ Exec[systemd daemon-reload for nrpe2nodexp-check_tcp-mss-clamper_status.service (nrpe2nodexp-check_tcp-mss-clamper_status.service)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

Admin::User[pfischer]

Parameters differences:

--- Admin::User[pfischer].orig
+++ Admin::User[pfischer]

+    uid      => 41223
+    groups   => []
+    gid      => 500
+    ensure   => present
+    ssh_keys => ['ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFjit1wuGOgmJNp2JMHRcy6LyfJWXwjacns04JXyXyLA pfischer production']
+    home_dir => /home/pfischer
+    comment  => Peter Fischer
+    shell    => /bin/bash

Class[Profile::Cumin::Target]

Parameters differences:

--- Class[Profile::Cumin::Target].orig
+++ Class[Profile::Cumin::Target]

@@
-    cluster => insetup
+    cluster => elasticsearch

Package[wikimedia-lvs-realserver]

Parameters differences:

--- Package[wikimedia-lvs-realserver].orig
+++ Package[wikimedia-lvs-realserver]

+    require  => File[/etc/default/wikimedia-lvs-realserver]
+    provider => apt
+    ensure   => present

Sudo::User[nrpe-check_check_tcp-mss-clamper_status]

Parameters differences:

--- Sudo::User[nrpe-check_check_tcp-mss-clamper_status].orig
+++ Sudo::User[nrpe-check_check_tcp-mss-clamper_status]

+    privileges => []
+    tag        => nrpe::check
+    require    => ['Class[Sudo]']
+    user       => nagios
+    ensure     => absent

Java::Cacert[wmf:puppetca.pem]

Parameters differences:

--- Java::Cacert[wmf:puppetca.pem].orig
+++ Java::Cacert[wmf:puppetca.pem]

+    require   => Alternatives::Java[11]
+    path      => /usr/share/ca-certificates/wikimedia/Puppet5_Internal_CA.crt
+    group     => root
+    owner     => root
+    storepass => changeit
+    ensure    => present

Systemd::Unit[nrpe2nodexp-opensearch_unassigned_shard_check_9600.service]

Parameters differences:

--- Systemd::Unit[nrpe2nodexp-opensearch_unassigned_shard_check_9600.service].orig
+++ Systemd::Unit[nrpe2nodexp-opensearch_unassigned_shard_check_9600.service]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => nrpe2nodexp-opensearch_unassigned_shard_check_9600.service
+    ensure            => absent

Nginx::Site[production-search-psi-codfw]

Parameters differences:

--- Nginx::Site[production-search-psi-codfw].orig
+++ Nginx::Site[production-search-psi-codfw]

+    require => Exec[tlsproxy localssl default_server on ports [9643]]
+    ensure  => present

Package[python3-conftool]

Parameters differences:

--- Package[python3-conftool].orig
+++ Package[python3-conftool]

+    provider => apt
+    ensure   => installed

Exec[mask_default_prometheus_elasticsearch_exporter]

Parameters differences:

--- Exec[mask_default_prometheus_elasticsearch_exporter].orig
+++ Exec[mask_default_prometheus_elasticsearch_exporter]

+    creates => /etc/systemd/system/prometheus-elasticsearch-exporter.service
+    command => /bin/systemctl mask prometheus-elasticsearch-exporter.service

File[/etc/default/opensearch]

Parameters differences:

--- File[/etc/default/opensearch].orig
+++ File[/etc/default/opensearch]

+    mode    => 0444
+    require => Package[opensearch]
+    group   => root
+    owner   => root
+    ensure  => file

Content differences:

--- /etc/default/opensearch.orig
+++ /etc/default/opensearch
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: Apache-2.0
+# Default env file provided by upstream always reset OPENSEARCH_PATH_CONF
+# when called from /usr/share/opensearch/bin/opensearch-env
+# only set it when not defined.
+# This file is still required to set OPENSEARCH_PATH_CONF by postinst scripts
+OPENSEARCH_PATH_CONF=${OPENSEARCH_PATH_CONF-/etc/opensearch}

File[/etc/nagios/nrpe.d/check_opensearch_unassigned_shard_check_9600.cfg]

Parameters differences:

--- File[/etc/nagios/nrpe.d/check_opensearch_unassigned_shard_check_9600.cfg].orig
+++ File[/etc/nagios/nrpe.d/check_opensearch_unassigned_shard_check_9600.cfg]

+    mode    => 0444
+    require => Package[nagios-nrpe-server]
+    tag     => nrpe::check
+    notify  => Service[nagios-nrpe-server]
+    group   => root
+    owner   => root
+    ensure  => present

Content differences:

--- /etc/nagios/nrpe.d/check_opensearch_unassigned_shard_check_9600.cfg.orig
+++ /etc/nagios/nrpe.d/check_opensearch_unassigned_shard_check_9600.cfg
@@ -0,0 +1,2 @@
+# File generated by puppet. DO NOT edit by hand
+command[check_opensearch_unassigned_shard_check_9600]=/usr/lib/nagios/plugins/check_elasticsearch_unassigned_shards.py --url http://localhost:9600 --timeout 4

File[/etc/opensearch/production-search-codfw/logging.yml]

Parameters differences:

--- File[/etc/opensearch/production-search-codfw/logging.yml].orig
+++ File[/etc/opensearch/production-search-codfw/logging.yml]

+    group  => root
+    owner  => root
+    ensure => absent

Admin::Hashuser[pfischer]

Parameters differences:

--- Admin::Hashuser[pfischer].orig
+++ Admin::Hashuser[pfischer]

+    ensure_ssh_key => True

Monitoring::Exported_nagios_service[cirrussearch2089 raid_md]

Parameters differences:

--- Monitoring::Exported_nagios_service[cirrussearch2089 raid_md].orig
+++ Monitoring::Exported_nagios_service[cirrussearch2089 raid_md]

@@
-    notifications_enabled => 0
+    notifications_enabled => 1
@@
-    servicegroups         => insetup_codfw
+    servicegroups         => elasticsearch_codfw
@@
-    contact_groups        => admins
+    contact_groups        => team-data-platform,team-discovery

Exec[/usr/sbin/tc qdisc del dev eno12399np0 clsact]

Parameters differences:

--- Exec[/usr/sbin/tc qdisc del dev eno12399np0 clsact].orig
+++ Exec[/usr/sbin/tc qdisc del dev eno12399np0 clsact]

+    onlyif => /usr/sbin/tc qdisc show dev eno12399np0 | grep -q clsact

File[/etc/udev/rules.d/40-opensearch-readahead.rules]

Parameters differences:

--- File[/etc/udev/rules.d/40-opensearch-readahead.rules].orig
+++ File[/etc/udev/rules.d/40-opensearch-readahead.rules]

+    mode   => 0444
+    notify => Exec[udev_reload]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/udev/rules.d/40-opensearch-readahead.rules.orig
+++ /etc/udev/rules.d/40-opensearch-readahead.rules
@@ -0,0 +1 @@
+SUBSYSTEM=="block", KERNEL=="md1", ACTION=="add|change", ATTR{bdi/read_ahead_kb}="16"

Exec[disable-rp-filter-eno12399np0]

Parameters differences:

--- Exec[disable-rp-filter-eno12399np0].orig
+++ Exec[disable-rp-filter-eno12399np0]

+    unless  => /usr/sbin/sysctl -n net.ipv4.conf.eno12399np0.rp_filter |grep -- '0'
+    command => /usr/sbin/sysctl -q net.ipv4.conf.eno12399np0.rp_filter=0

Systemd::Timer[clean_up_dlq]

Parameters differences:

--- Systemd::Timer[clean_up_dlq].orig
+++ Systemd::Timer[clean_up_dlq]

+    fixed_random_delay => False
+    unit_name          => clean_up_dlq.service
+    timer_intervals    => [{'start': 'OnCalendar', 'interval': '1h'}]
+    splay              => 0
+    accuracy           => 15sec
+    ensure             => absent

File[/etc/opensearch/production-search-codfw/java-security.policy]

Parameters differences:

--- File[/etc/opensearch/production-search-codfw/java-security.policy].orig
+++ File[/etc/opensearch/production-search-codfw/java-security.policy]

+    mode   => 0440
+    source => puppet:///modules/profile/opensearch/java-security.policy
+    group  => opensearch
+    owner  => opensearch
+    ensure => file

Exec[disable-rp-filter-ipip0]

Parameters differences:

--- Exec[disable-rp-filter-ipip0].orig
+++ Exec[disable-rp-filter-ipip0]

+    unless  => /usr/sbin/sysctl -n net.ipv4.conf.ipip0.rp_filter |grep -- '0'
+    command => /usr/sbin/sysctl -q net.ipv4.conf.ipip0.rp_filter=0
+    require => Interface::Ipip[ipip_ipv4]

Service[nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer]

Parameters differences:

--- Service[nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer].orig
+++ Service[nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer]

+    before   => ['Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer (nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer)]']
+    provider => systemd
+    enable   => False
+    ensure   => stopped

File[/etc/rsyslog.d/40-wmf-auto-restart-prometheus-elasticsearch-exporter-9200.conf]

Parameters differences:

--- File[/etc/rsyslog.d/40-wmf-auto-restart-prometheus-elasticsearch-exporter-9200.conf].orig
+++ File[/etc/rsyslog.d/40-wmf-auto-restart-prometheus-elasticsearch-exporter-9200.conf]

+    mode   => 0444
+    notify => Service[rsyslog]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/rsyslog.d/40-wmf-auto-restart-prometheus-elasticsearch-exporter-9200.conf.orig
+++ /etc/rsyslog.d/40-wmf-auto-restart-prometheus-elasticsearch-exporter-9200.conf
@@ -0,0 +1,10 @@
+# rsyslog.conf(5) configuration file for services.
+# This file is managed by Puppet.
+if $programname startswith "wmf_auto_restart_prometheus-elasticsearch-exporter-9200" then {
+    action(
+        type="omfile" file="/var/log/wmf_auto_restart_prometheus-elasticsearch-exporter-9200/syslog.log"
+        fileOwner="root" fileGroup="root"
+        fileCreateMode="0644"
+    )
+    & stop
+}

Monitoring::Exported_nagios_host[cirrussearch2089]

Parameters differences:

--- Monitoring::Exported_nagios_host[cirrussearch2089].orig
+++ Monitoring::Exported_nagios_host[cirrussearch2089]

@@
-    hostgroups            => insetup_codfw,lsw1-a4-codfw
+    hostgroups            => elasticsearch_codfw,lsw1-a4-codfw
@@
-    notifications_enabled => 0
+    notifications_enabled => 1
@@
-    contact_groups        => admins
+    contact_groups        => team-data-platform,team-discovery

File[/etc/rsyslog.d/50-udp-json-logback-compat.conf]

Parameters differences:

--- File[/etc/rsyslog.d/50-udp-json-logback-compat.conf].orig
+++ File[/etc/rsyslog.d/50-udp-json-logback-compat.conf]

+    mode   => 0444
+    notify => Service[rsyslog]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/rsyslog.d/50-udp-json-logback-compat.conf.orig
+++ /etc/rsyslog.d/50-udp-json-logback-compat.conf
@@ -0,0 +1,43 @@
+# Provide a UDP input to accept JSON payloads on localhost and forward them to logstash via Kakfa.
+
+module(load="imudp")
+module(load="mmjsonparse")
+module(load="omkafka")
+
+template(name="template_udp_json_logback_compat" type="list") {
+  property(name="$!all-json")
+}
+
+# Use the parsed json "level" field value as the kafka topic suffix
+template(name="udp_json_logback_compat_topic" type="string" string="logback-%!level:::lowercase%")
+
+# Use a separate (in memory) queue to limit message processing to this ruleset only.
+ruleset(name="ruleset_udp_json_logback_compat" queue.type="LinkedList") {
+
+  action(type="mmjsonparse" name="mmjsonparse_udp_json_logback_compat" cookie="" useRawMsg="on")
+
+  if $parsesuccess == "OK" then {
+    action(type="omkafka"
+           broker=["kafka-logging1001.eqiad.wmnet:9093","kafka-logging1002.eqiad.wmnet:9093","kafka-logging1003.eqiad.wmnet:9093","kafka-logging1004.eqiad.wmnet:9093","kafka-logging1005.eqiad.wmnet:9093"]
+           topic="udp_json_logback_compat_topic"
+           dynatopic="on"
+           dynatopic.cachesize="1000"
+           partitions.auto="on"
+           template="template_udp_json_logback_compat"
+           queue.type="LinkedList" queue.size="10000" queue.filename="udp_json_logback_compat"
+           queue.highWatermark="7000" queue.lowWatermark="6000"
+           queue.checkpointInterval="5"
+           queue.maxDiskSpace="40960000"
+           confParam=[ "security.protocol=ssl",
+                       "ssl.ca.location=/etc/ssl/certs/wmf-ca-certificates.crt",
+                       "compression.codec=snappy",
+                       "socket.timeout.ms=60000",
+                       "socket.keepalive.enable=true",
+                       "queue.buffering.max.ms=50",
+                       "batch.num.messages=1000" ]
+    )
+  }
+
+}
+
+input(type="imudp" port="11514" address="localhost" ruleset="ruleset_udp_json_logback_compat")

File[/lib/systemd/system/prometheus_ferm_mss.service]

Parameters differences:

--- File[/lib/systemd/system/prometheus_ferm_mss.service].orig
+++ File[/lib/systemd/system/prometheus_ferm_mss.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for prometheus_ferm_mss.service (prometheus_ferm_mss.service)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/prometheus_ferm_mss.service.orig
+++ /lib/systemd/system/prometheus_ferm_mss.service
@@ -0,0 +1,8 @@
+[Unit]
+Description=Regular job to collect MSS values of ferm-based hosts
+Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+
+[Service]
+Type=oneshot
+User=root
+ExecStart=/usr/local/bin/prometheus-ferm-mss -o /var/lib/prometheus/node.d/ferm-mss.prom -e 10.2.1.30:9200 -e 10.2.1.30:9243 -e 10.2.1.30:9643

Opensearch::Instance[production-search-psi-codfw]

Parameters differences:

--- Opensearch::Instance[production-search-psi-codfw].orig
+++ Opensearch::Instance[production-search-psi-codfw]

+    script_max_compilations_per_minute => 10000
+    heap_memory                        => 10G
+    ltr_cache_size                     => 100mb
+    plugins_dir                        => /usr/share/opensearch/plugins
+    compatibility_mode                 => False
+    gc_log                             => True
+    require                            => Package[opensearch]
+    logstash_logback_port              => 11514
+    bind_networks                      => ['_local_', '_site_']
+    cluster_name                       => production-search-psi-codfw
+    awareness_attributes               => row
+    certificate_name                   => search.discovery.wmnet
+    tls_port                           => 9643
+    filter_cache_size                  => 20%
+    rack                               => A4
+    node_name                          => cirrussearch2089-production-search-psi-codfw
+    transport_tcp_port                 => 9700
+    watermarks                         => {}
+    http_port                          => 9600
+    bulk_thread_pool_capacity          => 1000
+    logstash_host                      => localhost
+    short_cluster_name                 => psi
+    bulk_thread_pool_executors         => 6
+    disable_security_plugin            => False
+    use_cms_gc                         => False
+    auto_create_index                  => +apifeatureusage-*,+glent_*,-*
+    send_logs_to_logstash              => True
+    cms_gc_init_occupancy_fraction     => 75
+    plugins_mandatory                  => ['analysis-hebrew', 'analysis-icu', 'analysis-smartcn', 'analysis-stempel', 'analysis-ukrainian', 'cirrus-highlighter', 'opensearch-analysis-stconvert', 'opensearch-extra', 'opensearch-extra-analysis-esperanto', 'opensearch-extra-analysis-serbian', 'opensearch-extra-analysis-slovak', 'ltr']
+    holds_data                         => True
+    expected_nodes                     => 18
+    recover_after_time                 => 5m
+    search_shard_count_limit           => 5000
+    cluster_hosts                      => ['cirrussearch2089.codfw.wmnet', 'cirrussearch2062.codfw.wmnet', 'cirrussearch2069.codfw.wmnet', 'cirrussearch2075.codfw.wmnet', 'cirrussearch2076.codfw.wmnet', 'cirrussearch2091.codfw.wmnet', 'cirrussearch2094.codfw.wmnet', 'cirrussearch2095.codfw.wmnet', 'cirrussearch2064.codfw.wmnet', 'cirrussearch2078.codfw.wmnet', 'cirrussearch2079.codfw.wmnet', 'cirrussearch2080.codfw.wmnet', 'cirrussearch2096.codfw.wmnet', 'cirrussearch2110.codfw.wmnet', 'cirrussearch2101.codfw.wmnet', 'cirrussearch2066.codfw.wmnet', 'cirrussearch2071.codfw.wmnet', 'cirrussearch2072.codfw.wmnet', 'cirrussearch2082.codfw.wmnet', 'cirrussearch2083.codfw.wmnet', 'cirrussearch2103.codfw.wmnet', 'cirrussearch2113.codfw.wmnet', 'cirrussearch2107.codfw.wmnet', 'cirrussearch2108.codfw.wmnet', 'cirrussearch2068.codfw.wmnet', 'cirrussearch2085.codfw.wmnet', 'cirrussearch2109.codfw.wmnet', 'cirrussearch2115.codfw.wmnet']
+    recover_after_nodes                => 11
+    curator_uses_unicast_hosts         => True
+    publish_host                       => 10.192.6.18
+    version                            => 1
+    row                                => codfw-row-a
+    configure_curator                  => False
+    load_fixed_bitset_filters_eagerly  => True
+    unicast_hosts                      => ['cirrussearch2108.codfw.wmnet', 'cirrussearch2076.codfw.wmnet', 'cirrussearch2080.codfw.wmnet', 'cirrussearch2083.codfw.wmnet', 'cirrussearch2115.codfw.wmnet']
+    base_data_dir                      => /srv/opensearch

File[/usr/local/bin/depool-opensearch_1@production-search-psi-codfw]

Parameters differences:

--- File[/usr/local/bin/depool-opensearch_1@production-search-psi-codfw].orig
+++ File[/usr/local/bin/depool-opensearch_1@production-search-psi-codfw]

+    mode   => 0555
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /usr/local/bin/depool-opensearch_1@production-search-psi-codfw.orig
+++ /usr/local/bin/depool-opensearch_1@production-search-psi-codfw
@@ -0,0 +1,2 @@
+#!/bin/bash
+/usr/local/bin/safe-service-restart --pools search-psi-https --depool --retries 10 --wait 5

Exec[systemd daemon-reload for prometheus_lvs_realserver_mss.service (prometheus_lvs_realserver_mss.service)]

Parameters differences:

--- Exec[systemd daemon-reload for prometheus_lvs_realserver_mss.service (prometheus_lvs_realserver_mss.service)].orig
+++ Exec[systemd daemon-reload for prometheus_lvs_realserver_mss.service (prometheus_lvs_realserver_mss.service)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

Admin::Hashuser[dcausse]

Parameters differences:

--- Admin::Hashuser[dcausse].orig
+++ Admin::Hashuser[dcausse]

+    ensure_ssh_key => True

User[dr0ptp4kt]

Parameters differences:

--- User[dr0ptp4kt].orig
+++ User[dr0ptp4kt]

+    allowdupe  => False
+    uid        => 2962
+    shell      => /bin/bash
+    gid        => 500
+    ensure     => present
+    home       => /home/dr0ptp4kt
+    groups     => []
+    comment    => Adam Baso
+    managehome => False

Mount[/var/lib/nginx]

Parameters differences:

--- Mount[/var/lib/nginx].orig
+++ Mount[/var/lib/nginx]

+    device  => tmpfs
+    dump    => 0
+    ensure  => mounted
+    pass    => 0
+    require => File[/var/lib/nginx]
+    options => defaults,noatime,uid=0,gid=0,mode=755,size=1g
+    fstype  => tmpfs
+    before  => Service[nginx]

Sudo::User[nrpe-check_opensearch_unassigned_shard_check_9600]

Parameters differences:

--- Sudo::User[nrpe-check_opensearch_unassigned_shard_check_9600].orig
+++ Sudo::User[nrpe-check_opensearch_unassigned_shard_check_9600]

+    privileges => []
+    tag        => nrpe::check
+    require    => ['Class[Sudo]']
+    user       => nagios
+    ensure     => absent

Interface::Ipip[ipip_ipv6]

Parameters differences:

--- Interface::Ipip[ipip_ipv6].orig
+++ Interface::Ipip[ipip_ipv6]

+    family    => inet6
+    interface => ipip60
+    ensure    => present

Exec[symlink /etc/opensearch/production-search-psi-codfw/sudachi]

Parameters differences:

--- Exec[symlink /etc/opensearch/production-search-psi-codfw/sudachi].orig
+++ Exec[symlink /etc/opensearch/production-search-psi-codfw/sudachi]

+    creates => /etc/opensearch/production-search-psi-codfw/sudachi
+    onlyif  => /usr/bin/test -d '/usr/share/opensearch/config/sudachi'
+    command => /usr/bin/ln -s '/usr/share/opensearch/config/sudachi' '/etc/opensearch/production-search-psi-codfw/sudachi'

Systemd::Timer[nrpe2nodexp-opensearch_unassigned_shard_check_9200]

Parameters differences:

--- Systemd::Timer[nrpe2nodexp-opensearch_unassigned_shard_check_9200].orig
+++ Systemd::Timer[nrpe2nodexp-opensearch_unassigned_shard_check_9200]

+    fixed_random_delay => False
+    unit_name          => nrpe2nodexp-opensearch_unassigned_shard_check_9200.service
+    timer_intervals    => [{'start': 'OnUnitInactiveSec', 'interval': '720min'}, {'interval': '1s', 'start': 'OnActiveSec'}]
+    splay              => 0
+    accuracy           => 15sec
+    ensure             => absent

Logstash::Conf[input-gelf-gelf_relay]

Parameters differences:

--- Logstash::Conf[input-gelf-gelf_relay].orig
+++ Logstash::Conf[input-gelf-gelf_relay]

+    priority => 10
+    ensure   => present

Motd::Script[insetup::data_platform_ferm]

Parameters differences:

--- Motd::Script[insetup::data_platform_ferm].orig
+++ Motd::Script[insetup::data_platform_ferm]

-    priority => 5
-    ensure   => present

File[/lib/systemd/system/prometheus_ferm_mss.timer]

Parameters differences:

--- File[/lib/systemd/system/prometheus_ferm_mss.timer].orig
+++ File[/lib/systemd/system/prometheus_ferm_mss.timer]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for prometheus_ferm_mss.timer (prometheus_ferm_mss.timer)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/prometheus_ferm_mss.timer.orig
+++ /lib/systemd/system/prometheus_ferm_mss.timer
@@ -0,0 +1,12 @@
+[Unit]
+Description=Periodic execution of prometheus_ferm_mss.service
+
+[Timer]
+Unit=prometheus_ferm_mss.service
+# Accuracy sets the maximum time interval around the execution time we want to allow
+AccuracySec=15sec
+OnCalendar=minutely
+RandomizedDelaySec=0
+
+[Install]
+WantedBy=multi-user.target

Exec[opensearch-create-keystore-production-search-codfw]

Parameters differences:

--- Exec[opensearch-create-keystore-production-search-codfw].orig
+++ Exec[opensearch-create-keystore-production-search-codfw]

+    environment => ['OPENSEARCH_PATH_CONF=/etc/opensearch/production-search-codfw']
+    creates     => /etc/opensearch/production-search-codfw/opensearch.keystore
+    command     => /usr/share/opensearch/bin/opensearch-keystore create
+    before      => File[/etc/opensearch/production-search-codfw/opensearch.keystore]

Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_unassigned_shard_check_9600]

Parameters differences:

--- Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_unassigned_shard_check_9600].orig
+++ Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_unassigned_shard_check_9600]

+    passive_checks_enabled => 1
+    is_volatile            => 0
+    contact_groups         => admins,team-discovery
+    notification_options   => c,r,f
+    service_description    => OpenSearch unassigned shard check - 9600
+    max_check_attempts     => 1
+    active_checks_enabled  => 1
+    notification_interval  => 0
+    check_command          => nrpe_check!check_opensearch_unassigned_shard_check_9600!10
+    retry_interval         => 120
+    check_freshness        => 0
+    notes_url              => https://wikitech.wikimedia.org/wiki/Search#Administration
+    notifications_enabled  => 1
+    check_period           => 24x7
+    ensure                 => present
+    check_interval         => 720
+    notification_period    => 24x7
+    host_name              => cirrussearch2089
+    servicegroups          => elasticsearch_codfw

Exec[systemd daemon-reload for nrpe2nodexp-check_tcp-mss-clamper_status.timer (nrpe2nodexp-check_tcp-mss-clamper_status.timer)]

Parameters differences:

--- Exec[systemd daemon-reload for nrpe2nodexp-check_tcp-mss-clamper_status.timer (nrpe2nodexp-check_tcp-mss-clamper_status.timer)].orig
+++ Exec[systemd daemon-reload for nrpe2nodexp-check_tcp-mss-clamper_status.timer (nrpe2nodexp-check_tcp-mss-clamper_status.timer)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

Class[Cumin::Selector]

Parameters differences:

--- Class[Cumin::Selector].orig
+++ Class[Cumin::Selector]

@@
-    cluster => insetup
+    cluster => elasticsearch

Systemd::Unit[nrpe2nodexp-opensearch_shard_size_check_9600.timer]

Parameters differences:

--- Systemd::Unit[nrpe2nodexp-opensearch_shard_size_check_9600.timer].orig
+++ Systemd::Unit[nrpe2nodexp-opensearch_shard_size_check_9600.timer]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => nrpe2nodexp-opensearch_shard_size_check_9600.timer
+    ensure            => absent

Systemd::Timer::Job[wmf_auto_restart_prometheus-elasticsearch-exporter-9200]

Parameters differences:

--- Systemd::Timer::Job[wmf_auto_restart_prometheus-elasticsearch-exporter-9200].orig
+++ Systemd::Timer::Job[wmf_auto_restart_prometheus-elasticsearch-exporter-9200]

+    description               => Auto restart job: prometheus-elasticsearch-exporter-9200
+    syslog_match_startswith   => True
+    monitoring_contact_groups => admins
+    logging_enabled           => True
+    success_exit_status       => []
+    require                   => File[/usr/local/sbin/wmf-auto-restart]
+    fixed_random_delay        => False
+    private_tmp               => False
+    logfile_basedir           => /var/log
+    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+    send_mail_only_on_error   => True
+    ignore_errors             => False
+    logfile_group             => root
+    interval                  => {'start': 'OnCalendar', 'interval': 'Mon,Tue,Wed,Thu,Fri *-*-* 2:54:00'}
+    send_mail                 => False
+    logfile_name              => syslog.log
+    user                      => root
+    ensure                    => present
+    send_mail_to              => root@cirrussearch2089.codfw.wmnet
+    logfile_perms             => all
+    syslog_force_stop         => True
+    environment               => {}
+    command                   => /usr/local/sbin/wmf-auto-restart -s prometheus-elasticsearch-exporter-9200
+    monitoring_enabled        => False

Systemd::Service[prometheus_lvs_realserver_mss]

Parameters differences:

--- Systemd::Service[prometheus_lvs_realserver_mss].orig
+++ Systemd::Service[prometheus_lvs_realserver_mss]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => present
+    monitoring_critical      => False
+    require                  => Systemd::Unit[prometheus_lvs_realserver_mss.service]
+    monitoring_enabled       => False
+    unit_type                => timer
+    restart                  => False

Sudo::User[nrpe-check_opensearch_shard_size_check_9600]

Parameters differences:

--- Sudo::User[nrpe-check_opensearch_shard_size_check_9600].orig
+++ Sudo::User[nrpe-check_opensearch_shard_size_check_9600]

+    privileges => []
+    tag        => nrpe::check
+    require    => ['Class[Sudo]']
+    user       => nagios
+    ensure     => absent

Systemd::Unit[opensearch-hot-threads-log.timer]

Parameters differences:

--- Systemd::Unit[opensearch-hot-threads-log.timer].orig
+++ Systemd::Unit[opensearch-hot-threads-log.timer]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => opensearch-hot-threads-log.timer
+    ensure            => present

File[/etc/ferm/conf.d/10_opensearch-inter-node-9700]

Parameters differences:

--- File[/etc/ferm/conf.d/10_opensearch-inter-node-9700].orig
+++ File[/etc/ferm/conf.d/10_opensearch-inter-node-9700]

+    mode    => 0400
+    require => File[/etc/ferm/conf.d]
+    tag     => ferm
+    notify  => Service[ferm]
+    group   => root
+    owner   => root
+    ensure  => present

Content differences:

--- /etc/ferm/conf.d/10_opensearch-inter-node-9700.orig
+++ /etc/ferm/conf.d/10_opensearch-inter-node-9700
@@ -0,0 +1,8 @@
+# Autogenerated by puppet. DO NOT EDIT BY HAND!
+#
+# 
+&R_SERVICE(tcp, 9700, @resolve((cirrussearch2061.codfw.wmnet cirrussearch2062.codfw.wmnet cirrussearch2063.codfw.wmnet cirrussearch2064.codfw.wmnet cirrussearch2065.codfw.wmnet cirrussearch2066.codfw.wmnet cirrussearch2067.codfw.wmnet cirrussearch2068.codfw.wmnet cirrussearch2069.codfw.wmnet cirrussearch2070.codfw.wmnet cirrussearch2071.codfw.wmnet cirrussearch2072.codfw.wmnet cirrussearch2073.codfw.wmnet cirrussearch2074.codfw.wmnet cirrussearch2075.codfw.wmnet cirrussearch2076.codfw.wmnet cirrussearch2077.codfw.wmnet cirrussearch2078.codfw.wmnet cirrussearch2079.codfw.wmnet cirrussearch2080.codfw.wmnet cirrussearch2081.codfw.wmnet cirrussearch2082.codfw.wmnet cirrussearch2083.codfw.wmnet cirrussearch2084.codfw.wmnet cirrussearch2085.codfw.wmnet cirrussearch2086.codfw.wmnet cirrussearch2087.codfw.wmnet cirrussearch2088.codfw.wmnet cirrussearch2089.codfw.wmnet cirrussearch2090.codfw.wmnet cirrussearch2091.codfw.wmnet cirrussearch2092.codfw.wmnet cirrussearch2093.codfw.wmnet cirrussearch2094.codfw.wmnet cirrussearch2095.codfw.wmnet cirrussearch2096.codfw.wmnet cirrussearch2097.codfw.wmnet cirrussearch2098.codfw.wmnet cirrussearch2099.codfw.wmnet cirrussearch2100.codfw.wmnet cirrussearch2101.codfw.wmnet cirrussearch2102.codfw.wmnet cirrussearch2103.codfw.wmnet cirrussearch2104.codfw.wmnet cirrussearch2105.codfw.wmnet cirrussearch2106.codfw.wmnet cirrussearch2107.codfw.wmnet cirrussearch2108.codfw.wmnet cirrussearch2109.codfw.wmnet cirrussearch2110.codfw.wmnet cirrussearch2111.codfw.wmnet cirrussearch2112.codfw.wmnet cirrussearch2113.codfw.wmnet cirrussearch2114.codfw.wmnet cirrussearch2115.codfw.wmnet)));
+
+
+
+&NO_TRACK(tcp, 9700);

Nrpe::Check[check_opensearch_shards_9600]

Parameters differences:

--- Nrpe::Check[check_opensearch_shards_9600].orig
+++ Nrpe::Check[check_opensearch_shards_9600]

+    command => /usr/lib/nagios/plugins/check_elasticsearch.py --ignore-status --url http://localhost:9600 --shards-inactive '>=0.15' --timeout 4
+    before  => Monitoring::Service[opensearch_shards_9600]
+    ensure  => present

File[/var/log/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]

Parameters differences:

--- File[/var/log/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200].orig
+++ File[/var/log/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]

+    mode   => 0755
+    backup => False
+    force  => True
+    owner  => root
+    ensure => directory
+    group  => root

Logrotate::Conf[opensearch-production-search-psi-codfw-gc-log-cleanup]

Parameters differences:

--- Logrotate::Conf[opensearch-production-search-psi-codfw-gc-log-cleanup].orig
+++ Logrotate::Conf[opensearch-production-search-psi-codfw-gc-log-cleanup]

+    ensure => present

File[/lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9600.timer]

Parameters differences:

--- File[/lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9600.timer].orig
+++ File[/lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9600.timer]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9600.timer (nrpe2nodexp-opensearch_shard_size_check_9600.timer)]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9600.timer.orig
+++ /lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9600.timer
@@ -0,0 +1,13 @@
+[Unit]
+Description=Periodic execution of nrpe2nodexp-opensearch_shard_size_check_9600.service
+
+[Timer]
+Unit=nrpe2nodexp-opensearch_shard_size_check_9600.service
+# Accuracy sets the maximum time interval around the execution time we want to allow
+AccuracySec=15sec
+OnUnitInactiveSec=1440min
+OnActiveSec=1s
+RandomizedDelaySec=0
+
+[Install]
+WantedBy=multi-user.target

Systemd::Service[tcp-mss-clamper]

Parameters differences:

--- Systemd::Service[tcp-mss-clamper].orig
+++ Systemd::Service[tcp-mss-clamper]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    monitoring_notes_url     => https://wikitech.wikimedia.org/wiki/LVS#IPIP_encapsulation_experiments
+    ensure                   => absent
+    monitoring_critical      => False
+    monitoring_enabled       => True
+    unit_type                => service
+    restart                  => False

Nrpe::Check[check_opensearch_shards_9200]

Parameters differences:

--- Nrpe::Check[check_opensearch_shards_9200].orig
+++ Nrpe::Check[check_opensearch_shards_9200]

+    command => /usr/lib/nagios/plugins/check_elasticsearch.py --ignore-status --url http://localhost:9200 --shards-inactive '>=0.15' --timeout 4
+    before  => Monitoring::Service[opensearch_shards_9200]
+    ensure  => present

Systemd::Timer[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]

Parameters differences:

--- Systemd::Timer[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200].orig
+++ Systemd::Timer[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]

+    fixed_random_delay => False
+    unit_name          => wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service
+    timer_intervals    => [{'start': 'OnCalendar', 'interval': 'Mon,Tue,Wed,Thu,Fri *-*-* 18:6:00'}]
+    splay              => 0
+    accuracy           => 15sec
+    ensure             => present

File[/etc/nginx/sites-available]

Parameters differences:

--- File[/etc/nginx/sites-available].orig
+++ File[/etc/nginx/sites-available]

+    purge   => True
+    force   => True
+    before  => ['Service[nginx]']
+    owner   => root
+    recurse => True
+    ensure  => directory
+    group   => root

File[/etc/logstash/log4j2.properties]

Parameters differences:

--- File[/etc/logstash/log4j2.properties].orig
+++ File[/etc/logstash/log4j2.properties]

+    mode    => 0444
+    require => Package[logstash]
+    group   => root
+    owner   => root
+    ensure  => file

Content differences:

--- /etc/logstash/log4j2.properties.orig
+++ /etc/logstash/log4j2.properties
@@ -0,0 +1,51 @@
+status = error
+name = LogstashPropertiesConfig
+
+appender.rolling.type = RollingFile
+appender.rolling.name = plain_rolling
+appender.rolling.fileName = ${sys:ls.logs}/logstash-${sys:ls.log.format}.log
+appender.rolling.filePattern = ${sys:ls.logs}/logstash-${sys:ls.log.format}-%d{yyyy-MM-dd}-%i.log.gz
+appender.rolling.policies.type = Policies
+appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
+appender.rolling.policies.time.interval = 1
+appender.rolling.policies.time.modulate = true
+appender.rolling.layout.type = PatternLayout
+appender.rolling.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %-.10000m%n
+appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
+appender.rolling.policies.size.size = 128M
+appender.rolling.strategy.type = DefaultRolloverStrategy
+appender.rolling.strategy.max = 4
+appender.rolling.strategy.action.type = Delete
+appender.rolling.strategy.action.basepath = ${sys:ls.logs}
+appender.rolling.strategy.action.condition.type = IfFileName
+appender.rolling.strategy.action.condition.glob = logstash-${sys:ls.log.format}-*
+appender.rolling.strategy.action.ifAny.type = IfAny
+appender.rolling.strategy.action.ifAny.ifLastModified.type = IfLastModified
+appender.rolling.strategy.action.ifAny.ifLastModified.age = 30D
+appender.rolling.strategy.action.ifAny.ifAccumulatedFileSize.type = IfAccumulatedFileSize
+appender.rolling.strategy.action.ifAny.ifAccumulatedFileSize.exceeds = 15GB
+
+appender.json_rolling.type = RollingFile
+appender.json_rolling.name = json_rolling
+appender.json_rolling.fileName = ${sys:ls.logs}/logstash-${sys:ls.log.format}.log
+appender.json_rolling.filePattern = ${sys:ls.logs}/logstash-${sys:ls.log.format}-%d{yyyy-MM-dd}.log
+appender.json_rolling.policies.type = Policies
+appender.json_rolling.policies.time.type = TimeBasedTriggeringPolicy
+appender.json_rolling.policies.time.interval = 1
+appender.json_rolling.policies.time.modulate = true
+appender.json_rolling.layout.type = JSONLayout
+appender.json_rolling.layout.compact = true
+appender.json_rolling.layout.eventEol = true
+appender.json_rolling.strategy.type = DefaultRolloverStrategy
+appender.json_rolling.strategy.action.type = Delete
+appender.json_rolling.strategy.action.basepath = ${sys:ls.logs}
+appender.json_rolling.strategy.action.condition.type = IfFileName
+appender.json_rolling.strategy.action.condition.glob = logstash-${sys:ls.log.format}-*
+appender.json_rolling.strategy.action.ifAny.type = IfAny
+appender.json_rolling.strategy.action.ifAny.ifLastModified.type = IfLastModified
+appender.json_rolling.strategy.action.ifAny.ifLastModified.age = 30D
+appender.json_rolling.strategy.action.ifAny.ifAccumulatedFileSize.type = IfAccumulatedFileSize
+appender.json_rolling.strategy.action.ifAny.ifAccumulatedFileSize.exceeds = 10GB
+
+rootLogger.level = ${sys:ls.log.level}
+rootLogger.appenderRef.rolling.ref = ${sys:ls.log.format}_rolling

Package[opensearch]

Parameters differences:

--- Package[opensearch].orig
+++ Package[opensearch]

+    provider => apt
+    ensure   => present

Service[prometheus-wmf-elasticsearch-exporter-9200]

Parameters differences:

--- Service[prometheus-wmf-elasticsearch-exporter-9200].orig
+++ Service[prometheus-wmf-elasticsearch-exporter-9200]

+    enable => True
+    ensure => running

Exec[mask_nginx.service]

Parameters differences:

--- Exec[mask_nginx.service].orig
+++ Exec[mask_nginx.service]

+    creates => /etc/systemd/system/nginx.service
+    unless  => /usr/bin/dpkg -s nginx-light | /bin/grep -q '^Status: install ok installed$'
+    command => /bin/systemctl mask nginx.service

Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_unassigned_shard_check_9200]

Parameters differences:

--- Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_unassigned_shard_check_9200].orig
+++ Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_unassigned_shard_check_9200]

+    passive_checks_enabled => 1
+    is_volatile            => 0
+    contact_groups         => admins,team-discovery
+    notification_options   => c,r,f
+    service_description    => OpenSearch unassigned shard check - 9200
+    max_check_attempts     => 1
+    active_checks_enabled  => 1
+    notification_interval  => 0
+    check_command          => nrpe_check!check_opensearch_unassigned_shard_check_9200!10
+    retry_interval         => 120
+    check_freshness        => 0
+    notes_url              => https://wikitech.wikimedia.org/wiki/Search#Administration
+    notifications_enabled  => 1
+    check_period           => 24x7
+    ensure                 => present
+    check_interval         => 720
+    notification_period    => 24x7
+    host_name              => cirrussearch2089
+    servicegroups          => elasticsearch_codfw

File[/etc/ssh/userkeys/dcausse]

Parameters differences:

--- File[/etc/ssh/userkeys/dcausse].orig
+++ File[/etc/ssh/userkeys/dcausse]

+    mode      => 0444
+    show_diff => False
+    force     => True
+    owner     => root
+    ensure    => file
+    group     => root

Content differences:

--- /etc/ssh/userkeys/dcausse.orig
+++ /etc/ssh/userkeys/dcausse
@@ -0,0 +1 @@
+ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC95t2F3S61Za6Y+mZUs4abK95cXm9FedpnUJe2ii3byeFG5QPL9E8+zcZ70o2Mt0NjM5Nitla/Tb+U2BRjk3/YmWEkZxd9crUFRi3IVHjKhEOYhLSSjb2NJdNyWUoUgkQ8DspVDotsnIupdLuHY0Dyh+O4dFI7xnRKIUShm/B//BDCu03PXR4ikYkmtR/+LeKbMh3qSGgvfgv/G4hIiKjyiQt7Md+ROW0YqMp6yAtqIJ+2in1hU3mxy47vfVOFTcL5nNLOLoxJGVSsm0ancVbzrkqruk1o/iFxuz9AzxnX8EAozaNUH7PP3/ybD7ocK7v4SiYpibT2epWPkORgUV7h dcausse_prod@dizzy

File[/usr/local/bin/pool]

Parameters differences:

--- File[/usr/local/bin/pool].orig
+++ File[/usr/local/bin/pool]

+    mode   => 0555
+    source => puppet:///modules/conftool/conftool-simple-command.sh
+    group  => root
+    owner  => root
+    ensure => present

Tlsproxy::Localssl[production-search-codfw]

Parameters differences:

--- Tlsproxy::Localssl[production-search-codfw].orig
+++ Tlsproxy::Localssl[production-search-codfw]

+    acme_chief         => False
+    server_aliases     => ['search.discovery.wmnet', 'search-chi.discovery.wmnet', 'search-omega.discovery.wmnet', 'search-psi.discovery.wmnet', 'search.svc.codfw.wmnet', 'search-chi.svc.codfw.wmnet', 'search-omega.svc.codfw.wmnet', 'search-psi.svc.codfw.wmnet']
+    upstream_ip        => 10.192.6.18
+    upstream_ports     => [9200]
+    default_server     => True
+    keepalive_timeout  => 60
+    require            => ['Class[Profile::Tlsproxy::Instance]']
+    access_log         => False
+    acme_certname      => production-search-codfw
+    tls_ports          => [9243]
+    enable_http2       => False
+    read_timeout       => 180
+    only_get_requests  => False
+    keepalive_requests => 100
+    server_name        => cirrussearch2089.codfw.wmnet
+    skip_private       => False
+    cfssl_paths        => {'cert': '/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.pem', 'key': '/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet-key.pem', 'chain': '/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chain.pem', 'chained': '/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chained.pem'}

Systemd::Timer::Job[nrpe2nodexp-opensearch_shards_9600]

Parameters differences:

--- Systemd::Timer::Job[nrpe2nodexp-opensearch_shards_9600].orig
+++ Systemd::Timer::Job[nrpe2nodexp-opensearch_shards_9600]

+    description               => execution of nrpe2nodexp for the check_opensearch_shards_9600 command.
+    syslog_match_startswith   => True
+    monitoring_contact_groups => admins
+    logging_enabled           => False
+    success_exit_status       => []
+    fixed_random_delay        => False
+    private_tmp               => False
+    logfile_basedir           => /var/log
+    group                     => prometheus-node-exporter
+    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+    syslog_identifier         => nrpe2nodexp-opensearch_shards_9600
+    send_mail_only_on_error   => True
+    ignore_errors             => True
+    logfile_group             => root
+    interval                  => [{'start': 'OnUnitInactiveSec', 'interval': '1min'}]
+    send_mail                 => False
+    logfile_name              => syslog.log
+    user                      => nagios
+    ensure                    => absent
+    send_mail_to              => root@cirrussearch2089.codfw.wmnet
+    logfile_perms             => all
+    syslog_force_stop         => True
+    environment               => {}
+    command                   => /usr/local/bin/nrpe2nodexp --alert-rule-hash "7811fe11f3315f2bd38a6e6de0f1b6b0" --timeout 10 --check-command "check_opensearch_shards_9600"
+    monitoring_enabled        => False

Interface::Ip[ipip_ipv4 ipv4]

Parameters differences:

--- Interface::Ip[ipip_ipv4 ipv4].orig
+++ Interface::Ip[ipip_ipv4 ipv4]

+    address   => 127.0.0.42
+    require   => Augeas[ipip0_set_up]
+    interface => ipip0
+    prefixlen => 32
+    ensure    => present

Logrotate::Conf[opensearch-disable-readahead]

Parameters differences:

--- Logrotate::Conf[opensearch-disable-readahead].orig
+++ Logrotate::Conf[opensearch-disable-readahead]

+    ensure => absent

Admin::User[dcausse]

Parameters differences:

--- Admin::User[dcausse].orig
+++ Admin::User[dcausse]

+    uid      => 12338
+    groups   => []
+    gid      => 500
+    ensure   => present
+    ssh_keys => ['ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC95t2F3S61Za6Y+mZUs4abK95cXm9FedpnUJe2ii3byeFG5QPL9E8+zcZ70o2Mt0NjM5Nitla/Tb+U2BRjk3/YmWEkZxd9crUFRi3IVHjKhEOYhLSSjb2NJdNyWUoUgkQ8DspVDotsnIupdLuHY0Dyh+O4dFI7xnRKIUShm/B//BDCu03PXR4ikYkmtR/+LeKbMh3qSGgvfgv/G4hIiKjyiQt7Md+ROW0YqMp6yAtqIJ+2in1hU3mxy47vfVOFTcL5nNLOLoxJGVSsm0ancVbzrkqruk1o/iFxuz9AzxnX8EAozaNUH7PP3/ybD7ocK7v4SiYpibT2epWPkORgUV7h dcausse_prod@dizzy']
+    home_dir => /home/dcausse
+    comment  => David Causse
+    shell    => /bin/bash

Systemd::Service[nrpe2nodexp-check_tcp-mss-clamper_status]

Parameters differences:

--- Systemd::Service[nrpe2nodexp-check_tcp-mss-clamper_status].orig
+++ Systemd::Service[nrpe2nodexp-check_tcp-mss-clamper_status]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => absent
+    monitoring_critical      => False
+    require                  => Systemd::Unit[nrpe2nodexp-check_tcp-mss-clamper_status.service]
+    monitoring_enabled       => False
+    unit_type                => timer
+    restart                  => False

Rsyslog::Conf[nrpe2nodexp-check_tcp-mss-clamper_status]

Parameters differences:

--- Rsyslog::Conf[nrpe2nodexp-check_tcp-mss-clamper_status].orig
+++ Rsyslog::Conf[nrpe2nodexp-check_tcp-mss-clamper_status]

+    mode     => 0444
+    priority => 25
+    ensure   => absent

Class[Adduser]

Parameters differences:

--- Class[Adduser].orig
+++ Class[Adduser]

@@
-    before => ['Package[puppet]', 'Package[facter]', 'Package[augeas-tools]', 'Package[virt-what]', 'Package[puppet-module-puppetlabs-augeas-core]', 'Package[python3-prometheus-client]', 'Package[python3-yaml]', 'Package[ruby-net-ssh]', 'Package[openssl]', 'Package[ssl-cert]', 'Package[ca-certificates]', 'Package[wmf-certificates]', 'Package[ntp]', 'Package[systemd-timesyncd]', 'Package[exim4-config]', 'Package[exim4-daemon-light]', 'Package[logrotate]', 'Package[prometheus-node-exporter]', 'Package[bsdutils]', 'Package[smartmontools]', 'Package[rsyslog]', 'Package[rsyslog-openssl]', 'Package[cadvisor]', 'Package[acct]', 'Package[byobu]', 'Package[colordiff]', 'Package[curl]', 'Package[debian-goodies]', 'Package[ethtool]', 'Package[gdb]', 'Package[gdisk]', 'Package[git]', 'Package[htop]', 'Package[httpry]', 'Package[iotop]', 'Package[iperf]', 'Package[jq]', 'Package[libtemplate-perl]', 'Package[lldpd]', 'Package[lshw]', 'Package[molly-guard]', 'Package[moreutils]', 'Package[net-tools]', 'Package[numactl]', 'Package[ncdu]', 'Package[ngrep]', 'Package[pigz]', 'Package[psmisc]', 'Package[pv]', 'Package[python3]', 'Package[screen]', 'Package[strace]', 'Package[sysstat]', 'Package[tcpdump]', 'Package[tmux]', 'Package[tree]', 'Package[vim]', 'Package[vim-addon-manager]', 'Package[vim-scripts]', 'Package[wipe]', 'Package[xfsprogs]', 'Package[zsh]', 'Package[icdiff]', 'Package[linux-perf]', 'Package[bsd-mailx]', 'Package[ack]', 'Package[netcat-openbsd]', 'Package[tshark]', 'Package[fzf]', 'Package[ripgrep]', 'Package[fd-find]', 'Package[kitty-terminfo]', 'Package[mtr-tiny]', 'Package[tzdata]', 'Package[python3-wmflib]', 'Package[quickstack]', 'Package[bind9-dnsutils]', 'Package[dstat]', 'Package[apport]', 'Package[command-not-found]', 'Package[command-not-found-data]', 'Package[ecryptfs-utils]', 'Package[mlocate]', 'Package[os-prober]', 'Package[python3-apport]', 'Package[wpasupplicant]', 'Package[atop]', 'Package[apt-listchanges]', 'Package[isc-dhcp-client]', 'Package[libpython2.7]', 'Package[libpython2.7-dev]', 'Package[libpython2.7-minimal]', 'Package[python2.7]', 'Package[libpython2.7-stdlib]', 'Package[python2.7-dev]', 'Package[python2.7-minimal]', 'Package[python2.7-dbg]', 'Package[python2.7-doc]', 'Package[python2.7-examples]', 'Package[libpython2.7-testsuite]', 'Package[intel-microcode]', 'Package[rasdaemon]', 'Package[libsnmp30]', 'Package[libdns-export1104]', 'Package[libdns1104]', 'Package[libisc-export1100]', 'Package[libisc1100]', 'Package[multiarch-support]', 'Package[libjson-c3]', 'Package[libpython3.7]', 'Package[libpython3.7-minimal]', 'Package[libpython3.7-stdlib]', 'Package[python3.7]', 'Package[python3.7-minimal]', 'Package[libevent-2.1-6]', 'Package[libwireshark11]', 'Package[libwiretap8]', 'Package[libwsutil9]', 'Package[libwscodecs2]', 'Package[libperl5.28]', 'Package[libmpdec2]', 'Package[perl-modules-5.28]', 'Package[libhogweed4]', 'Package[libnettle6]', 'Package[libprocps7]', 'Package[libip6tc0]', 'Package[libip4tc0]', 'Package[libiptc0]', 'Package[openssh-client]', 'Package[openssh-server]', 'Package[debdeploy-client]', 'Package[python3-dateutil]', 'Package[sudo]', 'Package[golang-cfssl]', 'Package[debmonitor-client]', 'Package[perccli]', 'Package[nagios-nrpe-server]', 'Package[monitoring-plugins]', 'Package[monitoring-plugins-basic]', 'Package[monitoring-plugins-standard]', 'Package[liburiparser1]', 'Package[iucode-tool]', 'Package[freeipmi-tools]', 'Package[freeipmi-ipmiseld]', 'Package[rsyslog-kafka]', 'Package[emacs-nox]', 'Package[prometheus-ipmi-exporter]', 'Package[libnet-dns-perl]', 'Package[iptables]', 'Package[ferm]', 'Package[ulogd2]', 'Package[conntrack]', 'Package[ruby-sys-filesystem]', 'Package[prometheus-rsyslog-exporter]', 'Package[python3-click]', 'Package[python3-box]', 'Package[initramfs-tools]', 'Package[confd]', 'Package[python3-toml]']
+    before => ['Package[puppet]', 'Package[facter]', 'Package[augeas-tools]', 'Package[virt-what]', 'Package[puppet-module-puppetlabs-augeas-core]', 'Package[python3-prometheus-client]', 'Package[python3-yaml]', 'Package[ruby-net-ssh]', 'Package[openssl]', 'Package[ssl-cert]', 'Package[ca-certificates]', 'Package[wmf-certificates]', 'Package[ntp]', 'Package[systemd-timesyncd]', 'Package[exim4-config]', 'Package[exim4-daemon-light]', 'Package[logrotate]', 'Package[prometheus-node-exporter]', 'Package[bsdutils]', 'Package[smartmontools]', 'Package[rsyslog]', 'Package[rsyslog-openssl]', 'Package[cadvisor]', 'Package[acct]', 'Package[byobu]', 'Package[colordiff]', 'Package[curl]', 'Package[debian-goodies]', 'Package[ethtool]', 'Package[gdb]', 'Package[gdisk]', 'Package[git]', 'Package[htop]', 'Package[httpry]', 'Package[iotop]', 'Package[iperf]', 'Package[jq]', 'Package[libtemplate-perl]', 'Package[lldpd]', 'Package[lshw]', 'Package[molly-guard]', 'Package[moreutils]', 'Package[net-tools]', 'Package[numactl]', 'Package[ncdu]', 'Package[ngrep]', 'Package[pigz]', 'Package[psmisc]', 'Package[pv]', 'Package[python3]', 'Package[screen]', 'Package[strace]', 'Package[sysstat]', 'Package[tcpdump]', 'Package[tmux]', 'Package[tree]', 'Package[vim]', 'Package[vim-addon-manager]', 'Package[vim-scripts]', 'Package[wipe]', 'Package[xfsprogs]', 'Package[zsh]', 'Package[icdiff]', 'Package[linux-perf]', 'Package[bsd-mailx]', 'Package[ack]', 'Package[netcat-openbsd]', 'Package[tshark]', 'Package[fzf]', 'Package[ripgrep]', 'Package[fd-find]', 'Package[kitty-terminfo]', 'Package[mtr-tiny]', 'Package[tzdata]', 'Package[python3-wmflib]', 'Package[quickstack]', 'Package[bind9-dnsutils]', 'Package[dstat]', 'Package[apport]', 'Package[command-not-found]', 'Package[command-not-found-data]', 'Package[ecryptfs-utils]', 'Package[mlocate]', 'Package[os-prober]', 'Package[python3-apport]', 'Package[wpasupplicant]', 'Package[atop]', 'Package[apt-listchanges]', 'Package[isc-dhcp-client]', 'Package[libpython2.7]', 'Package[libpython2.7-dev]', 'Package[libpython2.7-minimal]', 'Package[python2.7]', 'Package[libpython2.7-stdlib]', 'Package[python2.7-dev]', 'Package[python2.7-minimal]', 'Package[python2.7-dbg]', 'Package[python2.7-doc]', 'Package[python2.7-examples]', 'Package[libpython2.7-testsuite]', 'Package[intel-microcode]', 'Package[rasdaemon]', 'Package[libsnmp30]', 'Package[libdns-export1104]', 'Package[libdns1104]', 'Package[libisc-export1100]', 'Package[libisc1100]', 'Package[multiarch-support]', 'Package[libjson-c3]', 'Package[libpython3.7]', 'Package[libpython3.7-minimal]', 'Package[libpython3.7-stdlib]', 'Package[python3.7]', 'Package[python3.7-minimal]', 'Package[libevent-2.1-6]', 'Package[libwireshark11]', 'Package[libwiretap8]', 'Package[libwsutil9]', 'Package[libwscodecs2]', 'Package[libperl5.28]', 'Package[libmpdec2]', 'Package[perl-modules-5.28]', 'Package[libhogweed4]', 'Package[libnettle6]', 'Package[libprocps7]', 'Package[libip6tc0]', 'Package[libip4tc0]', 'Package[libiptc0]', 'Package[openssh-client]', 'Package[openssh-server]', 'Package[debdeploy-client]', 'Package[python3-dateutil]', 'Package[sudo]', 'Package[golang-cfssl]', 'Package[debmonitor-client]', 'Package[perccli]', 'Package[nagios-nrpe-server]', 'Package[monitoring-plugins]', 'Package[monitoring-plugins-basic]', 'Package[monitoring-plugins-standard]', 'Package[liburiparser1]', 'Package[iucode-tool]', 'Package[freeipmi-tools]', 'Package[freeipmi-ipmiseld]', 'Package[rsyslog-kafka]', 'Package[emacs-nox]', 'Package[prometheus-ipmi-exporter]', 'Package[libnet-dns-perl]', 'Package[iptables]', 'Package[ferm]', 'Package[ulogd2]', 'Package[conntrack]', 'Package[wikimedia-lvs-realserver]', 'Package[python3-conftool]', 'Package[python3-poolcounter]', 'Package[tcp-mss-clamper]', 'Package[edac-utils]', 'Package[opensearch]', 'Package[python3-requests]', 'Package[wmf-opensearch-search-plugins]', 'Package[opensearch-madvise]', 'Package[logstash]', 'Package[logstash-old-name]', 'Package[ruby-sys-filesystem]', 'Package[prometheus-rsyslog-exporter]', 'Package[python3-click]', 'Package[python3-box]', 'Package[initramfs-tools]', 'Package[confd]', 'Package[python3-toml]', 'Package[openjdk-11-jdk]', 'Package[prometheus-elasticsearch-exporter]', 'Package[nginx-light]', 'Package[nginx-common]']

File[/etc/opensearch/production-search-codfw/scripts]

Parameters differences:

--- File[/etc/opensearch/production-search-codfw/scripts].orig
+++ File[/etc/opensearch/production-search-codfw/scripts]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => directory

Ssh::Userkey[pfischer]

Parameters differences:

--- Ssh::Userkey[pfischer].orig
+++ Ssh::Userkey[pfischer]

+    user   => pfischer
+    ensure => present

Rsyslog::Conf[opensearch-disable-readahead]

Parameters differences:

--- Rsyslog::Conf[opensearch-disable-readahead].orig
+++ Rsyslog::Conf[opensearch-disable-readahead]

+    mode     => 0444
+    require  => File[/var/log/opensearch-disable-readahead]
+    priority => 40
+    ensure   => absent

File[/lib/systemd/system/opensearch-disable-readahead-production-search-psi-codfw.timer]

Parameters differences:

--- File[/lib/systemd/system/opensearch-disable-readahead-production-search-psi-codfw.timer].orig
+++ File[/lib/systemd/system/opensearch-disable-readahead-production-search-psi-codfw.timer]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for opensearch-disable-readahead-production-search-psi-codfw.timer (opensearch-disable-readahead-production-search-psi-codfw.timer)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/opensearch-disable-readahead-production-search-psi-codfw.timer.orig
+++ /lib/systemd/system/opensearch-disable-readahead-production-search-psi-codfw.timer
@@ -0,0 +1,13 @@
+[Unit]
+Description=Periodic execution of opensearch-disable-readahead-production-search-psi-codfw.service
+
+[Timer]
+Unit=opensearch-disable-readahead-production-search-psi-codfw.service
+# Accuracy sets the maximum time interval around the execution time we want to allow
+AccuracySec=15sec
+OnUnitActiveSec=30min
+OnBootSec=1min
+RandomizedDelaySec=0
+
+[Install]
+WantedBy=multi-user.target

File[/etc/apt/sources.list.d/wikimedia-opensearch.list]

Parameters differences:

--- File[/etc/apt/sources.list.d/wikimedia-opensearch.list].orig
+++ File[/etc/apt/sources.list.d/wikimedia-opensearch.list]

+    mode   => 0444
+    notify => Exec[apt_repository_wikimedia-opensearch]
+    group  => root
+    owner  => root
+    ensure => file

Content differences:

--- /etc/apt/sources.list.d/wikimedia-opensearch.list.orig
+++ /etc/apt/sources.list.d/wikimedia-opensearch.list
@@ -0,0 +1,2 @@
+deb http://apt.wikimedia.org/wikimedia bullseye-wikimedia thirdparty/opensearch1
+deb-src http://apt.wikimedia.org/wikimedia bullseye-wikimedia thirdparty/opensearch1

Exec[update_java_alternatives_11]

Parameters differences:

--- Exec[update_java_alternatives_11].orig
+++ Exec[update_java_alternatives_11]

+    unless  => /usr/bin/update-alternatives --query java | /bin/grep 'Value: /usr/lib/jvm/java-11-openjdk-amd64'
+    command => /usr/sbin/update-java-alternatives -s /usr/lib/jvm/java-1.11.0-openjdk-amd64

Class[Opensearch::Packages]

Parameters differences:

--- Class[Opensearch::Packages].orig
+++ Class[Opensearch::Packages]

+    package_name          => opensearch
+    send_logs_to_logstash => True

File[/lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service]

Parameters differences:

--- File[/lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service].orig
+++ File[/lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service (wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service.orig
+++ /lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service
@@ -0,0 +1,8 @@
+[Unit]
+Description=Auto restart job: prometheus-elasticsearch-exporter-9600
+Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+
+[Service]
+Type=oneshot
+User=root
+ExecStart=/usr/local/sbin/wmf-auto-restart -s prometheus-elasticsearch-exporter-9600

File[/lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer]

Parameters differences:

--- File[/lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer].orig
+++ File[/lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer (wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer.orig
+++ /lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer
@@ -0,0 +1,12 @@
+[Unit]
+Description=Periodic execution of wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service
+
+[Timer]
+Unit=wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service
+# Accuracy sets the maximum time interval around the execution time we want to allow
+AccuracySec=15sec
+OnCalendar=Mon,Tue,Wed,Thu,Fri *-*-* 2:54:00
+RandomizedDelaySec=0
+
+[Install]
+WantedBy=multi-user.target

Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_shards_9200]

Parameters differences:

--- Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_shards_9200].orig
+++ Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_shards_9200]

+    passive_checks_enabled => 1
+    is_volatile            => 0
+    contact_groups         => admins,team-discovery
+    notification_options   => c,r,f
+    service_description    => OpenSearch health check for shards on 9200
+    max_check_attempts     => 3
+    active_checks_enabled  => 1
+    notification_interval  => 0
+    check_command          => nrpe_check!check_opensearch_shards_9200!10
+    retry_interval         => 1
+    check_freshness        => 0
+    notes_url              => https://wikitech.wikimedia.org/wiki/Search#Administration
+    notifications_enabled  => 1
+    check_period           => 24x7
+    ensure                 => present
+    check_interval         => 1
+    notification_period    => 24x7
+    host_name              => cirrussearch2089
+    servicegroups          => elasticsearch_codfw

File[/etc/ssh/userkeys/dr0ptp4kt]

Parameters differences:

--- File[/etc/ssh/userkeys/dr0ptp4kt].orig
+++ File[/etc/ssh/userkeys/dr0ptp4kt]

+    mode      => 0444
+    show_diff => False
+    force     => True
+    owner     => root
+    ensure    => file
+    group     => root

Content differences:

--- /etc/ssh/userkeys/dr0ptp4kt.orig
+++ /etc/ssh/userkeys/dr0ptp4kt
@@ -0,0 +1 @@
+ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoKHi5isY9FixH31qz/81V7fOHsorLZI/NLKr9Z6Xawl2a2Ih0ZV/pJtD+BTu1ufK2QOdgobeRSrnybzf2/1aCqi3Z9H2XxJhMCfnLb/9AIcKJ9tN63T4nRnjLoPsmRgDQrOSIqY5NfLKzXBsQOqc3chZ5SaDf8f09OdBk+Obn5vhr6yWh4GhrfTzoZUfp6+JRiueZZYuGMIKdBAH82s9TyuhuGWvHJmO9WC1MJOV/3hIcim+X0xR+BNLEU/Uj4OPEXC0/EiXh2CJDLugBpLU28RF+Y16TRj/WmO2H0H6qVdmkiK7Ez9PCbsy4RFPq4hdART9QiQbQJzZzaYSAkSFV abaso@wikimedia.org

File[/etc/logrotate.d/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]

Parameters differences:

--- File[/etc/logrotate.d/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200].orig
+++ File[/etc/logrotate.d/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/logrotate.d/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.orig
+++ /etc/logrotate.d/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200
@@ -0,0 +1,12 @@
+# logrotate(8) config for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200
+
+/var/log/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200/*.log {
+    daily
+    copytruncate
+    missingok
+    compress
+    delaycompress
+    notifempty
+    rotate 15
+    size 256M
+}

File_line[auto_restart_file_presence_prometheus-wmf-elasticsearch-exporter-9600]

Parameters differences:

--- File_line[auto_restart_file_presence_prometheus-wmf-elasticsearch-exporter-9600].orig
+++ File_line[auto_restart_file_presence_prometheus-wmf-elasticsearch-exporter-9600]

+    require => File[/etc/debdeploy-client/autorestarts.conf]
+    path    => /etc/debdeploy-client/autorestarts.conf
+    ensure  => present
+    line    => prometheus-wmf-elasticsearch-exporter-9600

Profile::Auto_restarts::Service[prometheus-elasticsearch-exporter-9600]

Parameters differences:

--- Profile::Auto_restarts::Service[prometheus-elasticsearch-exporter-9600].orig
+++ Profile::Auto_restarts::Service[prometheus-elasticsearch-exporter-9600]

+    ensure => present

File[/etc/nagios/nrpe.d/check_opensearch_shard_size_check_9600.cfg]

Parameters differences:

--- File[/etc/nagios/nrpe.d/check_opensearch_shard_size_check_9600.cfg].orig
+++ File[/etc/nagios/nrpe.d/check_opensearch_shard_size_check_9600.cfg]

+    mode    => 0444
+    require => Package[nagios-nrpe-server]
+    tag     => nrpe::check
+    notify  => Service[nagios-nrpe-server]
+    group   => root
+    owner   => root
+    ensure  => present

Content differences:

--- /etc/nagios/nrpe.d/check_opensearch_shard_size_check_9600.cfg.orig
+++ /etc/nagios/nrpe.d/check_opensearch_shard_size_check_9600.cfg
@@ -0,0 +1,2 @@
+# File generated by puppet. DO NOT edit by hand
+command[check_opensearch_shard_size_check_9600]=/usr/lib/nagios/plugins/check_elasticsearch_shard_size.py --url http://localhost:9600 --shard-size-warning 110 --shard-size-critical 140 --timeout 4

Monitoring::Exported_nagios_service[cirrussearch2089 elasticsearch-https-production-search-codfw]

Parameters differences:

--- Monitoring::Exported_nagios_service[cirrussearch2089 elasticsearch-https-production-search-codfw].orig
+++ Monitoring::Exported_nagios_service[cirrussearch2089 elasticsearch-https-production-search-codfw]

+    passive_checks_enabled => 1
+    is_volatile            => 0
+    contact_groups         => team-data-platform,team-discovery
+    notification_options   => c,r,f
+    service_description    => Elasticsearch HTTPS for production-search-codfw
+    max_check_attempts     => 3
+    active_checks_enabled  => 1
+    notification_interval  => 0
+    check_command          => check_ssl_on_port!cirrussearch2089.codfw.wmnet!9243
+    retry_interval         => 1
+    check_freshness        => 0
+    notes_url              => https://wikitech.wikimedia.org/wiki/Search
+    notifications_enabled  => 1
+    check_period           => 24x7
+    ensure                 => present
+    check_interval         => 1
+    notification_period    => 24x7
+    host_name              => cirrussearch2089
+    servicegroups          => elasticsearch_codfw

Sysctl::Conffile[ubuntu defaults]

File[/usr/local/bin/drain]

Parameters differences:

--- File[/usr/local/bin/drain].orig
+++ File[/usr/local/bin/drain]

+    mode   => 0555
+    source => puppet:///modules/conftool/conftool-simple-command.sh
+    group  => root
+    owner  => root
+    ensure => present

Class[Profile::Opensearch::Cirrus::Server]

Parameters differences:

--- Class[Profile::Opensearch::Cirrus::Server].orig
+++ Class[Profile::Opensearch::Cirrus::Server]

+    expose_http          => True
+    certificate_domains  => ['search.discovery.wmnet', 'search-chi.discovery.wmnet', 'search-omega.discovery.wmnet', 'search-psi.discovery.wmnet', 'search.svc.codfw.wmnet', 'search-chi.svc.codfw.wmnet', 'search-omega.svc.codfw.wmnet', 'search-psi.svc.codfw.wmnet']
+    ssl_provider         => cfssl
+    cluster              => elasticsearch
+    ferm_ro_srange       => 
+    base_data_dir        => /srv/opensearch
+    enable_remote_search => True
+    ferm_srange          => $DOMAIN_NETWORKS
+    storage_device       => md1

Ssh::Userkey[dcausse]

Parameters differences:

--- Ssh::Userkey[dcausse].orig
+++ Ssh::Userkey[dcausse]

+    user   => dcausse
+    ensure => present

Interface::Clsact[clsact_eno12399np0]

Parameters differences:

--- Interface::Clsact[clsact_eno12399np0].orig
+++ Interface::Clsact[clsact_eno12399np0]

+    interface => eno12399np0
+    ensure    => absent

Package[prometheus-elasticsearch-exporter]

Parameters differences:

--- Package[prometheus-elasticsearch-exporter].orig
+++ Package[prometheus-elasticsearch-exporter]

+    provider => apt
+    ensure   => installed

Service[opensearch]

Parameters differences:

--- Service[opensearch].orig
+++ Service[opensearch]

+    require => Package[opensearch]
+    enable  => False
+    ensure  => stopped

Monitoring::Service[opensearch_shard_size_check_9200]

Parameters differences:

--- Monitoring::Service[opensearch_shard_size_check_9200].orig
+++ Monitoring::Service[opensearch_shard_size_check_9200]

+    check_command  => nrpe_check!check_opensearch_shard_size_check_9200!10
+    retry_interval => 180
+    description    => OpenSearch shard size check - 9200
+    critical       => False
+    notes_url      => https://wikitech.wikimedia.org/wiki/Search#If_it_has_been_indexed
+    retries        => 3
+    config_dir     => /etc/nagios
+    freshness      => 36000
+    migration_task => T321808
+    ensure         => present
+    check_interval => 1440
+    contact_group  => admins,team-discovery
+    host           => cirrussearch2089
+    passive        => False

Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9200.timer (nrpe2nodexp-opensearch_shard_size_check_9200.timer)]

Parameters differences:

--- Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9200.timer (nrpe2nodexp-opensearch_shard_size_check_9200.timer)].orig
+++ Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9200.timer (nrpe2nodexp-opensearch_shard_size_check_9200.timer)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

File[/etc/default/prometheus-elasticsearch-exporter]

Parameters differences:

--- File[/etc/default/prometheus-elasticsearch-exporter].orig
+++ File[/etc/default/prometheus-elasticsearch-exporter]

+    group  => root
+    owner  => root
+    ensure => absent

File[/etc/tmpfiles.d/opensearch-production-search-psi-codfw.conf]

Parameters differences:

--- File[/etc/tmpfiles.d/opensearch-production-search-psi-codfw.conf].orig
+++ File[/etc/tmpfiles.d/opensearch-production-search-psi-codfw.conf]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/tmpfiles.d/opensearch-production-search-psi-codfw.conf.orig
+++ /etc/tmpfiles.d/opensearch-production-search-psi-codfw.conf
@@ -0,0 +1 @@
+d    /run/opensearch-production-search-psi-codfw  0755 opensearch opensearch - -

Motd::Script[cluster_memberships]

Parameters differences:

--- Motd::Script[cluster_memberships].orig
+++ Motd::Script[cluster_memberships]

+    priority => 96
+    ensure   => present
+    source   => puppet:///modules/opensearch/opensearch.motd

File[/etc/opensearch/production-search-psi-codfw]

Parameters differences:

--- File[/etc/opensearch/production-search-psi-codfw].orig
+++ File[/etc/opensearch/production-search-psi-codfw]

+    mode   => 0755
+    group  => root
+    owner  => root
+    ensure => directory

File[/etc/opensearch/production-search-psi-codfw/logging.yml]

Parameters differences:

--- File[/etc/opensearch/production-search-psi-codfw/logging.yml].orig
+++ File[/etc/opensearch/production-search-psi-codfw/logging.yml]

+    group  => root
+    owner  => root
+    ensure => absent

Class[Nginx]

Parameters differences:

--- Class[Nginx].orig
+++ Class[Nginx]

+    variant      => light
+    lib_on_tmpfs => True
+    tmpfs_size   => 1g
+    managed      => False
+    modules      => []
+    ensure       => present

File[/lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service]

Parameters differences:

--- File[/lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service].orig
+++ File[/lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service (wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service.orig
+++ /lib/systemd/system/wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service
@@ -0,0 +1,8 @@
+[Unit]
+Description=Auto restart job: prometheus-elasticsearch-exporter-9200
+Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+
+[Service]
+Type=oneshot
+User=root
+ExecStart=/usr/local/sbin/wmf-auto-restart -s prometheus-elasticsearch-exporter-9200

Logrotate::Conf[nginx]

Parameters differences:

--- Logrotate::Conf[nginx].orig
+++ Logrotate::Conf[nginx]

+    tag    => nginx
+    ensure => present
+    source => puppet:///modules/profile/tlsproxy/logrotate

Service[opensearch-production-search-codfw-gc-log-cleanup.timer]

Parameters differences:

--- Service[opensearch-production-search-codfw-gc-log-cleanup.timer].orig
+++ Service[opensearch-production-search-codfw-gc-log-cleanup.timer]

+    provider => systemd
+    enable   => True
+    ensure   => running

File[/usr/local/bin/prometheus-wmf-elasticsearch-exporter]

Parameters differences:

--- File[/usr/local/bin/prometheus-wmf-elasticsearch-exporter].orig
+++ File[/usr/local/bin/prometheus-wmf-elasticsearch-exporter]

+    mode   => 0555
+    source => puppet:///modules/prometheus/usr/local/bin/prometheus-wmf-elasticsearch-exporter.py
+    group  => root
+    owner  => root
+    ensure => present

File[/etc/systemd/system/nginx.service.d/security.conf]

Parameters differences:

--- File[/etc/systemd/system/nginx.service.d/security.conf].orig
+++ File[/etc/systemd/system/nginx.service.d/security.conf]

+    mode    => 0444
+    require => File[/etc/systemd/system/nginx.service.d]
+    source  => puppet:///modules/profile/tlsproxy/nginx-security.conf
+    group   => root
+    before  => Class[Nginx]
+    owner   => root
+    ensure  => present

Exec[create chained cert /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chain.pem]

Parameters differences:

--- Exec[create chained cert /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chain.pem].orig
+++ Exec[create chained cert /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chain.pem]

+    unless    => /usr/bin/test "$(/bin/cat /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.pem /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chain.pem | sha512sum)" == "$(/bin/cat /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chained.pem | sha512sum)"

+    command   => /bin/cat /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.pem /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chain.pem > /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chained.pem
+    subscribe => ['Exec[renew certificate - discovery__cirrussearch2089_codfw_wmnet]', 'File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chain.pem]', 'File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.pem]']
+    notify    => ['Exec[nginx-reload]', 'Exec[nginx-reload]']

File[/lib/systemd/system/nrpe2nodexp-opensearch_shards_9600.service]

Parameters differences:

--- File[/lib/systemd/system/nrpe2nodexp-opensearch_shards_9600.service].orig
+++ File[/lib/systemd/system/nrpe2nodexp-opensearch_shards_9600.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9600.service (nrpe2nodexp-opensearch_shards_9600.service)]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /lib/systemd/system/nrpe2nodexp-opensearch_shards_9600.service.orig
+++ /lib/systemd/system/nrpe2nodexp-opensearch_shards_9600.service
@@ -0,0 +1,11 @@
+[Unit]
+Description=execution of nrpe2nodexp for the check_opensearch_shards_9600 command.
+Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+
+[Service]
+Type=oneshot
+User=nagios
+
+Group=prometheus-node-exporter
+SyslogIdentifier=nrpe2nodexp-opensearch_shards_9600
+ExecStart=-/usr/local/bin/nrpe2nodexp --alert-rule-hash "7811fe11f3315f2bd38a6e6de0f1b6b0" --timeout 10 --check-command "check_opensearch_shards_9600"

File[/etc/logrotate.d/clean_up_dlq]

Parameters differences:

--- File[/etc/logrotate.d/clean_up_dlq].orig
+++ File[/etc/logrotate.d/clean_up_dlq]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /etc/logrotate.d/clean_up_dlq.orig
+++ /etc/logrotate.d/clean_up_dlq
@@ -0,0 +1,12 @@
+# logrotate(8) config for clean_up_dlq
+
+/var/log/clean_up_dlq/*.log {
+    daily
+    copytruncate
+    missingok
+    compress
+    delaycompress
+    notifempty
+    rotate 15
+    size 256M
+}

File[/etc/opensearch_hot_threads.d]

Parameters differences:

--- File[/etc/opensearch_hot_threads.d].orig
+++ File[/etc/opensearch_hot_threads.d]

+    mode   => 0755
+    group  => root
+    owner  => root
+    ensure => directory

File[/usr/local/bin/depool-opensearch_1@production-search-codfw]

Parameters differences:

--- File[/usr/local/bin/depool-opensearch_1@production-search-codfw].orig
+++ File[/usr/local/bin/depool-opensearch_1@production-search-codfw]

+    mode   => 0555
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /usr/local/bin/depool-opensearch_1@production-search-codfw.orig
+++ /usr/local/bin/depool-opensearch_1@production-search-codfw
@@ -0,0 +1,2 @@
+#!/bin/bash
+/usr/local/bin/safe-service-restart --pools search search-https --depool --retries 10 --wait 5

Systemd::Unit[opensearch-production-search-codfw-gc-log-cleanup.timer]

Parameters differences:

--- Systemd::Unit[opensearch-production-search-codfw-gc-log-cleanup.timer].orig
+++ Systemd::Unit[opensearch-production-search-codfw-gc-log-cleanup.timer]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => opensearch-production-search-codfw-gc-log-cleanup.timer
+    ensure            => present

Exec[systemd daemon-reload for clean_up_dlq.service (clean_up_dlq.service)]

Parameters differences:

--- Exec[systemd daemon-reload for clean_up_dlq.service (clean_up_dlq.service)].orig
+++ Exec[systemd daemon-reload for clean_up_dlq.service (clean_up_dlq.service)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

Systemd::Unit[nrpe2nodexp-check_tcp-mss-clamper_status.timer]

Parameters differences:

--- Systemd::Unit[nrpe2nodexp-check_tcp-mss-clamper_status.timer].orig
+++ Systemd::Unit[nrpe2nodexp-check_tcp-mss-clamper_status.timer]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => nrpe2nodexp-check_tcp-mss-clamper_status.timer
+    ensure            => absent

Ferm::Service[opensearch-https-9643]

Parameters differences:

--- Ferm::Service[opensearch-https-9643].orig
+++ Ferm::Service[opensearch-https-9643]

+    proto   => tcp
+    port    => 9643
+    prio    => 10
+    desc    => 
+    notrack => False
+    srange  => $DOMAIN_NETWORKS
+    ensure  => present

Systemd::Timer::Job[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]

Parameters differences:

--- Systemd::Timer::Job[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200].orig
+++ Systemd::Timer::Job[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200]

+    description               => Auto restart job: prometheus-wmf-elasticsearch-exporter-9200
+    syslog_match_startswith   => True
+    monitoring_contact_groups => admins
+    logging_enabled           => True
+    success_exit_status       => []
+    require                   => File[/usr/local/sbin/wmf-auto-restart]
+    fixed_random_delay        => False
+    private_tmp               => False
+    logfile_basedir           => /var/log
+    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+    send_mail_only_on_error   => True
+    ignore_errors             => False
+    logfile_group             => root
+    interval                  => {'start': 'OnCalendar', 'interval': 'Mon,Tue,Wed,Thu,Fri *-*-* 18:6:00'}
+    send_mail                 => False
+    logfile_name              => syslog.log
+    user                      => root
+    ensure                    => present
+    send_mail_to              => root@cirrussearch2089.codfw.wmnet
+    logfile_perms             => all
+    syslog_force_stop         => True
+    environment               => {}
+    command                   => /usr/local/sbin/wmf-auto-restart -s prometheus-wmf-elasticsearch-exporter-9200
+    monitoring_enabled        => False

Service[nrpe2nodexp-opensearch_shards_9600.timer]

Parameters differences:

--- Service[nrpe2nodexp-opensearch_shards_9600.timer].orig
+++ Service[nrpe2nodexp-opensearch_shards_9600.timer]

+    before   => ['Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9600.timer (nrpe2nodexp-opensearch_shards_9600.timer)]']
+    provider => systemd
+    enable   => False
+    ensure   => stopped

Systemd::Unit[nrpe2nodexp-opensearch_shards_9200.timer]

Parameters differences:

--- Systemd::Unit[nrpe2nodexp-opensearch_shards_9200.timer].orig
+++ Systemd::Unit[nrpe2nodexp-opensearch_shards_9200.timer]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => nrpe2nodexp-opensearch_shards_9200.timer
+    ensure            => absent

Rsyslog::Conf[udp_json_logback_compat]

Parameters differences:

--- Rsyslog::Conf[udp_json_logback_compat].orig
+++ Rsyslog::Conf[udp_json_logback_compat]

+    mode     => 0444
+    priority => 50
+    ensure   => present

Cfssl::Cert[discovery__cirrussearch2089_codfw_wmnet]

Parameters differences:

--- Cfssl::Cert[discovery__cirrussearch2089_codfw_wmnet].orig
+++ Cfssl::Cert[discovery__cirrussearch2089_codfw_wmnet]

+    auto_renew      => True
+    common_name     => cirrussearch2089.codfw.wmnet
+    names           => []
+    provide_chain   => True
+    label           => discovery
+    hosts           => ['search.discovery.wmnet', 'search-chi.discovery.wmnet', 'search-omega.discovery.wmnet', 'search-psi.discovery.wmnet', 'search.svc.codfw.wmnet', 'search-chi.svc.codfw.wmnet', 'search-omega.svc.codfw.wmnet', 'search-psi.svc.codfw.wmnet']
+    ensure          => present
+    key             => {'algo': 'ecdsa', 'size': 256}
+    mode            => 0740
+    renew_seconds   => 952200
+    notify_services => []
+    environment     => ['GODEBUG=x509ignoreCN=0']
+    group           => root
+    owner           => root
+    before_services => []

File[/home/dr0ptp4kt]

Parameters differences:

--- File[/home/dr0ptp4kt].orig
+++ File[/home/dr0ptp4kt]

+    mode    => 0644
+    source  => puppet:///modules/admin/home/skel
+    force   => True
+    owner   => dr0ptp4kt
+    recurse => remote
+    ensure  => directory
+    group   => 500

Nrpe::Monitor_service[opensearch_shards_9600]

Parameters differences:

--- Nrpe::Monitor_service[opensearch_shards_9600].orig
+++ Nrpe::Monitor_service[opensearch_shards_9600]

+    timeout                     => 10
+    nrpe2nodexp_parse_perf_data => False
+    retry_interval              => 1
+    description                 => OpenSearch health check for shards on 9600
+    notes_url                   => https://wikitech.wikimedia.org/wiki/Search#Administration
+    critical                    => False
+    migration_task              => T371083
+    retries                     => 3
+    ensure                      => present
+    enable_nrpe2nodexp          => False
+    enable_icinga_check         => True
+    check_interval              => 1
+    contact_group               => admins,team-discovery
+    nrpe_command                => /usr/lib/nagios/plugins/check_elasticsearch.py --ignore-status --url http://localhost:9600 --shards-inactive '>=0.15' --timeout 4

Logrotate::Conf[opensearch-disable-readahead-production-search-codfw]

Parameters differences:

--- Logrotate::Conf[opensearch-disable-readahead-production-search-codfw].orig
+++ Logrotate::Conf[opensearch-disable-readahead-production-search-codfw]

+    ensure => present

Exec[opensearch-create-keystore-production-search-psi-codfw]

Parameters differences:

--- Exec[opensearch-create-keystore-production-search-psi-codfw].orig
+++ Exec[opensearch-create-keystore-production-search-psi-codfw]

+    environment => ['OPENSEARCH_PATH_CONF=/etc/opensearch/production-search-psi-codfw']
+    creates     => /etc/opensearch/production-search-psi-codfw/opensearch.keystore
+    command     => /usr/share/opensearch/bin/opensearch-keystore create
+    before      => File[/etc/opensearch/production-search-psi-codfw/opensearch.keystore]

Augeas[ipip0_127.0.0.42/32]

Parameters differences:

--- Augeas[ipip0_127.0.0.42/32].orig
+++ Augeas[ipip0_127.0.0.42/32]

+    onlyif  => match up[. = 'ip addr add 127.0.0.42/32 dev ipip0'] size == 0
+    context => /files/etc/network/interfaces/*[. = 'ipip0' and ./family = 'inet']
+    incl    => /etc/network/interfaces
+    lens    => Interfaces.lns
+    changes => set up[last()+1] 'ip addr add 127.0.0.42/32 dev ipip0'

File[/home/ebernhardson]

Parameters differences:

--- File[/home/ebernhardson].orig
+++ File[/home/ebernhardson]

+    mode    => 0644
+    source  => puppet:///modules/admin/home/ebernhardson
+    force   => True
+    owner   => ebernhardson
+    recurse => remote
+    ensure  => directory
+    group   => 500

Concat_fragment[main contacts]

Content differences:

--- main contacts.orig
+++ main contacts
@@ -1,3 +1,3 @@
 ---
-role::insetup::data_platform_ferm:
-- Data Platform
+role::cirrus::opensearch:
+- Search Platform

File[/etc/ferm/conf.d/10_clamp-mss-ipv4]

Parameters differences:

--- File[/etc/ferm/conf.d/10_clamp-mss-ipv4].orig
+++ File[/etc/ferm/conf.d/10_clamp-mss-ipv4]

+    mode    => 0400
+    require => File[/etc/ferm/conf.d]
+    tag     => ferm
+    notify  => Service[ferm]
+    group   => root
+    owner   => root
+    ensure  => present

Content differences:

--- /etc/ferm/conf.d/10_clamp-mss-ipv4.orig
+++ /etc/ferm/conf.d/10_clamp-mss-ipv4
@@ -0,0 +1,11 @@
+# Autogenerated by puppet. DO NOT EDIT BY HAND!
+#
+# 10_clamp-mss-ipv4: 
+
+domain (ip) {
+	table filter {
+		chain OUTPUT {
+			outerface (eno12399np0 lo) saddr @ipfilter(10.2.1.30) proto tcp sport (9200 9243 9643) tcp-flags (SYN) SYN TCPMSS set-mss 1440;
+		}
+	}
+}

Systemd::Timer::Job[opensearch-production-search-psi-codfw-gc-log-cleanup]

Parameters differences:

--- Systemd::Timer::Job[opensearch-production-search-psi-codfw-gc-log-cleanup].orig
+++ Systemd::Timer::Job[opensearch-production-search-psi-codfw-gc-log-cleanup]

+    description               => Cleanup GC logs
+    syslog_match_startswith   => True
+    monitoring_contact_groups => admins
+    logging_enabled           => True
+    success_exit_status       => []
+    fixed_random_delay        => False
+    private_tmp               => False
+    logfile_basedir           => /var/log
+    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+    send_mail_only_on_error   => True
+    ignore_errors             => False
+    logfile_group             => root
+    interval                  => {'start': 'OnCalendar', 'interval': '*-*-* 02:12:00'}
+    send_mail                 => False
+    logfile_name              => syslog.log
+    user                      => root
+    ensure                    => present
+    send_mail_to              => root@cirrussearch2089.codfw.wmnet
+    logfile_perms             => all
+    syslog_force_stop         => True
+    environment               => {}
+    command                   => /usr/bin/find /var/log/opensearch -name 'production-search-psi-codfw_jvm_gc.*.log*' -mtime +30 -delete
+    monitoring_enabled        => False

File[/lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9600.service]

Parameters differences:

--- File[/lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9600.service].orig
+++ File[/lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9600.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9600.service (nrpe2nodexp-opensearch_shard_size_check_9600.service)]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9600.service.orig
+++ /lib/systemd/system/nrpe2nodexp-opensearch_shard_size_check_9600.service
@@ -0,0 +1,11 @@
+[Unit]
+Description=execution of nrpe2nodexp for the check_opensearch_shard_size_check_9600 command.
+Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+
+[Service]
+Type=oneshot
+User=nagios
+
+Group=prometheus-node-exporter
+SyslogIdentifier=nrpe2nodexp-opensearch_shard_size_check_9600
+ExecStart=-/usr/local/bin/nrpe2nodexp --alert-rule-hash "6ce4fe694f5df8cba9e0032196c72747" --timeout 10 --check-command "check_opensearch_shard_size_check_9600"

Sysctl::Parameters[ubuntu defaults]

Parameters differences:

--- Sysctl::Parameters[ubuntu defaults].orig
+++ Sysctl::Parameters[ubuntu defaults]

@@
-    values => {'kernel.printk': [4, 4, 1, 7], 'kernel.kptr_restrict': 1, 'net.ipv4.conf.default.rp_filter': '1', 'net.ipv4.conf.all.rp_filter': '1', 'net.ipv4.tcp_syncookies': 1, 'kernel.yama.ptrace_scope': 1, 'fs.protected_hardlinks': 1, 'fs.protected_symlinks': 1, 'vm.mmap_min_addr': 65536}
+    values => {'kernel.printk': [4, 4, 1, 7], 'kernel.kptr_restrict': 1, 'net.ipv4.conf.default.rp_filter': '0', 'net.ipv4.conf.all.rp_filter': '0', 'net.ipv4.tcp_syncookies': 1, 'kernel.yama.ptrace_scope': 1, 'fs.protected_hardlinks': 1, 'fs.protected_symlinks': 1, 'vm.mmap_min_addr': 65536}

File[/usr/local/bin/decommission]

Parameters differences:

--- File[/usr/local/bin/decommission].orig
+++ File[/usr/local/bin/decommission]

+    mode   => 0555
+    source => puppet:///modules/conftool/conftool-simple-command.sh
+    group  => root
+    owner  => root
+    ensure => present

Prometheus::Elasticsearch_exporter[localhost:9200]

Parameters differences:

--- Prometheus::Elasticsearch_exporter[localhost:9200].orig
+++ Prometheus::Elasticsearch_exporter[localhost:9200]

+    extra_config       => 
+    prometheus_port    => 9108
+    elasticsearch_port => 9200

Ferm::Rule[clamp-mss-ipv6]

Parameters differences:

--- Ferm::Rule[clamp-mss-ipv6].orig
+++ Ferm::Rule[clamp-mss-ipv6]

+    prio   => 10
+    desc   => 
+    table  => filter
+    rule   => outerface (eno12399np0 lo) saddr @ipfilter(10.2.1.30) proto tcp sport (9200 9243 9643) tcp-flags (SYN) SYN TCPMSS set-mss 1400;
+    chain  => OUTPUT
+    ensure => present
+    domain => (ip6)

File[/lib/systemd/system/opensearch-disable-readahead-production-search-codfw.timer]

Parameters differences:

--- File[/lib/systemd/system/opensearch-disable-readahead-production-search-codfw.timer].orig
+++ File[/lib/systemd/system/opensearch-disable-readahead-production-search-codfw.timer]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for opensearch-disable-readahead-production-search-codfw.timer (opensearch-disable-readahead-production-search-codfw.timer)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/opensearch-disable-readahead-production-search-codfw.timer.orig
+++ /lib/systemd/system/opensearch-disable-readahead-production-search-codfw.timer
@@ -0,0 +1,13 @@
+[Unit]
+Description=Periodic execution of opensearch-disable-readahead-production-search-codfw.service
+
+[Timer]
+Unit=opensearch-disable-readahead-production-search-codfw.service
+# Accuracy sets the maximum time interval around the execution time we want to allow
+AccuracySec=15sec
+OnUnitActiveSec=30min
+OnBootSec=1min
+RandomizedDelaySec=0
+
+[Install]
+WantedBy=multi-user.target

File[/etc/logstash/conf.d/90-output-udp-gelf_relay.conf]

Parameters differences:

--- File[/etc/logstash/conf.d/90-output-udp-gelf_relay.conf].orig
+++ File[/etc/logstash/conf.d/90-output-udp-gelf_relay.conf]

+    mode         => 0440
+    validate_cmd => /usr/local/bin/logstash-config-test %
+    notify       => Service[logstash]
+    group        => logstash
+    owner        => logstash
+    ensure       => present

Content differences:

--- /etc/logstash/conf.d/90-output-udp-gelf_relay.conf.orig
+++ /etc/logstash/conf.d/90-output-udp-gelf_relay.conf
@@ -0,0 +1,9 @@
+output {
+
+      udp {
+          host => 'localhost'
+          port => '11514'
+          codec =>'json'
+      }
+
+}

Systemd::Timer[opensearch-disable-readahead-production-search-psi-codfw]

Parameters differences:

--- Systemd::Timer[opensearch-disable-readahead-production-search-psi-codfw].orig
+++ Systemd::Timer[opensearch-disable-readahead-production-search-psi-codfw]

+    fixed_random_delay => False
+    unit_name          => opensearch-disable-readahead-production-search-psi-codfw.service
+    timer_intervals    => [{'start': 'OnUnitActiveSec', 'interval': '30min'}, {'start': 'OnBootSec', 'interval': '1min'}]
+    splay              => 0
+    accuracy           => 15sec
+    ensure             => present

Exec[java__cacert_wmf:puppetca.pem]

Parameters differences:

--- Exec[java__cacert_wmf:puppetca.pem].orig
+++ Exec[java__cacert_wmf:puppetca.pem]

+    unless  => /usr/bin/keytool -list -cacerts -noprompt -storepass changeit -alias wmf:puppetca.pem
+    command => /usr/bin/keytool -import -trustcacerts -noprompt -cacerts     -file /usr/share/ca-certificates/wikimedia/Puppet5_Internal_CA.crt -storepass changeit -alias wmf:puppetca.pem

+    user    => root
+    group   => root

File[/etc/nagios/nrpe.d/check_opensearch_unassigned_shard_check_9200.cfg]

Parameters differences:

--- File[/etc/nagios/nrpe.d/check_opensearch_unassigned_shard_check_9200.cfg].orig
+++ File[/etc/nagios/nrpe.d/check_opensearch_unassigned_shard_check_9200.cfg]

+    mode    => 0444
+    require => Package[nagios-nrpe-server]
+    tag     => nrpe::check
+    notify  => Service[nagios-nrpe-server]
+    group   => root
+    owner   => root
+    ensure  => present

Content differences:

--- /etc/nagios/nrpe.d/check_opensearch_unassigned_shard_check_9200.cfg.orig
+++ /etc/nagios/nrpe.d/check_opensearch_unassigned_shard_check_9200.cfg
@@ -0,0 +1,2 @@
+# File generated by puppet. DO NOT edit by hand
+command[check_opensearch_unassigned_shard_check_9200]=/usr/lib/nagios/plugins/check_elasticsearch_unassigned_shards.py --url http://localhost:9200 --timeout 4

Nrpe::Check[check_opensearch_unassigned_shard_check_9600]

Parameters differences:

--- Nrpe::Check[check_opensearch_unassigned_shard_check_9600].orig
+++ Nrpe::Check[check_opensearch_unassigned_shard_check_9600]

+    command => /usr/lib/nagios/plugins/check_elasticsearch_unassigned_shards.py --url http://localhost:9600 --timeout 4
+    before  => Monitoring::Service[opensearch_unassigned_shard_check_9600]
+    ensure  => present

File[/lib/systemd/system/prometheus-wmf-elasticsearch-exporter-9600.service]

Parameters differences:

--- File[/lib/systemd/system/prometheus-wmf-elasticsearch-exporter-9600.service].orig
+++ File[/lib/systemd/system/prometheus-wmf-elasticsearch-exporter-9600.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for prometheus-wmf-elasticsearch-exporter-9600.service (prometheus-wmf-elasticsearch-exporter-9600)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/prometheus-wmf-elasticsearch-exporter-9600.service.orig
+++ /lib/systemd/system/prometheus-wmf-elasticsearch-exporter-9600.service
@@ -0,0 +1,13 @@
+[Unit]
+Description=Prometheus WMF Elasticsearch Exporter
+
+[Service]
+Restart=always
+User=prometheus
+ExecStart=/usr/local/bin/prometheus-wmf-elasticsearch-exporter \
+    --listen :9121 \
+    --port 9600 \
+    --indices 
+
+[Install]
+WantedBy=multi-user.target

Nrpe::Check[check_opensearch_unassigned_shard_check_9200]

Parameters differences:

--- Nrpe::Check[check_opensearch_unassigned_shard_check_9200].orig
+++ Nrpe::Check[check_opensearch_unassigned_shard_check_9200]

+    command => /usr/lib/nagios/plugins/check_elasticsearch_unassigned_shards.py --url http://localhost:9200 --timeout 4
+    before  => Monitoring::Service[opensearch_unassigned_shard_check_9200]
+    ensure  => present

File[/etc/opensearch/instances]

Parameters differences:

--- File[/etc/opensearch/instances].orig
+++ File[/etc/opensearch/instances]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => file

Content differences:

--- /etc/opensearch/instances.orig
+++ /etc/opensearch/instances
@@ -0,0 +1,2 @@
+opensearch_1@production-search-codfw
+opensearch_1@production-search-psi-codfw

Exec[systemd daemon-reload for prometheus-elasticsearch-exporter-9600.service (prometheus-elasticsearch-exporter-9600)]

Parameters differences:

--- Exec[systemd daemon-reload for prometheus-elasticsearch-exporter-9600.service (prometheus-elasticsearch-exporter-9600)].orig
+++ Exec[systemd daemon-reload for prometheus-elasticsearch-exporter-9600.service (prometheus-elasticsearch-exporter-9600)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload
+    before      => ['Service[prometheus-elasticsearch-exporter-9600]']

File[/etc/logrotate.d/opensearch]

Parameters differences:

--- File[/etc/logrotate.d/opensearch].orig
+++ File[/etc/logrotate.d/opensearch]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/logrotate.d/opensearch.orig
+++ /etc/logrotate.d/opensearch
@@ -0,0 +1,12 @@
+# This file is managed by Puppet.
+# puppet:///logrotate/logrotate.erb
+
+/var/log/opensearch/*.log {
+    notifempty
+    daily
+    copytruncate
+    rotate 7
+    compress
+    delaycompress
+    missingok
+}

Service[prometheus-wmf-elasticsearch-exporter-9600]

Parameters differences:

--- Service[prometheus-wmf-elasticsearch-exporter-9600].orig
+++ Service[prometheus-wmf-elasticsearch-exporter-9600]

+    enable => True
+    ensure => running

Sudo::User[nrpe-check_opensearch_shards_9600]

Parameters differences:

--- Sudo::User[nrpe-check_opensearch_shards_9600].orig
+++ Sudo::User[nrpe-check_opensearch_shards_9600]

+    privileges => []
+    tag        => nrpe::check
+    require    => ['Class[Sudo]']
+    user       => nagios
+    ensure     => absent

File_line[rm_eno12399np0_clsact_eno12399np0]

Parameters differences:

--- File_line[rm_eno12399np0_clsact_eno12399np0].orig
+++ File_line[rm_eno12399np0_clsact_eno12399np0]

+    match             => post-up /usr/sbin/tc qdisc add dev eno12399np0 clsact
+    path              => /etc/network/interfaces
+    ensure            => absent
+    match_for_absence => True

Exec[systemd daemon-reload for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service (wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service)]

Parameters differences:

--- Exec[systemd daemon-reload for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service (wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service)].orig
+++ Exec[systemd daemon-reload for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service (wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

Systemd::Unit[nrpe2nodexp-opensearch_shard_size_check_9600.service]

Parameters differences:

--- Systemd::Unit[nrpe2nodexp-opensearch_shard_size_check_9600.service].orig
+++ Systemd::Unit[nrpe2nodexp-opensearch_shard_size_check_9600.service]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => nrpe2nodexp-opensearch_shard_size_check_9600.service
+    ensure            => absent

Systemd::Unit[opensearch-disable-readahead-production-search-codfw.service]

Parameters differences:

--- Systemd::Unit[opensearch-disable-readahead-production-search-codfw.service].orig
+++ Systemd::Unit[opensearch-disable-readahead-production-search-codfw.service]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => opensearch-disable-readahead-production-search-codfw.service
+    ensure            => present

File[/lib/systemd/system/nrpe2nodexp-check_tcp-mss-clamper_status.timer]

Parameters differences:

--- File[/lib/systemd/system/nrpe2nodexp-check_tcp-mss-clamper_status.timer].orig
+++ File[/lib/systemd/system/nrpe2nodexp-check_tcp-mss-clamper_status.timer]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for nrpe2nodexp-check_tcp-mss-clamper_status.timer (nrpe2nodexp-check_tcp-mss-clamper_status.timer)]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /lib/systemd/system/nrpe2nodexp-check_tcp-mss-clamper_status.timer.orig
+++ /lib/systemd/system/nrpe2nodexp-check_tcp-mss-clamper_status.timer
@@ -0,0 +1,13 @@
+[Unit]
+Description=Periodic execution of nrpe2nodexp-check_tcp-mss-clamper_status.service
+
+[Timer]
+Unit=nrpe2nodexp-check_tcp-mss-clamper_status.service
+# Accuracy sets the maximum time interval around the execution time we want to allow
+AccuracySec=15sec
+OnUnitInactiveSec=10min
+OnActiveSec=1s
+RandomizedDelaySec=0
+
+[Install]
+WantedBy=multi-user.target

File[/etc/conftool/schema.yaml]

Parameters differences:

--- File[/etc/conftool/schema.yaml].orig
+++ File[/etc/conftool/schema.yaml]

+    mode   => 0444
+    source => puppet:///modules/profile/conftool/schema.yaml
+    group  => root
+    owner  => root
+    ensure => file

Logrotate::Conf[wmf_auto_restart_prometheus-elasticsearch-exporter-9200]

Parameters differences:

--- Logrotate::Conf[wmf_auto_restart_prometheus-elasticsearch-exporter-9200].orig
+++ Logrotate::Conf[wmf_auto_restart_prometheus-elasticsearch-exporter-9200]

+    ensure => present

File[/etc/nginx/sites-available/production-search-psi-codfw]

Parameters differences:

--- File[/etc/nginx/sites-available/production-search-psi-codfw].orig
+++ File[/etc/nginx/sites-available/production-search-psi-codfw]

+    notify => Exec[nginx-reload]
+    group  => root
+    before => ['Service[nginx]']
+    owner  => root
+    ensure => present

Content differences:

--- /etc/nginx/sites-available/production-search-psi-codfw.orig
+++ /etc/nginx/sites-available/production-search-psi-codfw
@@ -0,0 +1,60 @@
+# Proxy site configuration file for production-search-psi-codfw
+# This file is managed by Puppet!
+upstream local_fe_production-search-psi-codfw {
+    server 10.192.6.18:9600 max_fails=0;
+}
+
+# SSL proxying
+server {
+    # Enabling TCP Fast Open is safe for HTTP over TLS. There is no idempotency
+    # concern replaying TLS Client Hello.
+    # https://tools.ietf.org/html/rfc7413#section-6.3.2
+    listen [::]:9643 default_server deferred backlog=16384 reuseport ipv6only=on fastopen=150 ssl ;
+    listen 9643 default_server deferred backlog=16384 reuseport fastopen=150 ssl ;
+    ssl on;
+    server_name cirrussearch2089.codfw.wmnet search.discovery.wmnet search-chi.discovery.wmnet search-omega.discovery.wmnet search-psi.discovery.wmnet search.svc.codfw.wmnet search-chi.svc.codfw.wmnet search-omega.svc.codfw.wmnet search-psi.svc.codfw.wmnet;
+
+    error_log   /var/log/nginx/production-search-psi-codfw.error.log;
+    access_log   off;
+
+    ssl_certificate /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chained.pem;
+    ssl_certificate_key /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet-key.pem;
+
+    keepalive_timeout 60;
+    keepalive_requests 100;
+
+    location / {
+        proxy_pass http://local_fe_production-search-psi-codfw;
+        proxy_http_version 1.1;
+        proxy_set_header Connection close; # should be default
+
+        # this should be in sync with Varnish's first_byte_timeout
+        # and PHP's max_execution_time
+        proxy_read_timeout 180s;
+
+        proxy_set_header Host $host;
+        proxy_set_header X-Client-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto https;
+        proxy_set_header X-Connection-Properties "H2=$h2; SSR=$session_reused; SSL=$ssl_protocol; C=$ssl_cipher;";
+
+        # Don't clobber the Server header from the backend.
+        proxy_pass_header Server;
+
+        proxy_redirect off;
+        proxy_buffering off;
+        proxy_request_buffering on;
+
+        # With proxy buffering off, proxy_buffer_size is still used to
+        # define the singular chunk of buffer the content passes
+        # through.  Apparently if nginx can't read all the response
+        # headers in the first such chunk, it fails with 502.  Upping
+        # this to 8k (from def 4k) solves the one reported case in
+        # https://phabricator.wikimedia.org/T161819
+        # --
+        # Raising this further from 8K to 32K, at least temporarily, due to T226840
+        # Note 32KB is also the similar default limit in our varnishes (http_resp_size)
+        proxy_buffer_size       32k;
+        proxy_buffers         4 32k;
+    }
+}

File[/etc/apt/sources.list]

Parameters differences:

--- File[/etc/apt/sources.list].orig
+++ File[/etc/apt/sources.list]

@@
-    before => ['Exec[apt_repository_wikimedia]', 'Exec[apt_repository_wikimedia-private]', 'Exec[apt_repository_debian-debug]', 'Exec[apt_repository_component-puppet7-apt.wikimedia.org-wikimedia-bullseye-wikimedia]']
+    before => ['Exec[apt_repository_wikimedia]', 'Exec[apt_repository_wikimedia-private]', 'Exec[apt_repository_debian-debug]', 'Exec[apt_repository_wikimedia-opensearch]', 'Exec[apt_repository_wikimedia-opensearch-plugins]', 'Exec[apt_repository_component-puppet7-apt.wikimedia.org-wikimedia-bullseye-wikimedia]']

Exec[Refresh tmpfile opensearch-production-search-codfw]

Parameters differences:

--- Exec[Refresh tmpfile opensearch-production-search-codfw].orig
+++ Exec[Refresh tmpfile opensearch-production-search-codfw]

+    refreshonly => True
+    subscribe   => File[/etc/tmpfiles.d/opensearch-production-search-codfw.conf]
+    command     => /bin/systemd-tmpfiles --create --remove '/etc/tmpfiles.d/opensearch-production-search-codfw.conf'
+    user        => root

File_line[auto_restart_file_presence_prometheus-wmf-elasticsearch-exporter-9200]

Parameters differences:

--- File_line[auto_restart_file_presence_prometheus-wmf-elasticsearch-exporter-9200].orig
+++ File_line[auto_restart_file_presence_prometheus-wmf-elasticsearch-exporter-9200]

+    require => File[/etc/debdeploy-client/autorestarts.conf]
+    path    => /etc/debdeploy-client/autorestarts.conf
+    ensure  => present
+    line    => prometheus-wmf-elasticsearch-exporter-9200

Systemd::Timer[opensearch-disable-readahead]

Parameters differences:

--- Systemd::Timer[opensearch-disable-readahead].orig
+++ Systemd::Timer[opensearch-disable-readahead]

+    fixed_random_delay => False
+    unit_name          => opensearch-disable-readahead.service
+    timer_intervals    => [{'start': 'OnUnitActiveSec', 'interval': '30min'}, {'start': 'OnBootSec', 'interval': '1min'}]
+    splay              => 0
+    accuracy           => 15sec
+    ensure             => absent

Prometheus::Alert::Rule[check_opensearch_shard_size_check_9200_ce424a305cb5464806cbf788bde4430a]

Parameters differences:

--- Prometheus::Alert::Rule[check_opensearch_shard_size_check_9200_ce424a305cb5464806cbf788bde4430a].orig
+++ Prometheus::Alert::Rule[check_opensearch_shard_size_check_9200_ce424a305cb5464806cbf788bde4430a]

+    dashboard          => TODO
+    expr               => (nagios_nrpe_check_result{alert_rule_hash="ce424a305cb5464806cbf788bde4430a",check_name="check_opensearch_shard_size_check_9200", status=~"(WARNING|CRITICAL)", severity=~"(warning|critical)"} > 0) * on (instance) group_left (team) role_owner
+    team               => observability
+    description        => OpenSearch shard size check - 9200
+    runbook            => https://wikitech.wikimedia.org/wiki/Search#If_it_has_been_indexed
+    def_label_whitelst => ['team', 'severity']
+    ensure             => absent
+    logs               => https://logstash.wikimedia.org/app/dashboards#/view/2d343ac0-6df8-11f0-8e08-7fab0da52b33?_g=(filters:!((query:(match_phrase:(event.module:check_opensearch_shard_size_check_9200))),(query:(match_phrase:(host.name:{{$labels.instance|stripPort}})))))
+    severity           => info
+    summary            => OpenSearch shard size check - 9200
+    instance           => ops
+    group              => nrpechecks
+    alert_name         => check_opensearch_shard_size_check_9200
+    for                => 259200m
+    site               => codfw

Etcd::Client::Config[/etc/etcd/etcdrc]

Parameters differences:

--- Etcd::Client::Config[/etc/etcd/etcdrc].orig
+++ Etcd::Client::Config[/etc/etcd/etcdrc]

+    world_readable => True
+    settings       => {'host': None, 'port': None, 'srv_domain': 'conftool.codfw.wmnet', 'ca_cert': '/etc/ssl/certs/wmf-ca-certificates.crt', 'protocol': 'https', 'allow_reconnect': True}
+    owner          => root
+    ensure         => present
+    group          => root

Systemd::Service[opensearch-production-search-codfw-gc-log-cleanup]

Parameters differences:

--- Systemd::Service[opensearch-production-search-codfw-gc-log-cleanup].orig
+++ Systemd::Service[opensearch-production-search-codfw-gc-log-cleanup]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => present
+    monitoring_critical      => False
+    require                  => Systemd::Unit[opensearch-production-search-codfw-gc-log-cleanup.service]
+    monitoring_enabled       => False
+    unit_type                => timer
+    restart                  => False

File[/var/log/wmf_auto_restart_prometheus-elasticsearch-exporter-9200]

Parameters differences:

--- File[/var/log/wmf_auto_restart_prometheus-elasticsearch-exporter-9200].orig
+++ File[/var/log/wmf_auto_restart_prometheus-elasticsearch-exporter-9200]

+    mode   => 0755
+    backup => False
+    force  => True
+    owner  => root
+    ensure => directory
+    group  => root

Conftool::Scripts::Safe_service_restart[opensearch_1@production-search-psi-codfw]

Parameters differences:

--- Conftool::Scripts::Safe_service_restart[opensearch_1@production-search-psi-codfw].orig
+++ Conftool::Scripts::Safe_service_restart[opensearch_1@production-search-psi-codfw]

+    require         => ['Class[Conftool::Scripts]']
+    lvs_pools       => ['search-psi-https']
+    max_concurrency => 5

File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.csr]

Parameters differences:

--- File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.csr].orig
+++ File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.csr]

+    mode   => 0440
+    group  => root
+    owner  => root
+    ensure => file

Profile::Prometheus::Wmf_elasticsearch_exporter[cirrussearch2089:9600]

Parameters differences:

--- Profile::Prometheus::Wmf_elasticsearch_exporter[cirrussearch2089:9600].orig
+++ Profile::Prometheus::Wmf_elasticsearch_exporter[cirrussearch2089:9600]

+    indices_to_monitor => []
+    prometheus_port    => 9121
+    elasticsearch_port => 9600

Concat::Fragment[main contacts]

Systemd::Timer::Job[opensearch-disable-readahead]

Parameters differences:

--- Systemd::Timer::Job[opensearch-disable-readahead].orig
+++ Systemd::Timer::Job[opensearch-disable-readahead]

+    description               => Disables readahead on all open files every 30 minutes to alleviate Cirrussearch / opensearch IO load spikes
+    syslog_match_startswith   => True
+    monitoring_contact_groups => admins
+    logging_enabled           => True
+    success_exit_status       => []
+    fixed_random_delay        => False
+    private_tmp               => False
+    logfile_basedir           => /var/log
+    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+    send_mail_only_on_error   => True
+    ignore_errors             => False
+    logfile_group             => root
+    interval                  => [{'start': 'OnUnitActiveSec', 'interval': '30min'}, {'start': 'OnBootSec', 'interval': '1min'}]
+    send_mail                 => False
+    logfile_name              => syslog.log
+    user                      => root
+    ensure                    => absent
+    send_mail_to              => root@cirrussearch2089.codfw.wmnet
+    logfile_perms             => all
+    syslog_force_stop         => True
+    environment               => {}
+    command                   => /usr/local/bin/opensearch-disable-readahead.sh
+    monitoring_enabled        => False

Systemd::Unit[opensearch-production-search-codfw-gc-log-cleanup.service]

Parameters differences:

--- Systemd::Unit[opensearch-production-search-codfw-gc-log-cleanup.service].orig
+++ Systemd::Unit[opensearch-production-search-codfw-gc-log-cleanup.service]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => opensearch-production-search-codfw-gc-log-cleanup.service
+    ensure            => present

Class[Poolcounter::Client::Python]

Parameters differences:

--- Class[Poolcounter::Client::Python].orig
+++ Class[Poolcounter::Client::Python]

+    backends => [{'label': 'pc1', 'fqdn': 'poolcounter2005.codfw.wmnet'}, {'label': 'pc2', 'fqdn': 'poolcounter2006.codfw.wmnet'}]
+    ensure   => present

File[/var/log/opensearch-production-search-psi-codfw-gc-log-cleanup]

Parameters differences:

--- File[/var/log/opensearch-production-search-psi-codfw-gc-log-cleanup].orig
+++ File[/var/log/opensearch-production-search-psi-codfw-gc-log-cleanup]

+    mode   => 0755
+    backup => False
+    force  => True
+    owner  => root
+    ensure => directory
+    group  => root

Java::Package[openjdk-jdk-11]

Parameters differences:

--- Java::Package[openjdk-jdk-11].orig
+++ Java::Package[openjdk-jdk-11]

+    egd_source   => /dev/random
+    package_info => {'version': '11', 'variant': 'jdk'}
+    hardened_tls => False

File[/lib/systemd/system/opensearch-production-search-codfw-gc-log-cleanup.service]

Parameters differences:

--- File[/lib/systemd/system/opensearch-production-search-codfw-gc-log-cleanup.service].orig
+++ File[/lib/systemd/system/opensearch-production-search-codfw-gc-log-cleanup.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for opensearch-production-search-codfw-gc-log-cleanup.service (opensearch-production-search-codfw-gc-log-cleanup.service)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/opensearch-production-search-codfw-gc-log-cleanup.service.orig
+++ /lib/systemd/system/opensearch-production-search-codfw-gc-log-cleanup.service
@@ -0,0 +1,8 @@
+[Unit]
+Description=Cleanup GC logs
+Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+
+[Service]
+Type=oneshot
+User=root
+ExecStart=/usr/bin/find /var/log/opensearch -name 'production-search-codfw_jvm_gc.*.log*' -mtime +30 -delete

Service[wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer]

Parameters differences:

--- Service[wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer].orig
+++ Service[wmf_auto_restart_prometheus-elasticsearch-exporter-9200.timer]

+    provider => systemd
+    enable   => True
+    ensure   => running

File[/lib/systemd/system/nrpe2nodexp-check_tcp-mss-clamper_status.service]

Parameters differences:

--- File[/lib/systemd/system/nrpe2nodexp-check_tcp-mss-clamper_status.service].orig
+++ File[/lib/systemd/system/nrpe2nodexp-check_tcp-mss-clamper_status.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for nrpe2nodexp-check_tcp-mss-clamper_status.service (nrpe2nodexp-check_tcp-mss-clamper_status.service)]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /lib/systemd/system/nrpe2nodexp-check_tcp-mss-clamper_status.service.orig
+++ /lib/systemd/system/nrpe2nodexp-check_tcp-mss-clamper_status.service
@@ -0,0 +1,11 @@
+[Unit]
+Description=execution of nrpe2nodexp for the check_check_tcp-mss-clamper_status command.
+Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+
+[Service]
+Type=oneshot
+User=nagios
+
+Group=prometheus-node-exporter
+SyslogIdentifier=nrpe2nodexp-check_tcp-mss-clamper_status
+ExecStart=-/usr/local/bin/nrpe2nodexp --alert-rule-hash "15c652bc7390794191321c22a2885d1f" --timeout 10 --check-command "check_check_tcp-mss-clamper_status"

Systemd::Unit[clean_up_dlq.timer]

Parameters differences:

--- Systemd::Unit[clean_up_dlq.timer].orig
+++ Systemd::Unit[clean_up_dlq.timer]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => clean_up_dlq.timer
+    ensure            => absent

Class[Profile::Opensearch::Server]

Parameters differences:

--- Class[Profile::Opensearch::Server].orig
+++ Class[Profile::Opensearch::Server]

+    instances             => {'production-search-codfw': {'cluster_name': 'production-search-codfw', 'short_cluster_name': 'chi', 'http_port': 9200, 'tls_port': 9243, 'certificate_name': 'search.discovery.wmnet', 'transport_tcp_port': 9300, 'unicast_hosts': ['cirrussearch2093.codfw.wmnet', 'cirrussearch2061.codfw.wmnet', 'cirrussearch2081.codfw.wmnet', 'cirrussearch2084.codfw.wmnet', 'cirrussearch2111.codfw.wmnet'], 'cluster_hosts': ['cirrussearch2061.codfw.wmnet', 'cirrussearch2062.codfw.wmnet', 'cirrussearch2063.codfw.wmnet', 'cirrussearch2064.codfw.wmnet', 'cirrussearch2065.codfw.wmnet', 'cirrussearch2066.codfw.wmnet', 'cirrussearch2067.codfw.wmnet', 'cirrussearch2068.codfw.wmnet', 'cirrussearch2069.codfw.wmnet', 'cirrussearch2070.codfw.wmnet', 'cirrussearch2071.codfw.wmnet', 'cirrussearch2072.codfw.wmnet', 'cirrussearch2073.codfw.wmnet', 'cirrussearch2074.codfw.wmnet', 'cirrussearch2075.codfw.wmnet', 'cirrussearch2076.codfw.wmnet', 'cirrussearch2077.codfw.wmnet', 'cirrussearch2078.codfw.wmnet', 'cirrussearch2079.codfw.wmnet', 'cirrussearch2080.codfw.wmnet', 'cirrussearch2081.codfw.wmnet', 'cirrussearch2082.codfw.wmnet', 'cirrussearch2083.codfw.wmnet', 'cirrussearch2084.codfw.wmnet', 'cirrussearch2085.codfw.wmnet', 'cirrussearch2086.codfw.wmnet', 'cirrussearch2087.codfw.wmnet', 'cirrussearch2088.codfw.wmnet', 'cirrussearch2089.codfw.wmnet', 'cirrussearch2090.codfw.wmnet', 'cirrussearch2091.codfw.wmnet', 'cirrussearch2092.codfw.wmnet', 'cirrussearch2093.codfw.wmnet', 'cirrussearch2094.codfw.wmnet', 'cirrussearch2095.codfw.wmnet', 'cirrussearch2096.codfw.wmnet', 'cirrussearch2097.codfw.wmnet', 'cirrussearch2098.codfw.wmnet', 'cirrussearch2099.codfw.wmnet', 'cirrussearch2100.codfw.wmnet', 'cirrussearch2101.codfw.wmnet', 'cirrussearch2102.codfw.wmnet', 'cirrussearch2103.codfw.wmnet', 'cirrussearch2104.codfw.wmnet', 'cirrussearch2105.codfw.wmnet', 'cirrussearch2106.codfw.wmnet', 'cirrussearch2107.codfw.wmnet', 'cirrussearch2108.codfw.wmnet', 'cirrussearch2109.codfw.wmnet', 'cirrussearch2110.codfw.wmnet', 'cirrussearch2111.codfw.wmnet', 'cirrussearch2112.codfw.wmnet', 'cirrussearch2113.codfw.wmnet', 'cirrussearch2114.codfw.wmnet', 'cirrussearch2115.codfw.wmnet'], 'expected_nodes': 36, 'recover_after_nodes': 24, 'indices_to_monitor': ['commonswiki_file', 'wikidatawiki_content', 'enwiki_general', 'enwiki_content', 'viwiki_general', 'commonswiki_general', 'cebwiki_content', 'metawiki_general', 'dewiki_content', 'frwiki_content']}, 'production-search-omega-codfw': {'cluster_name': 'production-search-omega-codfw', 'short_cluster_name': 'omega', 'http_port': 9400, 'tls_port': 9443, 'certificate_name': 'search.discovery.wmnet', 'transport_tcp_port': 9500, 'heap_memory': '10G', 'unicast_hosts': ['cirrussearch2092.codfw.wmnet', 'cirrussearch2100.codfw.wmnet', 'cirrussearch2106.codfw.wmnet', 'cirrussearch2073.codfw.wmnet', 'cirrussearch2086.codfw.wmnet'], 'cluster_hosts': ['cirrussearch2087.codfw.wmnet', 'cirrussearch2088.codfw.wmnet', 'cirrussearch2061.codfw.wmnet', 'cirrussearch2073.codfw.wmnet', 'cirrussearch2074.codfw.wmnet', 'cirrussearch2090.codfw.wmnet', 'cirrussearch2111.codfw.wmnet', 'cirrussearch2092.codfw.wmnet', 'cirrussearch2093.codfw.wmnet', 'cirrussearch2063.codfw.wmnet', 'cirrussearch2070.codfw.wmnet', 'cirrussearch2077.codfw.wmnet', 'cirrussearch2097.codfw.wmnet', 'cirrussearch2098.codfw.wmnet', 'cirrussearch2099.codfw.wmnet', 'cirrussearch2100.codfw.wmnet', 'cirrussearch2065.codfw.wmnet', 'cirrussearch2081.codfw.wmnet', 'cirrussearch2102.codfw.wmnet', 'cirrussearch2112.codfw.wmnet', 'cirrussearch2104.codfw.wmnet', 'cirrussearch2105.codfw.wmnet', 'cirrussearch2106.codfw.wmnet', 'cirrussearch2067.codfw.wmnet', 'cirrussearch2084.codfw.wmnet', 'cirrussearch2086.codfw.wmnet', 'cirrussearch2114.codfw.wmnet'], 'expected_nodes': 18, 'recover_after_nodes': 11}, 'production-search-psi-codfw': {'cluster_name': 'production-search-psi-codfw', 'short_cluster_name': 'psi', 'http_port': 9600, 'tls_port': 9643, 'certificate_name': 'search.discovery.wmnet', 'transport_tcp_port': 9700, 'heap_memory': '10G', 'unicast_hosts': ['cirrussearch2108.codfw.wmnet', 'cirrussearch2076.codfw.wmnet', 'cirrussearch2080.codfw.wmnet', 'cirrussearch2083.codfw.wmnet', 'cirrussearch2115.codfw.wmnet'], 'cluster_hosts': ['cirrussearch2089.codfw.wmnet', 'cirrussearch2062.codfw.wmnet', 'cirrussearch2069.codfw.wmnet', 'cirrussearch2075.codfw.wmnet', 'cirrussearch2076.codfw.wmnet', 'cirrussearch2091.codfw.wmnet', 'cirrussearch2094.codfw.wmnet', 'cirrussearch2095.codfw.wmnet', 'cirrussearch2064.codfw.wmnet', 'cirrussearch2078.codfw.wmnet', 'cirrussearch2079.codfw.wmnet', 'cirrussearch2080.codfw.wmnet', 'cirrussearch2096.codfw.wmnet', 'cirrussearch2110.codfw.wmnet', 'cirrussearch2101.codfw.wmnet', 'cirrussearch2066.codfw.wmnet', 'cirrussearch2071.codfw.wmnet', 'cirrussearch2072.codfw.wmnet', 'cirrussearch2082.codfw.wmnet', 'cirrussearch2083.codfw.wmnet', 'cirrussearch2103.codfw.wmnet', 'cirrussearch2113.codfw.wmnet', 'cirrussearch2107.codfw.wmnet', 'cirrussearch2108.codfw.wmnet', 'cirrussearch2068.codfw.wmnet', 'cirrussearch2085.codfw.wmnet', 'cirrussearch2109.codfw.wmnet', 'cirrussearch2115.codfw.wmnet'], 'expected_nodes': 18, 'recover_after_nodes': 11}}
+    java_home             => /usr/lib/jvm/java-11-openjdk-amd64
+    s3_username           => search:platform
+    dc_settings           => {}
+    logstash_host         => localhost
+    exporter_extra_config => 
+    version               => 1.0.0
+    enable_curator        => False
+    require               => ['Class[Profile::Java]', 'Class[Profile::Netbox::Host]']
+    logstash_logback_port => 11514
+    base_data_dir         => /srv/opensearch
+    native_lib_path       => /usr/share/opensearch/plugins/opensearch-knn/lib
+    common_settings       => {'awareness_attributes': 'row', 'plugins_mandatory': ['analysis-hebrew', 'analysis-icu', 'analysis-smartcn', 'analysis-stempel', 'analysis-ukrainian', 'cirrus-highlighter', 'opensearch-analysis-stconvert', 'opensearch-extra', 'opensearch-extra-analysis-esperanto', 'opensearch-extra-analysis-serbian', 'opensearch-extra-analysis-slovak', 'ltr'], 'heap_memory': '30G', 'recover_after_time': '5m', 'search_shard_count_limit': 5000, 'ltr_cache_size': '100mb', 'auto_create_index': '+apifeatureusage-*,+glent_*,-*', 'script_max_compilations_per_minute': 10000, 'bulk_thread_pool_executors': 6, 'bulk_thread_pool_capacity': 1000, 'filter_cache_size': '20%'}

File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-shard-size-check-9200.conf]

Parameters differences:

--- File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-shard-size-check-9200.conf].orig
+++ File[/etc/rsyslog.d/25-nrpe2nodexp-opensearch-shard-size-check-9200.conf]

+    mode   => 0444
+    notify => Service[rsyslog]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /etc/rsyslog.d/25-nrpe2nodexp-opensearch-shard-size-check-9200.conf.orig
+++ /etc/rsyslog.d/25-nrpe2nodexp-opensearch-shard-size-check-9200.conf
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: Apache-2.0
+if $programname contains "nrpe2nodexp-opensearch_shard_size_check_9200" then {
+    if ($msg contains "\"ecs.version\": \"1.7.0\"") then {
+        # Send logs to kafka
+        set $.log_outputs = "kafka ecs_170 local";
+    } else {
+        # Filter out non-relevant nrpe2nodexp messages
+        stop
+    }
+}

File[/etc/rsyslog.d/40-opensearch-disable-readahead.conf]

Parameters differences:

--- File[/etc/rsyslog.d/40-opensearch-disable-readahead.conf].orig
+++ File[/etc/rsyslog.d/40-opensearch-disable-readahead.conf]

+    mode   => 0444
+    notify => Service[rsyslog]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /etc/rsyslog.d/40-opensearch-disable-readahead.conf.orig
+++ /etc/rsyslog.d/40-opensearch-disable-readahead.conf
@@ -0,0 +1,10 @@
+# rsyslog.conf(5) configuration file for services.
+# This file is managed by Puppet.
+if $programname startswith "opensearch-disable-readahead" then {
+    action(
+        type="omfile" file="/var/log/opensearch-disable-readahead/syslog.log"
+        fileOwner="root" fileGroup="root"
+        fileCreateMode="0644"
+    )
+    & stop
+}

Systemd::Timer[prometheus_lvs_realserver_mss]

Parameters differences:

--- Systemd::Timer[prometheus_lvs_realserver_mss].orig
+++ Systemd::Timer[prometheus_lvs_realserver_mss]

+    fixed_random_delay => False
+    unit_name          => prometheus_lvs_realserver_mss.service
+    timer_intervals    => [{'start': 'OnCalendar', 'interval': 'minutely'}]
+    splay              => 0
+    accuracy           => 15sec
+    ensure             => present

Systemd::Timer[nrpe2nodexp-opensearch_shard_size_check_9600]

Parameters differences:

--- Systemd::Timer[nrpe2nodexp-opensearch_shard_size_check_9600].orig
+++ Systemd::Timer[nrpe2nodexp-opensearch_shard_size_check_9600]

+    fixed_random_delay => False
+    unit_name          => nrpe2nodexp-opensearch_shard_size_check_9600.service
+    timer_intervals    => [{'start': 'OnUnitInactiveSec', 'interval': '1440min'}, {'interval': '1s', 'start': 'OnActiveSec'}]
+    splay              => 0
+    accuracy           => 15sec
+    ensure             => absent

File[/lib/systemd/system/opensearch_1@.service]

Parameters differences:

--- File[/lib/systemd/system/opensearch_1@.service].orig
+++ File[/lib/systemd/system/opensearch_1@.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for opensearch_1@.service (opensearch_1@.service)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/opensearch_1@.service.orig
+++ /lib/systemd/system/opensearch_1@.service
@@ -0,0 +1,73 @@
+[Unit]
+Description=OpenSearch (cluster %i)
+PartOf=opensearch.service
+Documentation=https://opensearch.org/docs
+Wants=network-online.target
+After=network-online.target
+
+[Service]
+Type=notify
+RuntimeDirectory=opensearch
+PrivateTmp=true
+Environment=OPENSEARCH_HOME=/usr/share/opensearch
+Environment=OPENSEARCH_PATH_CONF=/etc/opensearch/%i
+Environment=OPENSEARCH_JVM_OPTIONS=/etc/opensearch/%i/jvm.options
+Environment=PID_DIR=/run/opensearch-%i
+Environment=OPENSEARCH_SD_NOTIFY=true
+
+Environment=LD_LIBRARY_PATH=/usr/share/opensearch/plugins/opensearch-knn/lib
+Environment=JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
+
+WorkingDirectory=/usr/share/opensearch
+
+User=opensearch
+Group=opensearch
+
+# Force creation of /run/opensearch (see T276198)
+ExecStartPre=/bin/systemd-tmpfiles --create /etc/tmpfiles.d/opensearch-%i.conf
+ExecStart=/usr/share/opensearch/bin/opensearch \
+        -p ${PID_DIR}/%i.pid \
+        --quiet \
+        -Epath.logs=/var/log/opensearch \
+        -Epath.data=/srv/opensearch/%i
+
+# StandardOutput is configured to redirect to journalctl since
+# some error messages may be logged in standard output before
+# opensearch logging system is initialized. OpenSearch
+# stores its logs in /var/log/opensearch and does not use
+# journalctl by default. If you also want to enable journalctl
+# logging, you can simply remove the "quiet" option from ExecStart.
+StandardOutput=journal
+StandardError=inherit
+
+# Specifies the maximum file descriptor number that can be opened by this process
+LimitNOFILE=65535
+
+# Specifies the maximum number of processes
+LimitNPROC=4096
+
+# Specifies the maximum size of virtual memory
+LimitAS=infinity
+
+# Specifies the maximum file size
+LimitFSIZE=infinity
+
+# Disable timeout logic and wait until process is stopped
+TimeoutStopSec=0
+
+# SIGTERM signal is used to stop the Java process
+KillSignal=SIGTERM
+
+# Send the signal only to the JVM rather than its control group
+KillMode=process
+
+# Java process is never killed
+SendSIGKILL=no
+
+# When a JVM receives a SIGTERM signal it exits with code 143
+SuccessExitStatus=143
+
+[Install]
+WantedBy=multi-user.target
+
+# Built for packages-7.4.1 (packages)

File[/etc/sudoers.d/nrpe-check_opensearch_shard_size_check_9600]

Parameters differences:

--- File[/etc/sudoers.d/nrpe-check_opensearch_shard_size_check_9600].orig
+++ File[/etc/sudoers.d/nrpe-check_opensearch_shard_size_check_9600]

+    group   => root
+    require => Package[nagios-nrpe-server]
+    owner   => root
+    ensure  => absent

Exec[tlsproxy localssl default_server on ports [9243]]

Parameters differences:

--- Exec[tlsproxy localssl default_server on ports [9243]].orig
+++ Exec[tlsproxy localssl default_server on ports [9243]]

+    refreshonly => True
+    onlyif      => /bin/false
+    command     => /bin/true

Service[prometheus-elasticsearch-exporter]

Parameters differences:

--- Service[prometheus-elasticsearch-exporter].orig
+++ Service[prometheus-elasticsearch-exporter]

+    require => Package[prometheus-elasticsearch-exporter]
+    ensure  => stopped

Ssh::Userkey[ebernhardson]

Parameters differences:

--- Ssh::Userkey[ebernhardson].orig
+++ Ssh::Userkey[ebernhardson]

+    user   => ebernhardson
+    ensure => present

File[/lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service]

Parameters differences:

--- File[/lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service].orig
+++ File[/lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service (wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service.orig
+++ /lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service
@@ -0,0 +1,8 @@
+[Unit]
+Description=Auto restart job: prometheus-wmf-elasticsearch-exporter-9600
+Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+
+[Service]
+Type=oneshot
+User=root
+ExecStart=/usr/local/sbin/wmf-auto-restart -s prometheus-wmf-elasticsearch-exporter-9600

Package[nginx-common]

Parameters differences:

--- Package[nginx-common].orig
+++ Package[nginx-common]

+    provider => apt
+    ensure   => installed

Exec[systemd reload for nginx systemd fragments]

Parameters differences:

--- Exec[systemd reload for nginx systemd fragments].orig
+++ Exec[systemd reload for nginx systemd fragments]

+    refreshonly => True
+    subscribe   => ['File[/etc/systemd/system/nginx.service.d/numa.conf]', 'File[/etc/systemd/system/nginx.service.d/security.conf]']
+    command     => /bin/systemctl daemon-reload

File[/usr/local/bin/depool]

Parameters differences:

--- File[/usr/local/bin/depool].orig
+++ File[/usr/local/bin/depool]

+    mode   => 0555
+    source => puppet:///modules/conftool/conftool-simple-command.sh
+    group  => root
+    owner  => root
+    ensure => present

Exec[ip link add name ipip0 type ipip external]

Parameters differences:

--- Exec[ip link add name ipip0 type ipip external].orig
+++ Exec[ip link add name ipip0 type ipip external]

+    unless  => ip link show ipip0
+    path    => /bin:/usr/bin
+    returns => [0, 2]

Service[opensearch-disable-readahead-production-search-psi-codfw.timer]

Parameters differences:

--- Service[opensearch-disable-readahead-production-search-psi-codfw.timer].orig
+++ Service[opensearch-disable-readahead-production-search-psi-codfw.timer]

+    provider => systemd
+    enable   => True
+    ensure   => running

File[/usr/lib/nagios/plugins/check_elasticsearch_unassigned_shards.py]

Parameters differences:

--- File[/usr/lib/nagios/plugins/check_elasticsearch_unassigned_shards.py].orig
+++ File[/usr/lib/nagios/plugins/check_elasticsearch_unassigned_shards.py]

+    mode   => 0755
+    group  => root
+    owner  => root
+    source => puppet:///modules/icinga/elasticsearch/check_elasticsearch_unassigned_shards.py

Monitoring::Service[raid_md]

Parameters differences:

--- Monitoring::Service[raid_md].orig
+++ Monitoring::Service[raid_md]

@@
-    contact_group => admins
+    contact_group => team-data-platform,team-discovery

Class[Profile::Rsyslog::Udp_json_logback_compat]

Parameters differences:

--- Class[Profile::Rsyslog::Udp_json_logback_compat].orig
+++ Class[Profile::Rsyslog::Udp_json_logback_compat]

+    queue_enabled_sites   => ['ulsfo', 'esams', 'eqsin', 'eqiad', 'codfw', 'drmrs', 'magru']
+    port                  => 11514
+    logging_kafka_brokers => ['kafka-logging1001.eqiad.wmnet:9093', 'kafka-logging1002.eqiad.wmnet:9093', 'kafka-logging1003.eqiad.wmnet:9093', 'kafka-logging1004.eqiad.wmnet:9093', 'kafka-logging1005.eqiad.wmnet:9093']

Motd::Message[insetup::data_platform_ferm]

Parameters differences:

--- Motd::Message[insetup::data_platform_ferm].orig
+++ Motd::Message[insetup::data_platform_ferm]

-    message  => cirrussearch2089 is a Host being setup by Data Platform SREs (insetup::data_platform_ferm)
-    priority => 5
-    ensure   => present

Systemd::Service[prometheus-elasticsearch-exporter-9600]

Parameters differences:

--- Systemd::Service[prometheus-elasticsearch-exporter-9600].orig
+++ Systemd::Service[prometheus-elasticsearch-exporter-9600]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {'ensure': 'running'}
+    ensure                   => present
+    monitoring_critical      => False
+    require                  => Package[prometheus-elasticsearch-exporter]
+    monitoring_enabled       => False
+    unit_type                => service
+    restart                  => False

Systemd::Timer[opensearch-production-search-codfw-gc-log-cleanup]

Parameters differences:

--- Systemd::Timer[opensearch-production-search-codfw-gc-log-cleanup].orig
+++ Systemd::Timer[opensearch-production-search-codfw-gc-log-cleanup]

+    fixed_random_delay => False
+    unit_name          => opensearch-production-search-codfw-gc-log-cleanup.service
+    timer_intervals    => [{'start': 'OnCalendar', 'interval': '*-*-* 02:12:00'}]
+    splay              => 0
+    accuracy           => 15sec
+    ensure             => present

Exec[systemd daemon-reload for prometheus_lvs_realserver_mss.timer (prometheus_lvs_realserver_mss.timer)]

Parameters differences:

--- Exec[systemd daemon-reload for prometheus_lvs_realserver_mss.timer (prometheus_lvs_realserver_mss.timer)].orig
+++ Exec[systemd daemon-reload for prometheus_lvs_realserver_mss.timer (prometheus_lvs_realserver_mss.timer)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload
+    before      => ['Service[prometheus_lvs_realserver_mss.timer]']

Sudo::User[nrpe-check_opensearch_shard_size_check_9200]

Parameters differences:

--- Sudo::User[nrpe-check_opensearch_shard_size_check_9200].orig
+++ Sudo::User[nrpe-check_opensearch_shard_size_check_9200]

+    privileges => []
+    tag        => nrpe::check
+    require    => ['Class[Sudo]']
+    user       => nagios
+    ensure     => absent

Profile::Auto_restarts::Service[prometheus-elasticsearch-exporter-9200]

Parameters differences:

--- Profile::Auto_restarts::Service[prometheus-elasticsearch-exporter-9200].orig
+++ Profile::Auto_restarts::Service[prometheus-elasticsearch-exporter-9200]

+    ensure => present

Exec[systemd daemon-reload for wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service (wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service)]

Parameters differences:

--- Exec[systemd daemon-reload for wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service (wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service)].orig
+++ Exec[systemd daemon-reload for wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service (wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

File[/lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer]

Parameters differences:

--- File[/lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer].orig
+++ File[/lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer (nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer)]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer.orig
+++ /lib/systemd/system/nrpe2nodexp-opensearch_unassigned_shard_check_9200.timer
@@ -0,0 +1,13 @@
+[Unit]
+Description=Periodic execution of nrpe2nodexp-opensearch_unassigned_shard_check_9200.service
+
+[Timer]
+Unit=nrpe2nodexp-opensearch_unassigned_shard_check_9200.service
+# Accuracy sets the maximum time interval around the execution time we want to allow
+AccuracySec=15sec
+OnUnitInactiveSec=720min
+OnActiveSec=1s
+RandomizedDelaySec=0
+
+[Install]
+WantedBy=multi-user.target

File[/etc/ferm/conf.d/10_opensearch-http-9200]

Parameters differences:

--- File[/etc/ferm/conf.d/10_opensearch-http-9200].orig
+++ File[/etc/ferm/conf.d/10_opensearch-http-9200]

+    mode    => 0400
+    require => File[/etc/ferm/conf.d]
+    tag     => ferm
+    notify  => Service[ferm]
+    group   => root
+    owner   => root
+    ensure  => present

Content differences:

--- /etc/ferm/conf.d/10_opensearch-http-9200.orig
+++ /etc/ferm/conf.d/10_opensearch-http-9200
@@ -0,0 +1,8 @@
+# Autogenerated by puppet. DO NOT EDIT BY HAND!
+#
+# 
+&R_SERVICE(tcp, 9200, $DOMAIN_NETWORKS);
+
+
+
+&NO_TRACK(tcp, 9200);

Logrotate::Conf[prometheus_ferm_mss]

Parameters differences:

--- Logrotate::Conf[prometheus_ferm_mss].orig
+++ Logrotate::Conf[prometheus_ferm_mss]

+    ensure => present

Rsyslog::Conf[prometheus_ferm_mss]

Parameters differences:

--- Rsyslog::Conf[prometheus_ferm_mss].orig
+++ Rsyslog::Conf[prometheus_ferm_mss]

+    mode     => 0444
+    require  => File[/var/log/prometheus_ferm_mss]
+    priority => 40
+    ensure   => present

Rsyslog::Conf[nrpe2nodexp-opensearch_shards_9600]

Parameters differences:

--- Rsyslog::Conf[nrpe2nodexp-opensearch_shards_9600].orig
+++ Rsyslog::Conf[nrpe2nodexp-opensearch_shards_9600]

+    mode     => 0444
+    priority => 25
+    ensure   => absent

File[/etc/systemd/system/nginx.service.d]

Parameters differences:

--- File[/etc/systemd/system/nginx.service.d].orig
+++ File[/etc/systemd/system/nginx.service.d]

+    mode   => 0555
+    group  => root
+    owner  => root
+    ensure => directory

File[/etc/etcd]

Parameters differences:

--- File[/etc/etcd].orig
+++ File[/etc/etcd]

+    mode   => 0755
+    group  => root
+    owner  => root
+    ensure => directory

Exec[systemd daemon-reload for opensearch-disable-readahead-production-search-codfw.timer (opensearch-disable-readahead-production-search-codfw.timer)]

Parameters differences:

--- Exec[systemd daemon-reload for opensearch-disable-readahead-production-search-codfw.timer (opensearch-disable-readahead-production-search-codfw.timer)].orig
+++ Exec[systemd daemon-reload for opensearch-disable-readahead-production-search-codfw.timer (opensearch-disable-readahead-production-search-codfw.timer)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload
+    before      => ['Service[opensearch-disable-readahead-production-search-codfw.timer]']

File[/etc/poolcounter-backends.yaml]

Parameters differences:

--- File[/etc/poolcounter-backends.yaml].orig
+++ File[/etc/poolcounter-backends.yaml]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/poolcounter-backends.yaml.orig
+++ /etc/poolcounter-backends.yaml
@@ -0,0 +1,3 @@
+---
+- pc1:poolcounter2005.codfw.wmnet:1
+- pc2:poolcounter2006.codfw.wmnet:1

File[/var/log/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]

Parameters differences:

--- File[/var/log/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600].orig
+++ File[/var/log/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]

+    mode   => 0755
+    backup => False
+    force  => True
+    owner  => root
+    ensure => directory
+    group  => root

File[/var/log/opensearch-disable-readahead-production-search-codfw]

Parameters differences:

--- File[/var/log/opensearch-disable-readahead-production-search-codfw].orig
+++ File[/var/log/opensearch-disable-readahead-production-search-codfw]

+    mode   => 0755
+    backup => False
+    force  => True
+    owner  => root
+    ensure => directory
+    group  => root

Class[Profile::Tlsproxy::Instance]

Parameters differences:

--- Class[Profile::Tlsproxy::Instance].orig
+++ Class[Profile::Tlsproxy::Instance]

+    ssl_compatibility_mode     => compat
+    require                    => ['Class[Profile::Tcp_fast_open]']
+    bootstrap_protection       => False
+    numa_iface                 => lo
+    nginx_client_max_body_size => 100m
+    nginx_variant              => light

Systemd::Unit[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service]

Parameters differences:

--- Systemd::Unit[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service].orig
+++ Systemd::Unit[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service
+    ensure            => present

Exec[/usr/sbin/tc qdisc del dev lo clsact]

Parameters differences:

--- Exec[/usr/sbin/tc qdisc del dev lo clsact].orig
+++ Exec[/usr/sbin/tc qdisc del dev lo clsact]

+    onlyif => /usr/sbin/tc qdisc show dev lo | grep -q clsact

File[/etc/etcd/etcdrc]

Parameters differences:

--- File[/etc/etcd/etcdrc].orig
+++ File[/etc/etcd/etcdrc]

+    mode      => 0444
+    show_diff => True
+    group     => root
+    owner     => root
+    ensure    => present

Content differences:

--- /etc/etcd/etcdrc.orig
+++ /etc/etcd/etcdrc
@@ -0,0 +1,5 @@
+allow_reconnect: true
+ca_cert: /etc/ssl/certs/wmf-ca-certificates.crt
+protocol: https
+srv_domain: conftool.codfw.wmnet
+

Systemd::Unit[nrpe2nodexp-opensearch_shards_9600.timer]

Parameters differences:

--- Systemd::Unit[nrpe2nodexp-opensearch_shards_9600.timer].orig
+++ Systemd::Unit[nrpe2nodexp-opensearch_shards_9600.timer]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => nrpe2nodexp-opensearch_shards_9600.timer
+    ensure            => absent

Exec[stop-default-nginx]

Parameters differences:

--- Exec[stop-default-nginx].orig
+++ Exec[stop-default-nginx]

+    refreshonly => True
+    subscribe   => Package[nginx-light]
+    command     => /usr/sbin/service nginx stop
+    before      => Service[nginx]

Systemd::Service[prometheus-wmf-elasticsearch-exporter-9200]

Parameters differences:

--- Systemd::Service[prometheus-wmf-elasticsearch-exporter-9200].orig
+++ Systemd::Service[prometheus-wmf-elasticsearch-exporter-9200]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => present
+    monitoring_critical      => False
+    require                  => File[/usr/local/bin/prometheus-wmf-elasticsearch-exporter]
+    subscribe                => File[/usr/local/bin/prometheus-wmf-elasticsearch-exporter]
+    monitoring_enabled       => False
+    unit_type                => service
+    restart                  => True

File[/var/lib/prometheus/node.d/role_owner.prom]

Content differences:

--- /var/lib/prometheus/node.d/role_owner.prom.orig
+++ /var/lib/prometheus/node.d/role_owner.prom
@@ -1,3 +1,3 @@
 # HELP role_owner The team owner of the server role
 # TYPE role_owner gauge
-role_owner{team="data-platform",role="insetup::data_platform_ferm",cluster="insetup"} 1.0
+role_owner{team="search-platform",role="cirrus::opensearch",cluster="elasticsearch"} 1.0

File[/etc/rsyslog.d/40-wmf-auto-restart-prometheus-elasticsearch-exporter-9600.conf]

Parameters differences:

--- File[/etc/rsyslog.d/40-wmf-auto-restart-prometheus-elasticsearch-exporter-9600.conf].orig
+++ File[/etc/rsyslog.d/40-wmf-auto-restart-prometheus-elasticsearch-exporter-9600.conf]

+    mode   => 0444
+    notify => Service[rsyslog]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/rsyslog.d/40-wmf-auto-restart-prometheus-elasticsearch-exporter-9600.conf.orig
+++ /etc/rsyslog.d/40-wmf-auto-restart-prometheus-elasticsearch-exporter-9600.conf
@@ -0,0 +1,10 @@
+# rsyslog.conf(5) configuration file for services.
+# This file is managed by Puppet.
+if $programname startswith "wmf_auto_restart_prometheus-elasticsearch-exporter-9600" then {
+    action(
+        type="omfile" file="/var/log/wmf_auto_restart_prometheus-elasticsearch-exporter-9600/syslog.log"
+        fileOwner="root" fileGroup="root"
+        fileCreateMode="0644"
+    )
+    & stop
+}

Exec[systemd daemon-reload for opensearch-production-search-psi-codfw-gc-log-cleanup.service (opensearch-production-search-psi-codfw-gc-log-cleanup.service)]

Parameters differences:

--- Exec[systemd daemon-reload for opensearch-production-search-psi-codfw-gc-log-cleanup.service (opensearch-production-search-psi-codfw-gc-log-cleanup.service)].orig
+++ Exec[systemd daemon-reload for opensearch-production-search-psi-codfw-gc-log-cleanup.service (opensearch-production-search-psi-codfw-gc-log-cleanup.service)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

Exec[systemd daemon-reload for opensearch-production-search-codfw-gc-log-cleanup.service (opensearch-production-search-codfw-gc-log-cleanup.service)]

Parameters differences:

--- Exec[systemd daemon-reload for opensearch-production-search-codfw-gc-log-cleanup.service (opensearch-production-search-codfw-gc-log-cleanup.service)].orig
+++ Exec[systemd daemon-reload for opensearch-production-search-codfw-gc-log-cleanup.service (opensearch-production-search-codfw-gc-log-cleanup.service)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

File[/etc/ferm/conf.d/10_opensearch-inter-node-9300]

Parameters differences:

--- File[/etc/ferm/conf.d/10_opensearch-inter-node-9300].orig
+++ File[/etc/ferm/conf.d/10_opensearch-inter-node-9300]

+    mode    => 0400
+    require => File[/etc/ferm/conf.d]
+    tag     => ferm
+    notify  => Service[ferm]
+    group   => root
+    owner   => root
+    ensure  => present

Content differences:

--- /etc/ferm/conf.d/10_opensearch-inter-node-9300.orig
+++ /etc/ferm/conf.d/10_opensearch-inter-node-9300
@@ -0,0 +1,8 @@
+# Autogenerated by puppet. DO NOT EDIT BY HAND!
+#
+# 
+&R_SERVICE(tcp, 9300, @resolve((cirrussearch2061.codfw.wmnet cirrussearch2062.codfw.wmnet cirrussearch2063.codfw.wmnet cirrussearch2064.codfw.wmnet cirrussearch2065.codfw.wmnet cirrussearch2066.codfw.wmnet cirrussearch2067.codfw.wmnet cirrussearch2068.codfw.wmnet cirrussearch2069.codfw.wmnet cirrussearch2070.codfw.wmnet cirrussearch2071.codfw.wmnet cirrussearch2072.codfw.wmnet cirrussearch2073.codfw.wmnet cirrussearch2074.codfw.wmnet cirrussearch2075.codfw.wmnet cirrussearch2076.codfw.wmnet cirrussearch2077.codfw.wmnet cirrussearch2078.codfw.wmnet cirrussearch2079.codfw.wmnet cirrussearch2080.codfw.wmnet cirrussearch2081.codfw.wmnet cirrussearch2082.codfw.wmnet cirrussearch2083.codfw.wmnet cirrussearch2084.codfw.wmnet cirrussearch2085.codfw.wmnet cirrussearch2086.codfw.wmnet cirrussearch2087.codfw.wmnet cirrussearch2088.codfw.wmnet cirrussearch2089.codfw.wmnet cirrussearch2090.codfw.wmnet cirrussearch2091.codfw.wmnet cirrussearch2092.codfw.wmnet cirrussearch2093.codfw.wmnet cirrussearch2094.codfw.wmnet cirrussearch2095.codfw.wmnet cirrussearch2096.codfw.wmnet cirrussearch2097.codfw.wmnet cirrussearch2098.codfw.wmnet cirrussearch2099.codfw.wmnet cirrussearch2100.codfw.wmnet cirrussearch2101.codfw.wmnet cirrussearch2102.codfw.wmnet cirrussearch2103.codfw.wmnet cirrussearch2104.codfw.wmnet cirrussearch2105.codfw.wmnet cirrussearch2106.codfw.wmnet cirrussearch2107.codfw.wmnet cirrussearch2108.codfw.wmnet cirrussearch2109.codfw.wmnet cirrussearch2110.codfw.wmnet cirrussearch2111.codfw.wmnet cirrussearch2112.codfw.wmnet cirrussearch2113.codfw.wmnet cirrussearch2114.codfw.wmnet cirrussearch2115.codfw.wmnet)));
+
+
+
+&NO_TRACK(tcp, 9300);

Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer (nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer)]

Parameters differences:

--- Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer (nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer)].orig
+++ Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer (nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

Systemd::Unit[nrpe2nodexp-opensearch_unassigned_shard_check_9200.service]

Parameters differences:

--- Systemd::Unit[nrpe2nodexp-opensearch_unassigned_shard_check_9200.service].orig
+++ Systemd::Unit[nrpe2nodexp-opensearch_unassigned_shard_check_9200.service]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => nrpe2nodexp-opensearch_unassigned_shard_check_9200.service
+    ensure            => absent

Service[logstash]

Parameters differences:

--- Service[logstash].orig
+++ Service[logstash]

+    hasrestart => True
+    enable     => True
+    hasstatus  => True
+    provider   => systemd
+    ensure     => running

Systemd::Unit[clean_up_dlq.service]

Parameters differences:

--- Systemd::Unit[clean_up_dlq.service].orig
+++ Systemd::Unit[clean_up_dlq.service]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => clean_up_dlq.service
+    ensure            => absent

File[/etc/rsyslog.d/40-clean-up-dlq.conf]

Parameters differences:

--- File[/etc/rsyslog.d/40-clean-up-dlq.conf].orig
+++ File[/etc/rsyslog.d/40-clean-up-dlq.conf]

+    mode   => 0444
+    notify => Service[rsyslog]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /etc/rsyslog.d/40-clean-up-dlq.conf.orig
+++ /etc/rsyslog.d/40-clean-up-dlq.conf
@@ -0,0 +1,10 @@
+# rsyslog.conf(5) configuration file for services.
+# This file is managed by Puppet.
+if $programname startswith "clean_up_dlq" then {
+    action(
+        type="omfile" file="/var/log/clean_up_dlq/syslog.log"
+        fileOwner="root" fileGroup="root"
+        fileCreateMode="0644"
+    )
+    & stop
+}

File[/etc/rsyslog.d/40-opensearch-production-search-codfw-gc-log-cleanup.conf]

Parameters differences:

--- File[/etc/rsyslog.d/40-opensearch-production-search-codfw-gc-log-cleanup.conf].orig
+++ File[/etc/rsyslog.d/40-opensearch-production-search-codfw-gc-log-cleanup.conf]

+    mode   => 0444
+    notify => Service[rsyslog]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/rsyslog.d/40-opensearch-production-search-codfw-gc-log-cleanup.conf.orig
+++ /etc/rsyslog.d/40-opensearch-production-search-codfw-gc-log-cleanup.conf
@@ -0,0 +1,10 @@
+# rsyslog.conf(5) configuration file for services.
+# This file is managed by Puppet.
+if $programname startswith "opensearch-production-search-codfw-gc-log-cleanup" then {
+    action(
+        type="omfile" file="/var/log/opensearch-production-search-codfw-gc-log-cleanup/syslog.log"
+        fileOwner="root" fileGroup="root"
+        fileCreateMode="0644"
+    )
+    & stop
+}

Service[tcp-mss-clamper]

Parameters differences:

--- Service[tcp-mss-clamper].orig
+++ Service[tcp-mss-clamper]

+    before => ['Exec[systemd daemon-reload for tcp-mss-clamper.service (tcp-mss-clamper)]']
+    enable => False
+    ensure => stopped

Monitoring::Exported_nagios_service[cirrussearch2089 cpu_microcode_status]

Parameters differences:

--- Monitoring::Exported_nagios_service[cirrussearch2089 cpu_microcode_status].orig
+++ Monitoring::Exported_nagios_service[cirrussearch2089 cpu_microcode_status]

@@
-    notifications_enabled => 0
+    notifications_enabled => 1
@@
-    servicegroups         => insetup_codfw
+    servicegroups         => elasticsearch_codfw

Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9600.service (nrpe2nodexp-opensearch_unassigned_shard_check_9600.service)]

Parameters differences:

--- Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9600.service (nrpe2nodexp-opensearch_unassigned_shard_check_9600.service)].orig
+++ Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9600.service (nrpe2nodexp-opensearch_unassigned_shard_check_9600.service)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

Admin::Groupmembers[elasticsearch-roots]

Parameters differences:

--- Admin::Groupmembers[elasticsearch-roots].orig
+++ Admin::Groupmembers[elasticsearch-roots]

+    before         => Exec[enforce-users-groups-cleanup]
+    default_member => root

Exec[apt_repository_wikimedia-opensearch]

Parameters differences:

--- Exec[apt_repository_wikimedia-opensearch].orig
+++ Exec[apt_repository_wikimedia-opensearch]

+    refreshonly => True
+    command     => /usr/bin/apt-get update

File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chained.pem]

Parameters differences:

--- File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chained.pem].orig
+++ File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chained.pem]

+    require => Exec[create chained cert /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chain.pem]
+    notify  => ['Exec[nginx-reload]', 'Exec[nginx-reload]']
+    group   => root
+    owner   => root
+    ensure  => file

Systemd::Service[wmf_auto_restart_prometheus-elasticsearch-exporter-9200]

Parameters differences:

--- Systemd::Service[wmf_auto_restart_prometheus-elasticsearch-exporter-9200].orig
+++ Systemd::Service[wmf_auto_restart_prometheus-elasticsearch-exporter-9200]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => present
+    monitoring_critical      => False
+    require                  => Systemd::Unit[wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service]
+    monitoring_enabled       => False
+    unit_type                => timer
+    restart                  => False

Exec[systemd daemon-reload for opensearch-production-search-codfw-gc-log-cleanup.timer (opensearch-production-search-codfw-gc-log-cleanup.timer)]

Parameters differences:

--- Exec[systemd daemon-reload for opensearch-production-search-codfw-gc-log-cleanup.timer (opensearch-production-search-codfw-gc-log-cleanup.timer)].orig
+++ Exec[systemd daemon-reload for opensearch-production-search-codfw-gc-log-cleanup.timer (opensearch-production-search-codfw-gc-log-cleanup.timer)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload
+    before      => ['Service[opensearch-production-search-codfw-gc-log-cleanup.timer]']

Systemd::Timer[prometheus_ferm_mss]

Parameters differences:

--- Systemd::Timer[prometheus_ferm_mss].orig
+++ Systemd::Timer[prometheus_ferm_mss]

+    fixed_random_delay => False
+    unit_name          => prometheus_ferm_mss.service
+    timer_intervals    => [{'start': 'OnCalendar', 'interval': 'minutely'}]
+    splay              => 0
+    accuracy           => 15sec
+    ensure             => present

Exec[systemd daemon-reload for opensearch_1@.service (opensearch_1@.service)]

Parameters differences:

--- Exec[systemd daemon-reload for opensearch_1@.service (opensearch_1@.service)].orig
+++ Exec[systemd daemon-reload for opensearch_1@.service (opensearch_1@.service)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

Exec[renew certificate - discovery__cirrussearch2089_codfw_wmnet]

Parameters differences:

--- Exec[renew certificate - discovery__cirrussearch2089_codfw_wmnet].orig
+++ Exec[renew certificate - discovery__cirrussearch2089_codfw_wmnet]

+    environment => ['GODEBUG=x509ignoreCN=0']
+    unless      => /usr/bin/openssl x509 -in /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.pem -checkend 952200
+    command     => /usr/bin/cfssl sign -config /etc/cfssl/client-cfssl.conf -tls-remote-ca /etc/ssl/certs/wmf-ca-certificates.crt -mutual-tls-client-cert /etc/cfssl/mutual_tls_client_cert.pem -mutual-tls-client-key /var/lib/puppet/ssl/private_keys/cirrussearch2089.codfw.wmnet.pem -label discovery  /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.csr | /usr/bin/cfssljson -bare /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet

+    require     => Exec[Generate cert discovery__cirrussearch2089_codfw_wmnet]

Sysctl::Parameters[TCP Fast Open]

Parameters differences:

--- Sysctl::Parameters[TCP Fast Open].orig
+++ Sysctl::Parameters[TCP Fast Open]

+    values   => {'net.ipv4.tcp_fastopen': 3}
+    priority => 70
+    ensure   => present

Systemd::Service[nrpe2nodexp-opensearch_shard_size_check_9200]

Parameters differences:

--- Systemd::Service[nrpe2nodexp-opensearch_shard_size_check_9200].orig
+++ Systemd::Service[nrpe2nodexp-opensearch_shard_size_check_9200]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => absent
+    monitoring_critical      => False
+    require                  => Systemd::Unit[nrpe2nodexp-opensearch_shard_size_check_9200.service]
+    monitoring_enabled       => False
+    unit_type                => timer
+    restart                  => False

Systemd::Syslog[opensearch-disable-readahead-production-search-psi-codfw]

Parameters differences:

--- Systemd::Syslog[opensearch-disable-readahead-production-search-psi-codfw].orig
+++ Systemd::Syslog[opensearch-disable-readahead-production-search-psi-codfw]

+    programname_comparison => startswith
+    force_stop             => True
+    ensure                 => present
+    base_dir               => /var/log
+    readable_by            => all
+    log_filename           => syslog.log
+    owner                  => root
+    group                  => root

Class[Profile::Base::Production]

Parameters differences:

--- Class[Profile::Base::Production].orig
+++ Class[Profile::Base::Production]

@@
-    role_description => Host being setup by Data Platform SREs
+    role_description => OpenSearch cirrus

Monitoring::Service[elasticsearch-https-production-search-codfw]

Parameters differences:

--- Monitoring::Service[elasticsearch-https-production-search-codfw].orig
+++ Monitoring::Service[elasticsearch-https-production-search-codfw]

+    check_command  => check_ssl_on_port!cirrussearch2089.codfw.wmnet!9243
+    retry_interval => 1
+    notes_url      => https://wikitech.wikimedia.org/wiki/Search
+    migration_task => T384998
+    description    => Elasticsearch HTTPS for production-search-codfw
+    retries        => 3
+    critical       => False
+    freshness      => 36000
+    config_dir     => /etc/nagios
+    ensure         => present
+    check_interval => 1
+    host           => cirrussearch2089
+    passive        => False

File[/var/lib/nginx]

Parameters differences:

--- File[/var/lib/nginx].orig
+++ File[/var/lib/nginx]

+    group  => root
+    before => ['Service[nginx]']
+    owner  => root
+    ensure => directory

File[/srv/opensearch/production-search-codfw]

Parameters differences:

--- File[/srv/opensearch/production-search-codfw].orig
+++ File[/srv/opensearch/production-search-codfw]

+    mode   => 0755
+    group  => opensearch
+    owner  => opensearch
+    ensure => directory

Systemd::Service[wmf_auto_restart_prometheus-elasticsearch-exporter-9600]

Parameters differences:

--- Systemd::Service[wmf_auto_restart_prometheus-elasticsearch-exporter-9600].orig
+++ Systemd::Service[wmf_auto_restart_prometheus-elasticsearch-exporter-9600]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => present
+    monitoring_critical      => False
+    require                  => Systemd::Unit[wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service]
+    monitoring_enabled       => False
+    unit_type                => timer
+    restart                  => False

Exec[systemd daemon-reload for opensearch-disable-readahead.timer (opensearch-disable-readahead.timer)]

Parameters differences:

--- Exec[systemd daemon-reload for opensearch-disable-readahead.timer (opensearch-disable-readahead.timer)].orig
+++ Exec[systemd daemon-reload for opensearch-disable-readahead.timer (opensearch-disable-readahead.timer)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

File[/etc/rsyslog.d/40-wmf-auto-restart-prometheus-wmf-elasticsearch-exporter-9600.conf]

Parameters differences:

--- File[/etc/rsyslog.d/40-wmf-auto-restart-prometheus-wmf-elasticsearch-exporter-9600.conf].orig
+++ File[/etc/rsyslog.d/40-wmf-auto-restart-prometheus-wmf-elasticsearch-exporter-9600.conf]

+    mode   => 0444
+    notify => Service[rsyslog]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/rsyslog.d/40-wmf-auto-restart-prometheus-wmf-elasticsearch-exporter-9600.conf.orig
+++ /etc/rsyslog.d/40-wmf-auto-restart-prometheus-wmf-elasticsearch-exporter-9600.conf
@@ -0,0 +1,10 @@
+# rsyslog.conf(5) configuration file for services.
+# This file is managed by Puppet.
+if $programname startswith "wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600" then {
+    action(
+        type="omfile" file="/var/log/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600/syslog.log"
+        fileOwner="root" fileGroup="root"
+        fileCreateMode="0644"
+    )
+    & stop
+}

Apt::Repository[wikimedia-opensearch]

Parameters differences:

--- Apt::Repository[wikimedia-opensearch].orig
+++ Apt::Repository[wikimedia-opensearch]

+    uri                      => http://apt.wikimedia.org/wikimedia
+    allow_releaseinfo_change => False
+    trust_repo               => False
+    components               => thirdparty/opensearch1
+    source                   => True
+    bin                      => True
+    ensure                   => present
+    dist                     => bullseye-wikimedia
+    before                   => Class[Opensearch]

File[/usr/local/bin/opensearch_hot_threads_logger.py]

Parameters differences:

--- File[/usr/local/bin/opensearch_hot_threads_logger.py].orig
+++ File[/usr/local/bin/opensearch_hot_threads_logger.py]

+    mode   => 0555
+    group  => root
+    owner  => root
+    source => puppet:///modules/opensearch/opensearch_hot_threads_logger.py

Monitoring::Exported_nagios_service[cirrussearch2089 disk_space]

Parameters differences:

--- Monitoring::Exported_nagios_service[cirrussearch2089 disk_space].orig
+++ Monitoring::Exported_nagios_service[cirrussearch2089 disk_space]

@@
-    notifications_enabled => 0
+    notifications_enabled => 1
@@
-    servicegroups         => insetup_codfw
+    servicegroups         => elasticsearch_codfw
@@
-    contact_groups        => admins
+    contact_groups        => team-data-platform,team-discovery

File[/etc/nagios/nrpe.d/check_opensearch_shard_size_check_9200.cfg]

Parameters differences:

--- File[/etc/nagios/nrpe.d/check_opensearch_shard_size_check_9200.cfg].orig
+++ File[/etc/nagios/nrpe.d/check_opensearch_shard_size_check_9200.cfg]

+    mode    => 0444
+    require => Package[nagios-nrpe-server]
+    tag     => nrpe::check
+    notify  => Service[nagios-nrpe-server]
+    group   => root
+    owner   => root
+    ensure  => present

Content differences:

--- /etc/nagios/nrpe.d/check_opensearch_shard_size_check_9200.cfg.orig
+++ /etc/nagios/nrpe.d/check_opensearch_shard_size_check_9200.cfg
@@ -0,0 +1,2 @@
+# File generated by puppet. DO NOT edit by hand
+command[check_opensearch_shard_size_check_9200]=/usr/lib/nagios/plugins/check_elasticsearch_shard_size.py --url http://localhost:9200 --shard-size-warning 110 --shard-size-critical 140 --timeout 4

Systemd::Timer[opensearch-hot-threads-log]

Parameters differences:

--- Systemd::Timer[opensearch-hot-threads-log].orig
+++ Systemd::Timer[opensearch-hot-threads-log]

+    fixed_random_delay => False
+    unit_name          => opensearch-hot-threads-log.service
+    timer_intervals    => [{'start': 'OnCalendar', 'interval': '*-*-* *:00/5:00'}]
+    splay              => 0
+    accuracy           => 15sec
+    ensure             => present

File[/srv/opensearch]

Parameters differences:

--- File[/srv/opensearch].orig
+++ File[/srv/opensearch]

+    mode    => 0755
+    require => Package[opensearch]
+    group   => opensearch
+    owner   => opensearch
+    ensure  => directory

File[/etc/logrotate.d/opensearch-production-search-psi-codfw-gc-log-cleanup]

Parameters differences:

--- File[/etc/logrotate.d/opensearch-production-search-psi-codfw-gc-log-cleanup].orig
+++ File[/etc/logrotate.d/opensearch-production-search-psi-codfw-gc-log-cleanup]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/logrotate.d/opensearch-production-search-psi-codfw-gc-log-cleanup.orig
+++ /etc/logrotate.d/opensearch-production-search-psi-codfw-gc-log-cleanup
@@ -0,0 +1,12 @@
+# logrotate(8) config for opensearch-production-search-psi-codfw-gc-log-cleanup
+
+/var/log/opensearch-production-search-psi-codfw-gc-log-cleanup/*.log {
+    daily
+    copytruncate
+    missingok
+    compress
+    delaycompress
+    notifempty
+    rotate 15
+    size 256M
+}

Logrotate::Conf[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]

Parameters differences:

--- Logrotate::Conf[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600].orig
+++ Logrotate::Conf[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]

+    ensure => present

Augeas[ipip60_add_up]

Parameters differences:

--- Augeas[ipip60_add_up].orig
+++ Augeas[ipip60_add_up]

+    onlyif  => match up[. = 'ip link add name ipip60 type ip6tnl external'] size == 0
+    context => /files/etc/network/interfaces/*[. = 'ipip60' and ./family = 'inet6']
+    incl    => /etc/network/interfaces
+    require => Interface::Manual[ipip_ipv6]
+    lens    => Interfaces.lns
+    changes => set up[last()+1] 'ip link add name ipip60 type ip6tnl external'

Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9200.service (nrpe2nodexp-opensearch_shards_9200.service)]

Parameters differences:

--- Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9200.service (nrpe2nodexp-opensearch_shards_9200.service)].orig
+++ Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9200.service (nrpe2nodexp-opensearch_shards_9200.service)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

File[/etc/nagios/nrpe.d/check_disk_space.cfg]

Content differences:

--- /etc/nagios/nrpe.d/check_disk_space.cfg.orig
+++ /etc/nagios/nrpe.d/check_disk_space.cfg
@@ -1,2 +1,2 @@
 # File generated by puppet. DO NOT edit by hand
-command[check_disk_space]=/usr/lib/nagios/plugins/check_disk -w 6% -c 3% -W 6% -K 3% -l -e -A -i "/srv/sd[a-b][1-3]" -i "/srv/nvme[0-9]n[0-9]p[0-9]" --exclude-type=fuse --exclude-type=fuse.fuse_dfs --exclude-type=tracefs
+command[check_disk_space]=/usr/lib/nagios/plugins/check_disk -w 15% -c 10% -W 6% -K 3% -l -e -A -i "/srv/sd[a-b][1-3]" --exclude-type=tracefs

Monitoring::Exported_nagios_service[cirrussearch2089 check_tcp-mss-clamper_status]

Parameters differences:

--- Monitoring::Exported_nagios_service[cirrussearch2089 check_tcp-mss-clamper_status].orig
+++ Monitoring::Exported_nagios_service[cirrussearch2089 check_tcp-mss-clamper_status]

+    passive_checks_enabled => 1
+    is_volatile            => 0
+    contact_groups         => admins
+    notification_options   => c,r,f
+    service_description    => Check unit status of tcp-mss-clamper
+    max_check_attempts     => 2
+    active_checks_enabled  => 1
+    notification_interval  => 0
+    check_command          => nrpe_check!check_check_tcp-mss-clamper_status!10
+    retry_interval         => 1
+    check_freshness        => 0
+    notes_url              => https://wikitech.wikimedia.org/wiki/LVS#IPIP_encapsulation_experiments
+    notifications_enabled  => 1
+    check_period           => 24x7
+    ensure                 => absent
+    check_interval         => 10
+    notification_period    => 24x7
+    host_name              => cirrussearch2089
+    servicegroups          => elasticsearch_codfw

Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_shard_size_check_9200]

Parameters differences:

--- Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_shard_size_check_9200].orig
+++ Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_shard_size_check_9200]

+    passive_checks_enabled => 1
+    is_volatile            => 0
+    contact_groups         => admins,team-discovery
+    notification_options   => c,r,f
+    service_description    => OpenSearch shard size check - 9200
+    max_check_attempts     => 3
+    active_checks_enabled  => 1
+    notification_interval  => 0
+    check_command          => nrpe_check!check_opensearch_shard_size_check_9200!10
+    retry_interval         => 180
+    check_freshness        => 0
+    notes_url              => https://wikitech.wikimedia.org/wiki/Search#If_it_has_been_indexed
+    notifications_enabled  => 1
+    check_period           => 24x7
+    ensure                 => present
+    check_interval         => 1440
+    notification_period    => 24x7
+    host_name              => cirrussearch2089
+    servicegroups          => elasticsearch_codfw

Nrpe::Monitor_service[opensearch_unassigned_shard_check_9200]

Parameters differences:

--- Nrpe::Monitor_service[opensearch_unassigned_shard_check_9200].orig
+++ Nrpe::Monitor_service[opensearch_unassigned_shard_check_9200]

+    timeout                     => 10
+    nrpe2nodexp_parse_perf_data => False
+    retry_interval              => 120
+    description                 => OpenSearch unassigned shard check - 9200
+    notes_url                   => https://wikitech.wikimedia.org/wiki/Search#Administration
+    critical                    => False
+    retries                     => 1
+    migration_task              => T371083
+    ensure                      => present
+    enable_nrpe2nodexp          => False
+    enable_icinga_check         => True
+    check_interval              => 720
+    contact_group               => admins,team-discovery
+    nrpe_command                => /usr/lib/nagios/plugins/check_elasticsearch_unassigned_shards.py --url http://localhost:9200 --timeout 4

File_line[auto_restart_file_presence_prometheus-elasticsearch-exporter-9200]

Parameters differences:

--- File_line[auto_restart_file_presence_prometheus-elasticsearch-exporter-9200].orig
+++ File_line[auto_restart_file_presence_prometheus-elasticsearch-exporter-9200]

+    require => File[/etc/debdeploy-client/autorestarts.conf]
+    path    => /etc/debdeploy-client/autorestarts.conf
+    ensure  => present
+    line    => prometheus-elasticsearch-exporter-9200

File[/etc/sudoers.d/nrpe-check_opensearch_shards_9600]

Parameters differences:

--- File[/etc/sudoers.d/nrpe-check_opensearch_shards_9600].orig
+++ File[/etc/sudoers.d/nrpe-check_opensearch_shards_9600]

+    group   => root
+    require => Package[nagios-nrpe-server]
+    owner   => root
+    ensure  => absent

Systemd::Timer[wmf_auto_restart_prometheus-elasticsearch-exporter-9600]

Parameters differences:

--- Systemd::Timer[wmf_auto_restart_prometheus-elasticsearch-exporter-9600].orig
+++ Systemd::Timer[wmf_auto_restart_prometheus-elasticsearch-exporter-9600]

+    fixed_random_delay => False
+    unit_name          => wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service
+    timer_intervals    => [{'start': 'OnCalendar', 'interval': 'Mon,Tue,Wed,Thu,Fri *-*-* 5:3:00'}]
+    splay              => 0
+    accuracy           => 15sec
+    ensure             => present

Systemd::Unit[prometheus-wmf-elasticsearch-exporter-9200]

Parameters differences:

--- Systemd::Unit[prometheus-wmf-elasticsearch-exporter-9200].orig
+++ Systemd::Unit[prometheus-wmf-elasticsearch-exporter-9200]

+    restart           => True
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => prometheus-wmf-elasticsearch-exporter-9200
+    ensure            => present

Service[nrpe2nodexp-opensearch_shard_size_check_9600.timer]

Parameters differences:

--- Service[nrpe2nodexp-opensearch_shard_size_check_9600.timer].orig
+++ Service[nrpe2nodexp-opensearch_shard_size_check_9600.timer]

+    before   => ['Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9600.timer (nrpe2nodexp-opensearch_shard_size_check_9600.timer)]']
+    provider => systemd
+    enable   => False
+    ensure   => stopped

File_line[auto_restart_file_presence_prometheus-elasticsearch-exporter-9600]

Parameters differences:

--- File_line[auto_restart_file_presence_prometheus-elasticsearch-exporter-9600].orig
+++ File_line[auto_restart_file_presence_prometheus-elasticsearch-exporter-9600]

+    require => File[/etc/debdeploy-client/autorestarts.conf]
+    path    => /etc/debdeploy-client/autorestarts.conf
+    ensure  => present
+    line    => prometheus-elasticsearch-exporter-9600

Exec[systemd daemon-reload for prometheus-elasticsearch-exporter-9200.service (prometheus-elasticsearch-exporter-9200)]

Parameters differences:

--- Exec[systemd daemon-reload for prometheus-elasticsearch-exporter-9200.service (prometheus-elasticsearch-exporter-9200)].orig
+++ Exec[systemd daemon-reload for prometheus-elasticsearch-exporter-9200.service (prometheus-elasticsearch-exporter-9200)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload
+    before      => ['Service[prometheus-elasticsearch-exporter-9200]']

Class[Lvs::Realserver]

Parameters differences:

--- Class[Lvs::Realserver].orig
+++ Class[Lvs::Realserver]

+    realserver_ips => ['10.2.1.30']

File[/etc/opensearch/production-search-psi-codfw/log4j2.properties]

Parameters differences:

--- File[/etc/opensearch/production-search-psi-codfw/log4j2.properties].orig
+++ File[/etc/opensearch/production-search-psi-codfw/log4j2.properties]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => file

Content differences:

--- /etc/opensearch/production-search-psi-codfw/log4j2.properties.orig
+++ /etc/opensearch/production-search-psi-codfw/log4j2.properties
@@ -0,0 +1,201 @@
+status = error
+
+# log action execution errors for easier debugging
+logger.action.name = org.opensearch.action
+logger.action.level = debug
+
+appender.console.type = Console
+appender.console.name = console
+appender.console.layout.type = PatternLayout
+appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %m%n
+
+######## Server JSON ############################
+appender.rolling.type = RollingFile
+appender.rolling.name = rolling
+appender.rolling.fileName = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}_server.json
+appender.rolling.layout.type = OpenSearchJsonLayout
+appender.rolling.layout.type_name = server
+
+appender.rolling.filePattern = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}-%d{yyyy-MM-dd}-%i.json.gz
+appender.rolling.policies.type = Policies
+appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
+appender.rolling.policies.time.interval = 1
+appender.rolling.policies.time.modulate = true
+appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
+appender.rolling.policies.size.size = 128MB
+appender.rolling.strategy.type = DefaultRolloverStrategy
+appender.rolling.strategy.fileIndex = nomax
+appender.rolling.strategy.action.type = Delete
+appender.rolling.strategy.action.basepath = ${sys:opensearch.logs.base_path}
+appender.rolling.strategy.action.condition.type = IfFileName
+appender.rolling.strategy.action.condition.glob = ${sys:opensearch.logs.cluster_name}-*
+appender.rolling.strategy.action.condition.nested_condition.type = IfAccumulatedFileSize
+appender.rolling.strategy.action.condition.nested_condition.exceeds = 2GB
+################################################
+######## Server -  old style pattern ###########
+appender.rolling_old.type = RollingFile
+appender.rolling_old.name = rolling_old
+appender.rolling_old.fileName = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}.log
+appender.rolling_old.layout.type = PatternLayout
+appender.rolling_old.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %m%n
+
+appender.rolling_old.filePattern = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}-%d{yyyy-MM-dd}-%i.log.gz
+appender.rolling_old.policies.type = Policies
+appender.rolling_old.policies.time.type = TimeBasedTriggeringPolicy
+appender.rolling_old.policies.time.interval = 1
+appender.rolling_old.policies.time.modulate = true
+appender.rolling_old.policies.size.type = SizeBasedTriggeringPolicy
+appender.rolling_old.policies.size.size = 128MB
+appender.rolling_old.strategy.type = DefaultRolloverStrategy
+appender.rolling_old.strategy.fileIndex = nomax
+appender.rolling_old.strategy.action.type = Delete
+appender.rolling_old.strategy.action.basepath = ${sys:opensearch.logs.base_path}
+appender.rolling_old.strategy.action.condition.type = IfFileName
+appender.rolling_old.strategy.action.condition.glob = ${sys:opensearch.logs.cluster_name}-*
+appender.rolling_old.strategy.action.condition.nested_condition.type = IfAccumulatedFileSize
+appender.rolling_old.strategy.action.condition.nested_condition.exceeds = 2GB
+################################################
+
+rootLogger.level = info
+rootLogger.appenderRef.console.ref = console
+rootLogger.appenderRef.rolling.ref = rolling
+rootLogger.appenderRef.rolling_old.ref = rolling_old
+
+######## Deprecation JSON #######################
+appender.deprecation_rolling.type = RollingFile
+appender.deprecation_rolling.name = deprecation_rolling
+appender.deprecation_rolling.fileName = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}_deprecation.json
+appender.deprecation_rolling.layout.type = OpenSearchJsonLayout
+appender.deprecation_rolling.layout.type_name = deprecation
+appender.deprecation_rolling.layout.opensearchmessagefields=x-opaque-id
+
+appender.deprecation_rolling.filePattern = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}_deprecation-%i.json.gz
+appender.deprecation_rolling.policies.type = Policies
+appender.deprecation_rolling.policies.size.type = SizeBasedTriggeringPolicy
+appender.deprecation_rolling.policies.size.size = 1GB
+appender.deprecation_rolling.strategy.type = DefaultRolloverStrategy
+appender.deprecation_rolling.strategy.max = 4
+#################################################
+######## Deprecation -  old style pattern #######
+appender.deprecation_rolling_old.type = RollingFile
+appender.deprecation_rolling_old.name = deprecation_rolling_old
+appender.deprecation_rolling_old.fileName = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}_deprecation.log
+appender.deprecation_rolling_old.layout.type = PatternLayout
+appender.deprecation_rolling_old.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %m%n
+
+appender.deprecation_rolling_old.filePattern = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}\
+  _deprecation-%i.log.gz
+appender.deprecation_rolling_old.policies.type = Policies
+appender.deprecation_rolling_old.policies.size.type = SizeBasedTriggeringPolicy
+appender.deprecation_rolling_old.policies.size.size = 1GB
+appender.deprecation_rolling_old.strategy.type = DefaultRolloverStrategy
+appender.deprecation_rolling_old.strategy.max = 4
+#################################################
+logger.deprecation.name = org.opensearch.deprecation
+logger.deprecation.level = warn
+logger.deprecation.appenderRef.deprecation_rolling.ref = deprecation_rolling
+logger.deprecation.appenderRef.deprecation_rolling_old.ref = deprecation_rolling_old
+logger.deprecation.additivity = false
+
+######## Search slowlog JSON ####################
+appender.index_search_slowlog_rolling.type = RollingFile
+appender.index_search_slowlog_rolling.name = index_search_slowlog_rolling
+appender.index_search_slowlog_rolling.fileName = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs\
+  .cluster_name}_index_search_slowlog.json
+appender.index_search_slowlog_rolling.layout.type = OpenSearchJsonLayout
+appender.index_search_slowlog_rolling.layout.type_name = index_search_slowlog
+appender.index_search_slowlog_rolling.layout.opensearchmessagefields=message,took,took_millis,total_hits,types,stats,search_type,total_shards,source,id
+
+appender.index_search_slowlog_rolling.filePattern = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs\
+  .cluster_name}_index_search_slowlog-%i.json.gz
+appender.index_search_slowlog_rolling.policies.type = Policies
+appender.index_search_slowlog_rolling.policies.size.type = SizeBasedTriggeringPolicy
+appender.index_search_slowlog_rolling.policies.size.size = 1GB
+appender.index_search_slowlog_rolling.strategy.type = DefaultRolloverStrategy
+appender.index_search_slowlog_rolling.strategy.max = 4
+#################################################
+######## Search slowlog -  old style pattern ####
+appender.index_search_slowlog_rolling_old.type = RollingFile
+appender.index_search_slowlog_rolling_old.name = index_search_slowlog_rolling_old
+appender.index_search_slowlog_rolling_old.fileName = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}\
+  _index_search_slowlog.log
+appender.index_search_slowlog_rolling_old.layout.type = PatternLayout
+appender.index_search_slowlog_rolling_old.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %m%n
+
+appender.index_search_slowlog_rolling_old.filePattern = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}\
+  _index_search_slowlog-%i.log.gz
+appender.index_search_slowlog_rolling_old.policies.type = Policies
+appender.index_search_slowlog_rolling_old.policies.size.type = SizeBasedTriggeringPolicy
+appender.index_search_slowlog_rolling_old.policies.size.size = 1GB
+appender.index_search_slowlog_rolling_old.strategy.type = DefaultRolloverStrategy
+appender.index_search_slowlog_rolling_old.strategy.max = 4
+#################################################
+logger.index_search_slowlog_rolling.name = index.search.slowlog
+logger.index_search_slowlog_rolling.level = trace
+logger.index_search_slowlog_rolling.appenderRef.index_search_slowlog_rolling.ref = index_search_slowlog_rolling
+logger.index_search_slowlog_rolling.appenderRef.index_search_slowlog_rolling_old.ref = index_search_slowlog_rolling_old
+logger.index_search_slowlog_rolling.additivity = false
+
+######## Indexing slowlog JSON ##################
+appender.index_indexing_slowlog_rolling.type = RollingFile
+appender.index_indexing_slowlog_rolling.name = index_indexing_slowlog_rolling
+appender.index_indexing_slowlog_rolling.fileName = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}\
+  _index_indexing_slowlog.json
+appender.index_indexing_slowlog_rolling.layout.type = OpenSearchJsonLayout
+appender.index_indexing_slowlog_rolling.layout.type_name = index_indexing_slowlog
+appender.index_indexing_slowlog_rolling.layout.opensearchmessagefields=message,took,took_millis,doc_type,id,routing,source
+
+appender.index_indexing_slowlog_rolling.filePattern = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}\
+  _index_indexing_slowlog-%i.json.gz
+appender.index_indexing_slowlog_rolling.policies.type = Policies
+appender.index_indexing_slowlog_rolling.policies.size.type = SizeBasedTriggeringPolicy
+appender.index_indexing_slowlog_rolling.policies.size.size = 1GB
+appender.index_indexing_slowlog_rolling.strategy.type = DefaultRolloverStrategy
+appender.index_indexing_slowlog_rolling.strategy.max = 4
+#################################################
+######## Indexing slowlog -  old style pattern ##
+appender.index_indexing_slowlog_rolling_old.type = RollingFile
+appender.index_indexing_slowlog_rolling_old.name = index_indexing_slowlog_rolling_old
+appender.index_indexing_slowlog_rolling_old.fileName = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}\
+  _index_indexing_slowlog.log
+appender.index_indexing_slowlog_rolling_old.layout.type = PatternLayout
+appender.index_indexing_slowlog_rolling_old.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %m%n
+
+appender.index_indexing_slowlog_rolling_old.filePattern = ${sys:opensearch.logs.base_path}${sys:file.separator}${sys:opensearch.logs.cluster_name}\
+  _index_indexing_slowlog-%i.log.gz
+appender.index_indexing_slowlog_rolling_old.policies.type = Policies
+appender.index_indexing_slowlog_rolling_old.policies.size.type = SizeBasedTriggeringPolicy
+appender.index_indexing_slowlog_rolling_old.policies.size.size = 1GB
+appender.index_indexing_slowlog_rolling_old.strategy.type = DefaultRolloverStrategy
+appender.index_indexing_slowlog_rolling_old.strategy.max = 4
+#################################################
+
+logger.index_indexing_slowlog.name = index.indexing.slowlog.index
+logger.index_indexing_slowlog.level = trace
+logger.index_indexing_slowlog.appenderRef.index_indexing_slowlog_rolling.ref = index_indexing_slowlog_rolling
+logger.index_indexing_slowlog.appenderRef.index_indexing_slowlog_rolling_old.ref = index_indexing_slowlog_rolling_old
+logger.index_indexing_slowlog.additivity = false
+
+
+### Custom settings
+monitorInterval=600
+
+# Add references to Logstash appender for interesting loggers
+
+rootLogger.appenderRef.ship_to_logstash.ref = ship_to_logstash
+logger.index_indexing_slowlog.appenderRef.ship_to_logstash.ref = ship_to_logstash
+logger.index_search_slowlog_rolling.appenderRef.ship_to_logstash.ref = ship_to_logstash
+logger.index_search_slowlog_rolling.appenderRef.ship_to_logstash.ref = ship_to_logstash
+
+# Define Logstash appender
+appender.ship_to_logstash.name=ship_to_logstash
+appender.ship_to_logstash.type=syslog
+appender.ship_to_logstash.protocol=udp
+appender.ship_to_logstash.facility=LOCAL0
+appender.ship_to_logstash.layout.type=OpenSearchJsonLayout
+appender.ship_to_logstash.layout.compact=true
+appender.ship_to_logstash.layout.complete=true
+appender.ship_to_logstash.host=localhost
+appender.ship_to_logstash.port=11514
+appender.ship_to_logstash.layout.type_name=opensearch
+

Systemd::Unit[opensearch-disable-readahead-production-search-psi-codfw.service]

Parameters differences:

--- Systemd::Unit[opensearch-disable-readahead-production-search-psi-codfw.service].orig
+++ Systemd::Unit[opensearch-disable-readahead-production-search-psi-codfw.service]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => opensearch-disable-readahead-production-search-psi-codfw.service
+    ensure            => present

Systemd::Timer::Job[nrpe2nodexp-opensearch_shards_9200]

Parameters differences:

--- Systemd::Timer::Job[nrpe2nodexp-opensearch_shards_9200].orig
+++ Systemd::Timer::Job[nrpe2nodexp-opensearch_shards_9200]

+    description               => execution of nrpe2nodexp for the check_opensearch_shards_9200 command.
+    syslog_match_startswith   => True
+    monitoring_contact_groups => admins
+    logging_enabled           => False
+    success_exit_status       => []
+    fixed_random_delay        => False
+    private_tmp               => False
+    logfile_basedir           => /var/log
+    group                     => prometheus-node-exporter
+    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+    syslog_identifier         => nrpe2nodexp-opensearch_shards_9200
+    send_mail_only_on_error   => True
+    ignore_errors             => True
+    logfile_group             => root
+    interval                  => [{'start': 'OnUnitInactiveSec', 'interval': '1min'}]
+    send_mail                 => False
+    logfile_name              => syslog.log
+    user                      => nagios
+    ensure                    => absent
+    send_mail_to              => root@cirrussearch2089.codfw.wmnet
+    logfile_perms             => all
+    syslog_force_stop         => True
+    environment               => {}
+    command                   => /usr/local/bin/nrpe2nodexp --alert-rule-hash "8cb398d9f4c08b77d96bfdab16485563" --timeout 10 --check-command "check_opensearch_shards_9200"
+    monitoring_enabled        => False

Systemd::Unit[opensearch-disable-readahead.service]

Parameters differences:

--- Systemd::Unit[opensearch-disable-readahead.service].orig
+++ Systemd::Unit[opensearch-disable-readahead.service]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => opensearch-disable-readahead.service
+    ensure            => absent

Service[opensearch-disable-readahead-production-search-codfw.timer]

Parameters differences:

--- Service[opensearch-disable-readahead-production-search-codfw.timer].orig
+++ Service[opensearch-disable-readahead-production-search-codfw.timer]

+    provider => systemd
+    enable   => True
+    ensure   => running

Service[nginx]

Parameters differences:

--- Service[nginx].orig
+++ Service[nginx]

+    hasrestart => True
+    require    => Package[nginx-light]
+    enable     => True
+    ensure     => running

File[/etc/nginx/sites-enabled]

Parameters differences:

--- File[/etc/nginx/sites-enabled].orig
+++ File[/etc/nginx/sites-enabled]

+    purge   => True
+    force   => True
+    before  => ['Service[nginx]']
+    owner   => root
+    recurse => True
+    ensure  => directory
+    group   => root

File[/etc/sudoers.d/nrpe-check_opensearch_unassigned_shard_check_9600]

Parameters differences:

--- File[/etc/sudoers.d/nrpe-check_opensearch_unassigned_shard_check_9600].orig
+++ File[/etc/sudoers.d/nrpe-check_opensearch_unassigned_shard_check_9600]

+    group   => root
+    require => Package[nagios-nrpe-server]
+    owner   => root
+    ensure  => absent

Ferm::Service[opensearch-https-9243]

Parameters differences:

--- Ferm::Service[opensearch-https-9243].orig
+++ Ferm::Service[opensearch-https-9243]

+    proto   => tcp
+    port    => 9243
+    prio    => 10
+    desc    => 
+    notrack => False
+    srange  => $DOMAIN_NETWORKS
+    ensure  => present

Systemd::Unit[opensearch-disable-readahead-production-search-psi-codfw.timer]

Parameters differences:

--- Systemd::Unit[opensearch-disable-readahead-production-search-psi-codfw.timer].orig
+++ Systemd::Unit[opensearch-disable-readahead-production-search-psi-codfw.timer]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => opensearch-disable-readahead-production-search-psi-codfw.timer
+    ensure            => present

File[/lib/systemd/system/nrpe2nodexp-opensearch_shards_9600.timer]

Parameters differences:

--- File[/lib/systemd/system/nrpe2nodexp-opensearch_shards_9600.timer].orig
+++ File[/lib/systemd/system/nrpe2nodexp-opensearch_shards_9600.timer]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9600.timer (nrpe2nodexp-opensearch_shards_9600.timer)]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /lib/systemd/system/nrpe2nodexp-opensearch_shards_9600.timer.orig
+++ /lib/systemd/system/nrpe2nodexp-opensearch_shards_9600.timer
@@ -0,0 +1,13 @@
+[Unit]
+Description=Periodic execution of nrpe2nodexp-opensearch_shards_9600.service
+
+[Timer]
+Unit=nrpe2nodexp-opensearch_shards_9600.service
+# Accuracy sets the maximum time interval around the execution time we want to allow
+AccuracySec=15sec
+OnUnitInactiveSec=1min
+OnActiveSec=1s
+RandomizedDelaySec=0
+
+[Install]
+WantedBy=multi-user.target

File[/etc/opensearch/production-search-codfw/opensearch.keystore]

Parameters differences:

--- File[/etc/opensearch/production-search-codfw/opensearch.keystore].orig
+++ File[/etc/opensearch/production-search-codfw/opensearch.keystore]

+    mode   => 0640
+    group  => opensearch
+    owner  => root
+    ensure => present

File[/usr/local/bin/safe-service-restart]

Parameters differences:

--- File[/usr/local/bin/safe-service-restart].orig
+++ File[/usr/local/bin/safe-service-restart]

+    mode   => 0555
+    source => puppet:///modules/conftool/safe-service-restart.py
+    group  => root
+    owner  => root
+    ensure => present

Class[Profile::Base]

Parameters differences:

--- Class[Profile::Base].orig
+++ Class[Profile::Base]

@@
-    enable_rp_filter => True
+    enable_rp_filter => False
@@
-    cluster          => insetup
+    cluster          => elasticsearch

Systemd::Timer::Job[nrpe2nodexp-opensearch_shard_size_check_9200]

Parameters differences:

--- Systemd::Timer::Job[nrpe2nodexp-opensearch_shard_size_check_9200].orig
+++ Systemd::Timer::Job[nrpe2nodexp-opensearch_shard_size_check_9200]

+    description               => execution of nrpe2nodexp for the check_opensearch_shard_size_check_9200 command.
+    syslog_match_startswith   => True
+    monitoring_contact_groups => admins
+    logging_enabled           => False
+    success_exit_status       => []
+    fixed_random_delay        => False
+    private_tmp               => False
+    logfile_basedir           => /var/log
+    group                     => prometheus-node-exporter
+    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+    syslog_identifier         => nrpe2nodexp-opensearch_shard_size_check_9200
+    send_mail_only_on_error   => True
+    ignore_errors             => True
+    logfile_group             => root
+    interval                  => [{'start': 'OnUnitInactiveSec', 'interval': '1440min'}]
+    send_mail                 => False
+    logfile_name              => syslog.log
+    user                      => nagios
+    ensure                    => absent
+    send_mail_to              => root@cirrussearch2089.codfw.wmnet
+    logfile_perms             => all
+    syslog_force_stop         => True
+    environment               => {}
+    command                   => /usr/local/bin/nrpe2nodexp --alert-rule-hash "ce424a305cb5464806cbf788bde4430a" --timeout 10 --check-command "check_opensearch_shard_size_check_9200"
+    monitoring_enabled        => False

Monitoring::Service[opensearch_shards_9200]

Parameters differences:

--- Monitoring::Service[opensearch_shards_9200].orig
+++ Monitoring::Service[opensearch_shards_9200]

+    check_command  => nrpe_check!check_opensearch_shards_9200!10
+    retry_interval => 1
+    description    => OpenSearch health check for shards on 9200
+    critical       => False
+    notes_url      => https://wikitech.wikimedia.org/wiki/Search#Administration
+    retries        => 3
+    config_dir     => /etc/nagios
+    freshness      => 36000
+    migration_task => T321808
+    ensure         => present
+    check_interval => 1
+    contact_group  => admins,team-discovery
+    host           => cirrussearch2089
+    passive        => False

Exec[Generate cert discovery__cirrussearch2089_codfw_wmnet refresh]

Parameters differences:

--- Exec[Generate cert discovery__cirrussearch2089_codfw_wmnet refresh].orig
+++ Exec[Generate cert discovery__cirrussearch2089_codfw_wmnet refresh]

+    environment => ['GODEBUG=x509ignoreCN=0']
+    subscribe   => File[/etc/cfssl/csr/discovery__cirrussearch2089_codfw_wmnet.csr]
+    command     => /usr/bin/cfssl gencert -config /etc/cfssl/client-cfssl.conf -tls-remote-ca /etc/ssl/certs/wmf-ca-certificates.crt -mutual-tls-client-cert /etc/cfssl/mutual_tls_client_cert.pem -mutual-tls-client-key /var/lib/puppet/ssl/private_keys/cirrussearch2089.codfw.wmnet.pem -label discovery  /etc/cfssl/csr/discovery__cirrussearch2089_codfw_wmnet.csr | /usr/bin/cfssljson -bare /etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet

+    refreshonly => True

Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9600.timer (nrpe2nodexp-opensearch_shards_9600.timer)]

Parameters differences:

--- Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9600.timer (nrpe2nodexp-opensearch_shards_9600.timer)].orig
+++ Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shards_9600.timer (nrpe2nodexp-opensearch_shards_9600.timer)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

Group[elasticsearch-roots]

Parameters differences:

--- Group[elasticsearch-roots].orig
+++ Group[elasticsearch-roots]

+    allowdupe => False
+    ensure    => present
+    gid       => 709

Systemd::Timer[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]

Parameters differences:

--- Systemd::Timer[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600].orig
+++ Systemd::Timer[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]

+    fixed_random_delay => False
+    unit_name          => wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.service
+    timer_intervals    => [{'start': 'OnCalendar', 'interval': 'Mon,Tue,Wed,Thu,Fri *-*-* 9:55:00'}]
+    splay              => 0
+    accuracy           => 15sec
+    ensure             => present

Service[nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer]

Parameters differences:

--- Service[nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer].orig
+++ Service[nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer]

+    before   => ['Exec[systemd daemon-reload for nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer (nrpe2nodexp-opensearch_unassigned_shard_check_9600.timer)]']
+    provider => systemd
+    enable   => False
+    ensure   => stopped

File[/usr/local/bin/logstash-config-test]

Parameters differences:

--- File[/usr/local/bin/logstash-config-test].orig
+++ File[/usr/local/bin/logstash-config-test]

+    mode    => 0555
+    require => Package[logstash]
+    source  => puppet:///modules/logstash/logstash-config-test
+    group   => root
+    owner   => root

Systemd::Syslog[wmf_auto_restart_prometheus-elasticsearch-exporter-9200]

Parameters differences:

--- Systemd::Syslog[wmf_auto_restart_prometheus-elasticsearch-exporter-9200].orig
+++ Systemd::Syslog[wmf_auto_restart_prometheus-elasticsearch-exporter-9200]

+    programname_comparison => startswith
+    force_stop             => True
+    ensure                 => present
+    base_dir               => /var/log
+    readable_by            => all
+    log_filename           => syslog.log
+    owner                  => root
+    group                  => root

Exec[systemd daemon-reload for prometheus_ferm_mss.timer (prometheus_ferm_mss.timer)]

Parameters differences:

--- Exec[systemd daemon-reload for prometheus_ferm_mss.timer (prometheus_ferm_mss.timer)].orig
+++ Exec[systemd daemon-reload for prometheus_ferm_mss.timer (prometheus_ferm_mss.timer)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload
+    before      => ['Service[prometheus_ferm_mss.timer]']

File[/etc/default/wikimedia-lvs-realserver]

Parameters differences:

--- File[/etc/default/wikimedia-lvs-realserver].orig
+++ File[/etc/default/wikimedia-lvs-realserver]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/default/wikimedia-lvs-realserver.orig
+++ /etc/default/wikimedia-lvs-realserver
@@ -0,0 +1,10 @@
+# This file is managed by puppet!
+
+
+
+# Location of the sysctl file containing LVS ARP settings
+SYSCTLFILE=/usr/share/wikimedia-lvs-realserver/sysctl.conf
+
+# LVS service IPs to be bound to the loopback interface,
+# separate using spaces
+LVS_SERVICE_IPS="10.2.1.30"

Systemd::Unit[nrpe2nodexp-opensearch_shards_9200.service]

Parameters differences:

--- Systemd::Unit[nrpe2nodexp-opensearch_shards_9200.service].orig
+++ Systemd::Unit[nrpe2nodexp-opensearch_shards_9200.service]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => nrpe2nodexp-opensearch_shards_9200.service
+    ensure            => absent

Logstash::Input::Gelf[gelf_relay]

Parameters differences:

--- Logstash::Input::Gelf[gelf_relay].orig
+++ Logstash::Input::Gelf[gelf_relay]

+    host      => localhost
+    port      => 12201
+    plugin_id => input/gelf/12201
+    tags      => ['input-gelf-relay']
+    priority  => 10
+    ensure    => present

File[/etc/logstash/logstash.yml]

Parameters differences:

--- File[/etc/logstash/logstash.yml].orig
+++ File[/etc/logstash/logstash.yml]

+    mode    => 0444
+    require => Package[logstash]
+    notify  => Service[logstash]
+    group   => root
+    owner   => root

Content differences:

--- /etc/logstash/logstash.yml.orig
+++ /etc/logstash/logstash.yml
@@ -0,0 +1,11 @@
+---
+path.data: "/var/lib/logstash"
+path.config: "/etc/logstash/conf.d"
+path.logs: "/var/log/logstash"
+pipeline.workers: 2
+pipeline.batch.size: 125
+pipeline.batch.delay: 50
+log.format: json
+dead_letter_queue.enable: false
+dead_letter_queue.max_bytes: 1024mb
+http.port: '9675'

Exec[systemd daemon-reload for prometheus-wmf-elasticsearch-exporter-9200.service (prometheus-wmf-elasticsearch-exporter-9200)]

Parameters differences:

--- Exec[systemd daemon-reload for prometheus-wmf-elasticsearch-exporter-9200.service (prometheus-wmf-elasticsearch-exporter-9200)].orig
+++ Exec[systemd daemon-reload for prometheus-wmf-elasticsearch-exporter-9200.service (prometheus-wmf-elasticsearch-exporter-9200)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload
+    notify      => ['Service[prometheus-wmf-elasticsearch-exporter-9200]']

Profile::Prometheus::Wmf_elasticsearch_exporter[cirrussearch2089:9200]

Parameters differences:

--- Profile::Prometheus::Wmf_elasticsearch_exporter[cirrussearch2089:9200].orig
+++ Profile::Prometheus::Wmf_elasticsearch_exporter[cirrussearch2089:9200]

+    indices_to_monitor => ['commonswiki_file', 'wikidatawiki_content', 'enwiki_general', 'enwiki_content', 'viwiki_general', 'commonswiki_general', 'cebwiki_content', 'metawiki_general', 'dewiki_content', 'frwiki_content']
+    prometheus_port    => 9120
+    elasticsearch_port => 9200

Nrpe::Monitor_service[raid_broadcom_raid]

Parameters differences:

--- Nrpe::Monitor_service[raid_broadcom_raid].orig
+++ Nrpe::Monitor_service[raid_broadcom_raid]

@@
-    contact_group => admins
+    contact_group => team-data-platform,team-discovery

Alternatives::Java[11]

Parameters differences:

--- Alternatives::Java[11].orig
+++ Alternatives::Java[11]

+    require => Java::Package[openjdk-jdk-11]

Monitoring::Service[opensearch_shards_9600]

Parameters differences:

--- Monitoring::Service[opensearch_shards_9600].orig
+++ Monitoring::Service[opensearch_shards_9600]

+    check_command  => nrpe_check!check_opensearch_shards_9600!10
+    retry_interval => 1
+    description    => OpenSearch health check for shards on 9600
+    critical       => False
+    notes_url      => https://wikitech.wikimedia.org/wiki/Search#Administration
+    retries        => 3
+    config_dir     => /etc/nagios
+    freshness      => 36000
+    migration_task => T321808
+    ensure         => present
+    check_interval => 1
+    contact_group  => admins,team-discovery
+    host           => cirrussearch2089
+    passive        => False

File[/lib/systemd/system/opensearch-disable-readahead.timer]

Parameters differences:

--- File[/lib/systemd/system/opensearch-disable-readahead.timer].orig
+++ File[/lib/systemd/system/opensearch-disable-readahead.timer]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for opensearch-disable-readahead.timer (opensearch-disable-readahead.timer)]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /lib/systemd/system/opensearch-disable-readahead.timer.orig
+++ /lib/systemd/system/opensearch-disable-readahead.timer
@@ -0,0 +1,13 @@
+[Unit]
+Description=Periodic execution of opensearch-disable-readahead.service
+
+[Timer]
+Unit=opensearch-disable-readahead.service
+# Accuracy sets the maximum time interval around the execution time we want to allow
+AccuracySec=15sec
+OnUnitActiveSec=30min
+OnBootSec=1min
+RandomizedDelaySec=0
+
+[Install]
+WantedBy=multi-user.target

Interface::Manual[ipip_ipv4]

Parameters differences:

--- Interface::Manual[ipip_ipv4].orig
+++ Interface::Manual[ipip_ipv4]

+    hotplug   => False
+    family    => inet
+    interface => ipip0
+    ensure    => present

Systemd::Syslog[prometheus_lvs_realserver_mss]

Parameters differences:

--- Systemd::Syslog[prometheus_lvs_realserver_mss].orig
+++ Systemd::Syslog[prometheus_lvs_realserver_mss]

+    programname_comparison => startswith
+    force_stop             => True
+    ensure                 => present
+    base_dir               => /var/log
+    readable_by            => all
+    log_filename           => syslog.log
+    owner                  => root
+    group                  => root

Systemd::Syslog[prometheus_ferm_mss]

Parameters differences:

--- Systemd::Syslog[prometheus_ferm_mss].orig
+++ Systemd::Syslog[prometheus_ferm_mss]

+    programname_comparison => startswith
+    force_stop             => True
+    ensure                 => present
+    base_dir               => /var/log
+    readable_by            => all
+    log_filename           => syslog.log
+    owner                  => root
+    group                  => root

Nrpe::Check[check_opensearch_shard_size_check_9200]

Parameters differences:

--- Nrpe::Check[check_opensearch_shard_size_check_9200].orig
+++ Nrpe::Check[check_opensearch_shard_size_check_9200]

+    command => /usr/lib/nagios/plugins/check_elasticsearch_shard_size.py --url http://localhost:9200 --shard-size-warning 110 --shard-size-critical 140 --timeout 4
+    before  => Monitoring::Service[opensearch_shard_size_check_9200]
+    ensure  => present

Exec[systemd daemon-reload for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer (wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer)]

Parameters differences:

--- Exec[systemd daemon-reload for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer (wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer)].orig
+++ Exec[systemd daemon-reload for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer (wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload
+    before      => ['Service[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer]']

File[/etc/conftool/config.yaml]

Parameters differences:

--- File[/etc/conftool/config.yaml].orig
+++ File[/etc/conftool/config.yaml]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/conftool/config.yaml.orig
+++ /etc/conftool/config.yaml
@@ -0,0 +1,14 @@
+---
+hosts: []
+tcpircbot_host: icinga.wikimedia.org
+tcpircbot_port: 9200
+driver_options:
+  allow_reconnect: true
+  suppress_san_warnings: false
+namespace: "/conftool"
+extensions_config:
+  reqconfig:
+    haproxy_reserved_slots:
+    - 0
+    varnish_acl_ipblocks: []
+conftool2git_address: puppetserver1003.eqiad.wmnet:1312

Augeas[ipip60_set_up]

Parameters differences:

--- Augeas[ipip60_set_up].orig
+++ Augeas[ipip60_set_up]

+    onlyif  => match up[. = 'ip link set up dev ipip60'] size == 0
+    context => /files/etc/network/interfaces/*[. = 'ipip60' and ./family = 'inet6']
+    incl    => /etc/network/interfaces
+    require => Augeas[ipip60_add_up]
+    lens    => Interfaces.lns
+    changes => set up[last()+1] 'ip link set up dev ipip60'

Service[nrpe2nodexp-opensearch_shard_size_check_9200.timer]

Parameters differences:

--- Service[nrpe2nodexp-opensearch_shard_size_check_9200.timer].orig
+++ Service[nrpe2nodexp-opensearch_shard_size_check_9200.timer]

+    before   => ['Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9200.timer (nrpe2nodexp-opensearch_shard_size_check_9200.timer)]']
+    provider => systemd
+    enable   => False
+    ensure   => stopped

Systemd::Tmpfile[opensearch-production-search-psi-codfw]

Parameters differences:

--- Systemd::Tmpfile[opensearch-production-search-psi-codfw].orig
+++ Systemd::Tmpfile[opensearch-production-search-psi-codfw]

+    group  => root
+    owner  => root
+    ensure => present

Exec[systemd daemon-reload for opensearch-hot-threads-log.service (opensearch-hot-threads-log.service)]

Parameters differences:

--- Exec[systemd daemon-reload for opensearch-hot-threads-log.service (opensearch-hot-threads-log.service)].orig
+++ Exec[systemd daemon-reload for opensearch-hot-threads-log.service (opensearch-hot-threads-log.service)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

File[/home/pfischer]

Parameters differences:

--- File[/home/pfischer].orig
+++ File[/home/pfischer]

+    mode    => 0644
+    source  => puppet:///modules/admin/home/skel
+    force   => True
+    owner   => pfischer
+    recurse => remote
+    ensure  => directory
+    group   => 500

Exec[systemd daemon-reload for wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service (wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service)]

Parameters differences:

--- Exec[systemd daemon-reload for wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service (wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service)].orig
+++ Exec[systemd daemon-reload for wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service (wmf_auto_restart_prometheus-elasticsearch-exporter-9600.service)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

Profile::Auto_restarts::Service[prometheus-wmf-elasticsearch-exporter-9600]

Parameters differences:

--- Profile::Auto_restarts::Service[prometheus-wmf-elasticsearch-exporter-9600].orig
+++ Profile::Auto_restarts::Service[prometheus-wmf-elasticsearch-exporter-9600]

+    ensure => present

Systemd::Unit[opensearch-production-search-psi-codfw-gc-log-cleanup.timer]

Parameters differences:

--- Systemd::Unit[opensearch-production-search-psi-codfw-gc-log-cleanup.timer].orig
+++ Systemd::Unit[opensearch-production-search-psi-codfw-gc-log-cleanup.timer]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => opensearch-production-search-psi-codfw-gc-log-cleanup.timer
+    ensure            => present

Class[Profile::Monitoring]

Parameters differences:

--- Class[Profile::Monitoring].orig
+++ Class[Profile::Monitoring]

@@
-    nrpe_check_disk_options => -w 6% -c 3% -W 6% -K 3% -l -e -A -i "/srv/sd[a-b][1-3]" -i "/srv/nvme[0-9]n[0-9]p[0-9]" --exclude-type=fuse --exclude-type=fuse.fuse_dfs --exclude-type=tracefs
+    nrpe_check_disk_options => -w 15% -c 10% -W 6% -K 3% -l -e -A -i "/srv/sd[a-b][1-3]" --exclude-type=tracefs
@@
-    nagios_group            => insetup_codfw
+    nagios_group            => elasticsearch_codfw
@@
-    notifications_enabled   => False
+    notifications_enabled   => True
@@
-    cluster                 => insetup
+    cluster                 => elasticsearch
@@
-    contact_group           => admins
+    contact_group           => team-data-platform,team-discovery

File[/lib/systemd/system/opensearch-hot-threads-log.timer]

Parameters differences:

--- File[/lib/systemd/system/opensearch-hot-threads-log.timer].orig
+++ File[/lib/systemd/system/opensearch-hot-threads-log.timer]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for opensearch-hot-threads-log.timer (opensearch-hot-threads-log.timer)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/opensearch-hot-threads-log.timer.orig
+++ /lib/systemd/system/opensearch-hot-threads-log.timer
@@ -0,0 +1,12 @@
+[Unit]
+Description=Periodic execution of opensearch-hot-threads-log.service
+
+[Timer]
+Unit=opensearch-hot-threads-log.service
+# Accuracy sets the maximum time interval around the execution time we want to allow
+AccuracySec=15sec
+OnCalendar=*-*-* *:00/5:00
+RandomizedDelaySec=0
+
+[Install]
+WantedBy=multi-user.target

Service[wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer]

Parameters differences:

--- Service[wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer].orig
+++ Service[wmf_auto_restart_prometheus-elasticsearch-exporter-9600.timer]

+    provider => systemd
+    enable   => True
+    ensure   => running

File[/var/log/clean_up_dlq]

Parameters differences:

--- File[/var/log/clean_up_dlq].orig
+++ File[/var/log/clean_up_dlq]

+    mode   => 0755
+    backup => False
+    force  => True
+    owner  => root
+    ensure => absent
+    group  => root

Prometheus::Alert::Rule[check_check_tcp-mss-clamper_status_15c652bc7390794191321c22a2885d1f]

Parameters differences:

--- Prometheus::Alert::Rule[check_check_tcp-mss-clamper_status_15c652bc7390794191321c22a2885d1f].orig
+++ Prometheus::Alert::Rule[check_check_tcp-mss-clamper_status_15c652bc7390794191321c22a2885d1f]

+    dashboard          => TODO
+    expr               => (nagios_nrpe_check_result{alert_rule_hash="15c652bc7390794191321c22a2885d1f",check_name="check_check_tcp-mss-clamper_status", status=~"(WARNING|CRITICAL)", severity=~"(warning|critical)"} > 0) * on (instance) group_left (team) role_owner
+    team               => observability
+    description        => Check unit status of tcp-mss-clamper
+    runbook            => https://wikitech.wikimedia.org/wiki/LVS#IPIP_encapsulation_experiments
+    def_label_whitelst => ['team', 'severity']
+    ensure             => absent
+    logs               => https://logstash.wikimedia.org/app/dashboards#/view/2d343ac0-6df8-11f0-8e08-7fab0da52b33?_g=(filters:!((query:(match_phrase:(event.module:check_check_tcp-mss-clamper_status))),(query:(match_phrase:(host.name:{{$labels.instance|stripPort}})))))
+    severity           => info
+    summary            => Check unit status of tcp-mss-clamper
+    instance           => ops
+    group              => nrpechecks
+    alert_name         => check_check_tcp-mss-clamper_status
+    for                => 10m
+    site               => codfw

Systemd::Service[nrpe2nodexp-opensearch_unassigned_shard_check_9200]

Parameters differences:

--- Systemd::Service[nrpe2nodexp-opensearch_unassigned_shard_check_9200].orig
+++ Systemd::Service[nrpe2nodexp-opensearch_unassigned_shard_check_9200]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => absent
+    monitoring_critical      => False
+    require                  => Systemd::Unit[nrpe2nodexp-opensearch_unassigned_shard_check_9200.service]
+    monitoring_enabled       => False
+    unit_type                => timer
+    restart                  => False

Package[wmf-opensearch-search-plugins]

Parameters differences:

--- Package[wmf-opensearch-search-plugins].orig
+++ Package[wmf-opensearch-search-plugins]

+    require  => ['Class[Java]', 'Package[opensearch]']
+    provider => apt
+    ensure   => present
+    before   => ['Service[opensearch_1@production-search-codfw]', 'Service[opensearch_1@production-search-psi-codfw]']

Service[prometheus-elasticsearch-exporter-9600]

Parameters differences:

--- Service[prometheus-elasticsearch-exporter-9600].orig
+++ Service[prometheus-elasticsearch-exporter-9600]

+    enable => True
+    ensure => running

Systemd::Service[opensearch-disable-readahead-production-search-psi-codfw]

Parameters differences:

--- Systemd::Service[opensearch-disable-readahead-production-search-psi-codfw].orig
+++ Systemd::Service[opensearch-disable-readahead-production-search-psi-codfw]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => present
+    monitoring_critical      => False
+    require                  => Systemd::Unit[opensearch-disable-readahead-production-search-psi-codfw.service]
+    monitoring_enabled       => False
+    unit_type                => timer
+    restart                  => False

Systemd::Unit[opensearch-production-search-psi-codfw-gc-log-cleanup.service]

Parameters differences:

--- Systemd::Unit[opensearch-production-search-psi-codfw-gc-log-cleanup.service].orig
+++ Systemd::Unit[opensearch-production-search-psi-codfw-gc-log-cleanup.service]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => opensearch-production-search-psi-codfw-gc-log-cleanup.service
+    ensure            => present

Nrpe::Plugin[check_systemd_unit_status]

Parameters differences:

--- Nrpe::Plugin[check_systemd_unit_status].orig
+++ Nrpe::Plugin[check_systemd_unit_status]

+    ensure => present
+    source => puppet:///modules/systemd/check_systemd_unit_status

Exec[ip addr add 127.0.0.42/32 dev ipip0]

Parameters differences:

--- Exec[ip addr add 127.0.0.42/32 dev ipip0].orig
+++ Exec[ip addr add 127.0.0.42/32 dev ipip0]

+    unless  => ip address show ipip0 | grep -q 127.0.0.42/32
+    path    => /bin:/usr/bin
+    returns => [0, 2]

Exec[s3-credentials-user-production-search-psi-codfw]

Parameters differences:

--- Exec[s3-credentials-user-production-search-psi-codfw].orig
+++ Exec[s3-credentials-user-production-search-psi-codfw]

+    require     => File[/etc/opensearch/production-search-psi-codfw/opensearch.keystore]
+    path        => /bin:/usr/bin
+    environment => ['OPENSEARCH_PATH_CONF=/etc/opensearch/production-search-psi-codfw']
+    unless      => /usr/share/opensearch/bin/opensearch-keystore list | grep s3.client.default.access_key
+    command     => echo 'search:platform' | /usr/share/opensearch/bin/opensearch-keystore add s3.client.default.access_key
+    group       => opensearch

Systemd::Timer[nrpe2nodexp-opensearch_shards_9600]

Parameters differences:

--- Systemd::Timer[nrpe2nodexp-opensearch_shards_9600].orig
+++ Systemd::Timer[nrpe2nodexp-opensearch_shards_9600]

+    fixed_random_delay => False
+    unit_name          => nrpe2nodexp-opensearch_shards_9600.service
+    timer_intervals    => [{'start': 'OnUnitInactiveSec', 'interval': '1min'}, {'interval': '1s', 'start': 'OnActiveSec'}]
+    splay              => 0
+    accuracy           => 15sec
+    ensure             => absent

Systemd::Timer::Job[nrpe2nodexp-opensearch_shard_size_check_9600]

Parameters differences:

--- Systemd::Timer::Job[nrpe2nodexp-opensearch_shard_size_check_9600].orig
+++ Systemd::Timer::Job[nrpe2nodexp-opensearch_shard_size_check_9600]

+    description               => execution of nrpe2nodexp for the check_opensearch_shard_size_check_9600 command.
+    syslog_match_startswith   => True
+    monitoring_contact_groups => admins
+    logging_enabled           => False
+    success_exit_status       => []
+    fixed_random_delay        => False
+    private_tmp               => False
+    logfile_basedir           => /var/log
+    group                     => prometheus-node-exporter
+    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+    syslog_identifier         => nrpe2nodexp-opensearch_shard_size_check_9600
+    send_mail_only_on_error   => True
+    ignore_errors             => True
+    logfile_group             => root
+    interval                  => [{'start': 'OnUnitInactiveSec', 'interval': '1440min'}]
+    send_mail                 => False
+    logfile_name              => syslog.log
+    user                      => nagios
+    ensure                    => absent
+    send_mail_to              => root@cirrussearch2089.codfw.wmnet
+    logfile_perms             => all
+    syslog_force_stop         => True
+    environment               => {}
+    command                   => /usr/local/bin/nrpe2nodexp --alert-rule-hash "6ce4fe694f5df8cba9e0032196c72747" --timeout 10 --check-command "check_opensearch_shard_size_check_9600"
+    monitoring_enabled        => False

File[/etc/rsyslog.d/25-nrpe2nodexp-check-tcp-mss-clamper-status.conf]

Parameters differences:

--- File[/etc/rsyslog.d/25-nrpe2nodexp-check-tcp-mss-clamper-status.conf].orig
+++ File[/etc/rsyslog.d/25-nrpe2nodexp-check-tcp-mss-clamper-status.conf]

+    mode   => 0444
+    notify => Service[rsyslog]
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /etc/rsyslog.d/25-nrpe2nodexp-check-tcp-mss-clamper-status.conf.orig
+++ /etc/rsyslog.d/25-nrpe2nodexp-check-tcp-mss-clamper-status.conf
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: Apache-2.0
+if $programname contains "nrpe2nodexp-check_tcp-mss-clamper_status" then {
+    if ($msg contains "\"ecs.version\": \"1.7.0\"") then {
+        # Send logs to kafka
+        set $.log_outputs = "kafka ecs_170 local";
+    } else {
+        # Filter out non-relevant nrpe2nodexp messages
+        stop
+    }
+}

Rsyslog::Conf[wmf_auto_restart_prometheus-elasticsearch-exporter-9200]

Parameters differences:

--- Rsyslog::Conf[wmf_auto_restart_prometheus-elasticsearch-exporter-9200].orig
+++ Rsyslog::Conf[wmf_auto_restart_prometheus-elasticsearch-exporter-9200]

+    mode     => 0444
+    require  => File[/var/log/wmf_auto_restart_prometheus-elasticsearch-exporter-9200]
+    priority => 40
+    ensure   => present

Systemd::Syslog[opensearch-production-search-psi-codfw-gc-log-cleanup]

Parameters differences:

--- Systemd::Syslog[opensearch-production-search-psi-codfw-gc-log-cleanup].orig
+++ Systemd::Syslog[opensearch-production-search-psi-codfw-gc-log-cleanup]

+    programname_comparison => startswith
+    force_stop             => True
+    ensure                 => present
+    base_dir               => /var/log
+    readable_by            => all
+    log_filename           => syslog.log
+    owner                  => root
+    group                  => root

File_line[rm_lo_clsact_lo]

Parameters differences:

--- File_line[rm_lo_clsact_lo].orig
+++ File_line[rm_lo_clsact_lo]

+    match             => post-up /usr/sbin/tc qdisc add dev lo clsact
+    path              => /etc/network/interfaces
+    ensure            => absent
+    match_for_absence => True

Nrpe::Check[check_check_tcp-mss-clamper_status]

Parameters differences:

--- Nrpe::Check[check_check_tcp-mss-clamper_status].orig
+++ Nrpe::Check[check_check_tcp-mss-clamper_status]

+    command => /usr/local/lib/nagios/plugins/check_systemd_unit_status tcp-mss-clamper
+    before  => Monitoring::Service[check_tcp-mss-clamper_status]
+    ensure  => absent

File[/usr/local/bin/pool-nginx]

Parameters differences:

--- File[/usr/local/bin/pool-nginx].orig
+++ File[/usr/local/bin/pool-nginx]

+    mode   => 0555
+    group  => root
+    before => ['Service[nginx]']
+    owner  => root
+    ensure => present

Content differences:

--- /usr/local/bin/pool-nginx.orig
+++ /usr/local/bin/pool-nginx
@@ -0,0 +1,2 @@
+#!/bin/bash
+/usr/local/bin/safe-service-restart --pools search-https search-psi-https --pool --retries 10 --wait 5

Profile::Prometheus::Elasticsearch_exporter[cirrussearch2089:9600]

Parameters differences:

--- Profile::Prometheus::Elasticsearch_exporter[cirrussearch2089:9600].orig
+++ Profile::Prometheus::Elasticsearch_exporter[cirrussearch2089:9600]

+    extra_config       => 
+    prometheus_port    => 9109
+    elasticsearch_port => 9600

Systemd::Timer::Job[opensearch-disable-readahead-production-search-codfw]

Parameters differences:

--- Systemd::Timer::Job[opensearch-disable-readahead-production-search-codfw].orig
+++ Systemd::Timer::Job[opensearch-disable-readahead-production-search-codfw]

+    description               => Disables readahead on all open files every 30 minutes to alleviate Cirrussearch / opensearch IO load spikes
+    syslog_match_startswith   => True
+    monitoring_contact_groups => admins
+    logging_enabled           => True
+    success_exit_status       => []
+    fixed_random_delay        => False
+    private_tmp               => False
+    logfile_basedir           => /var/log
+    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+    send_mail_only_on_error   => True
+    ignore_errors             => False
+    logfile_group             => root
+    interval                  => [{'start': 'OnUnitActiveSec', 'interval': '30min'}, {'start': 'OnBootSec', 'interval': '1min'}]
+    send_mail                 => False
+    logfile_name              => syslog.log
+    user                      => root
+    ensure                    => present
+    send_mail_to              => root@cirrussearch2089.codfw.wmnet
+    logfile_perms             => all
+    syslog_force_stop         => True
+    environment               => {}
+    command                   => /usr/local/bin/opensearch-disable-readahead.sh production-search-codfw /srv/opensearch
+    monitoring_enabled        => False

File[/etc/conftool/json-schema/]

Parameters differences:

--- File[/etc/conftool/json-schema/].orig
+++ File[/etc/conftool/json-schema/]

+    mode    => 0555
+    path    => /etc/conftool/json-schema
+    source  => puppet:///modules/profile/conftool/json-schema/
+    group   => root
+    owner   => root
+    recurse => True
+    ensure  => directory

File[/etc/ferm/conf.d/10_opensearch-http-9600]

Parameters differences:

--- File[/etc/ferm/conf.d/10_opensearch-http-9600].orig
+++ File[/etc/ferm/conf.d/10_opensearch-http-9600]

+    mode    => 0400
+    require => File[/etc/ferm/conf.d]
+    tag     => ferm
+    notify  => Service[ferm]
+    group   => root
+    owner   => root
+    ensure  => present

Content differences:

--- /etc/ferm/conf.d/10_opensearch-http-9600.orig
+++ /etc/ferm/conf.d/10_opensearch-http-9600
@@ -0,0 +1,8 @@
+# Autogenerated by puppet. DO NOT EDIT BY HAND!
+#
+# 
+&R_SERVICE(tcp, 9600, $DOMAIN_NETWORKS);
+
+
+
+&NO_TRACK(tcp, 9600);

Exec[tlsproxy localssl default_server on ports [9643]]

Parameters differences:

--- Exec[tlsproxy localssl default_server on ports [9643]].orig
+++ Exec[tlsproxy localssl default_server on ports [9643]]

+    refreshonly => True
+    onlyif      => /bin/false
+    command     => /bin/true

File[/etc/update-motd.d/05-insetup--data-platform-ferm]

Parameters differences:

--- File[/etc/update-motd.d/05-insetup--data-platform-ferm].orig
+++ File[/etc/update-motd.d/05-insetup--data-platform-ferm]

-    mode   => 0555
-    group  => root
-    owner  => root
-    ensure => present

Content differences:

--- /etc/update-motd.d/05-insetup--data-platform-ferm.orig
+++ /etc/update-motd.d/05-insetup--data-platform-ferm
@@ -1,2 +0,0 @@
-#!/bin/sh
-printf "%s\n" "cirrussearch2089 is a Host being setup by Data Platform SREs (insetup::data_platform_ferm)"

Rsyslog::Conf[nrpe2nodexp-opensearch_unassigned_shard_check_9600]

Parameters differences:

--- Rsyslog::Conf[nrpe2nodexp-opensearch_unassigned_shard_check_9600].orig
+++ Rsyslog::Conf[nrpe2nodexp-opensearch_unassigned_shard_check_9600]

+    mode     => 0444
+    priority => 25
+    ensure   => absent

File[/home/dcausse]

Parameters differences:

--- File[/home/dcausse].orig
+++ File[/home/dcausse]

+    mode    => 0644
+    source  => puppet:///modules/admin/home/skel
+    force   => True
+    owner   => dcausse
+    recurse => remote
+    ensure  => directory
+    group   => 500

Systemd::Service[opensearch-disable-readahead]

Parameters differences:

--- Systemd::Service[opensearch-disable-readahead].orig
+++ Systemd::Service[opensearch-disable-readahead]

+    override                 => False
+    monitoring_contact_group => admins
+    migration_task           => T321808
+    service_params           => {}
+    ensure                   => absent
+    monitoring_critical      => False
+    require                  => Systemd::Unit[opensearch-disable-readahead.service]
+    monitoring_enabled       => False
+    unit_type                => timer
+    restart                  => False

Monitoring::Exported_nagios_service[cirrussearch2089 elasticsearch-https-production-search-psi-codfw]

Parameters differences:

--- Monitoring::Exported_nagios_service[cirrussearch2089 elasticsearch-https-production-search-psi-codfw].orig
+++ Monitoring::Exported_nagios_service[cirrussearch2089 elasticsearch-https-production-search-psi-codfw]

+    passive_checks_enabled => 1
+    is_volatile            => 0
+    contact_groups         => team-data-platform,team-discovery
+    notification_options   => c,r,f
+    service_description    => Elasticsearch HTTPS for production-search-psi-codfw
+    max_check_attempts     => 3
+    active_checks_enabled  => 1
+    notification_interval  => 0
+    check_command          => check_ssl_on_port!cirrussearch2089.codfw.wmnet!9643
+    retry_interval         => 1
+    check_freshness        => 0
+    notes_url              => https://wikitech.wikimedia.org/wiki/Search
+    notifications_enabled  => 1
+    check_period           => 24x7
+    ensure                 => present
+    check_interval         => 1
+    notification_period    => 24x7
+    host_name              => cirrussearch2089
+    servicegroups          => elasticsearch_codfw

User[ebernhardson]

Parameters differences:

--- User[ebernhardson].orig
+++ User[ebernhardson]

+    allowdupe  => False
+    uid        => 3088
+    shell      => /bin/bash
+    gid        => 500
+    ensure     => present
+    home       => /home/ebernhardson
+    groups     => []
+    comment    => Erik Bernhardson
+    managehome => False

File[/etc/systemd/system/nginx.service.d/numa.conf]

Parameters differences:

--- File[/etc/systemd/system/nginx.service.d/numa.conf].orig
+++ File[/etc/systemd/system/nginx.service.d/numa.conf]

+    mode    => 0444
+    require => File[/etc/systemd/system/nginx.service.d]
+    group   => root
+    before  => Class[Nginx]
+    owner   => root
+    ensure  => present

Content differences:

--- /etc/systemd/system/nginx.service.d/numa.conf.orig
+++ /etc/systemd/system/nginx.service.d/numa.conf
@@ -0,0 +1,3 @@
+[Service]
+ExecStart=
+ExecStart=/usr/bin/numactl -a -N 0,1 /usr/sbin/nginx -g 'daemon on; master_process on;'

Systemd::Timer::Job[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]

Parameters differences:

--- Systemd::Timer::Job[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600].orig
+++ Systemd::Timer::Job[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600]

+    description               => Auto restart job: prometheus-wmf-elasticsearch-exporter-9600
+    syslog_match_startswith   => True
+    monitoring_contact_groups => admins
+    logging_enabled           => True
+    success_exit_status       => []
+    require                   => File[/usr/local/sbin/wmf-auto-restart]
+    fixed_random_delay        => False
+    private_tmp               => False
+    logfile_basedir           => /var/log
+    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+    send_mail_only_on_error   => True
+    ignore_errors             => False
+    logfile_group             => root
+    interval                  => {'start': 'OnCalendar', 'interval': 'Mon,Tue,Wed,Thu,Fri *-*-* 9:55:00'}
+    send_mail                 => False
+    logfile_name              => syslog.log
+    user                      => root
+    ensure                    => present
+    send_mail_to              => root@cirrussearch2089.codfw.wmnet
+    logfile_perms             => all
+    syslog_force_stop         => True
+    environment               => {}
+    command                   => /usr/local/sbin/wmf-auto-restart -s prometheus-wmf-elasticsearch-exporter-9600
+    monitoring_enabled        => False

File[/lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer]

Parameters differences:

--- File[/lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer].orig
+++ File[/lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer (wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer.orig
+++ /lib/systemd/system/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.timer
@@ -0,0 +1,12 @@
+[Unit]
+Description=Periodic execution of wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service
+
+[Timer]
+Unit=wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200.service
+# Accuracy sets the maximum time interval around the execution time we want to allow
+AccuracySec=15sec
+OnCalendar=Mon,Tue,Wed,Thu,Fri *-*-* 18:6:00
+RandomizedDelaySec=0
+
+[Install]
+WantedBy=multi-user.target

File[/lib/systemd/system/prometheus_lvs_realserver_mss.timer]

Parameters differences:

--- File[/lib/systemd/system/prometheus_lvs_realserver_mss.timer].orig
+++ File[/lib/systemd/system/prometheus_lvs_realserver_mss.timer]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for prometheus_lvs_realserver_mss.timer (prometheus_lvs_realserver_mss.timer)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/prometheus_lvs_realserver_mss.timer.orig
+++ /lib/systemd/system/prometheus_lvs_realserver_mss.timer
@@ -0,0 +1,12 @@
+[Unit]
+Description=Periodic execution of prometheus_lvs_realserver_mss.service
+
+[Timer]
+Unit=prometheus_lvs_realserver_mss.service
+# Accuracy sets the maximum time interval around the execution time we want to allow
+AccuracySec=15sec
+OnCalendar=minutely
+RandomizedDelaySec=0
+
+[Install]
+WantedBy=multi-user.target

File[/etc/rsyslog.d/40-wmf-auto-restart-prometheus-wmf-elasticsearch-exporter-9200.conf]

Parameters differences:

--- File[/etc/rsyslog.d/40-wmf-auto-restart-prometheus-wmf-elasticsearch-exporter-9200.conf].orig
+++ File[/etc/rsyslog.d/40-wmf-auto-restart-prometheus-wmf-elasticsearch-exporter-9200.conf]

+    mode   => 0444
+    notify => Service[rsyslog]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/rsyslog.d/40-wmf-auto-restart-prometheus-wmf-elasticsearch-exporter-9200.conf.orig
+++ /etc/rsyslog.d/40-wmf-auto-restart-prometheus-wmf-elasticsearch-exporter-9200.conf
@@ -0,0 +1,10 @@
+# rsyslog.conf(5) configuration file for services.
+# This file is managed by Puppet.
+if $programname startswith "wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200" then {
+    action(
+        type="omfile" file="/var/log/wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9200/syslog.log"
+        fileOwner="root" fileGroup="root"
+        fileCreateMode="0644"
+    )
+    & stop
+}

File[/etc/sudoers.d/nrpe-check_opensearch_shards_9200]

Parameters differences:

--- File[/etc/sudoers.d/nrpe-check_opensearch_shards_9200].orig
+++ File[/etc/sudoers.d/nrpe-check_opensearch_shards_9200]

+    group   => root
+    require => Package[nagios-nrpe-server]
+    owner   => root
+    ensure  => absent

Systemd::Unmask[nginx.service]

Parameters differences:

--- Systemd::Unmask[nginx.service].orig
+++ Systemd::Unmask[nginx.service]

+    refreshonly => True
+    unit        => nginx.service

Nrpe::Monitor_service[opensearch_shard_size_check_9600]

Parameters differences:

--- Nrpe::Monitor_service[opensearch_shard_size_check_9600].orig
+++ Nrpe::Monitor_service[opensearch_shard_size_check_9600]

+    timeout                     => 10
+    nrpe2nodexp_parse_perf_data => False
+    retry_interval              => 180
+    description                 => OpenSearch shard size check - 9600
+    notes_url                   => https://wikitech.wikimedia.org/wiki/Search#If_it_has_been_indexed
+    critical                    => False
+    migration_task              => T371083
+    retries                     => 3
+    ensure                      => present
+    enable_nrpe2nodexp          => False
+    enable_icinga_check         => True
+    check_interval              => 1440
+    contact_group               => admins,team-discovery
+    nrpe_command                => /usr/lib/nagios/plugins/check_elasticsearch_shard_size.py --url http://localhost:9600 --shard-size-warning 110 --shard-size-critical 140 --timeout 4

Systemd::Timer[nrpe2nodexp-opensearch_unassigned_shard_check_9600]

Parameters differences:

--- Systemd::Timer[nrpe2nodexp-opensearch_unassigned_shard_check_9600].orig
+++ Systemd::Timer[nrpe2nodexp-opensearch_unassigned_shard_check_9600]

+    fixed_random_delay => False
+    unit_name          => nrpe2nodexp-opensearch_unassigned_shard_check_9600.service
+    timer_intervals    => [{'start': 'OnUnitInactiveSec', 'interval': '720min'}, {'interval': '1s', 'start': 'OnActiveSec'}]
+    splay              => 0
+    accuracy           => 15sec
+    ensure             => absent

Cfssl::Csr[/etc/cfssl/csr/discovery__cirrussearch2089_codfw_wmnet.csr]

Parameters differences:

--- Cfssl::Csr[/etc/cfssl/csr/discovery__cirrussearch2089_codfw_wmnet.csr].orig
+++ Cfssl::Csr[/etc/cfssl/csr/discovery__cirrussearch2089_codfw_wmnet.csr]

+    common_name => cirrussearch2089.codfw.wmnet
+    names       => []
+    hosts       => ['search.discovery.wmnet', 'search-chi.discovery.wmnet', 'search-omega.discovery.wmnet', 'search-psi.discovery.wmnet', 'search.svc.codfw.wmnet', 'search-chi.svc.codfw.wmnet', 'search-omega.svc.codfw.wmnet', 'search-psi.svc.codfw.wmnet']
+    ensure      => present
+    key         => {'algo': 'ecdsa', 'size': 256}

File[/etc/sudoers.d/elasticsearch-roots]

Parameters differences:

--- File[/etc/sudoers.d/elasticsearch-roots].orig
+++ File[/etc/sudoers.d/elasticsearch-roots]

+    mode         => 0440
+    validate_cmd => /usr/sbin/visudo -cqf %
+    group        => root
+    owner        => root
+    ensure       => present

Content differences:

--- /etc/sudoers.d/elasticsearch-roots.orig
+++ /etc/sudoers.d/elasticsearch-roots
@@ -0,0 +1,3 @@
+# This file is managed by Puppet!
+
+%elasticsearch-roots ALL = (ALL) NOPASSWD: ALL

Package[opensearch-madvise]

Parameters differences:

--- Package[opensearch-madvise].orig
+++ Package[opensearch-madvise]

+    provider => apt
+    ensure   => present

Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9600.service (nrpe2nodexp-opensearch_shard_size_check_9600.service)]

Parameters differences:

--- Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9600.service (nrpe2nodexp-opensearch_shard_size_check_9600.service)].orig
+++ Exec[systemd daemon-reload for nrpe2nodexp-opensearch_shard_size_check_9600.service (nrpe2nodexp-opensearch_shard_size_check_9600.service)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

File[/var/log/wmf_auto_restart_prometheus-elasticsearch-exporter-9600]

Parameters differences:

--- File[/var/log/wmf_auto_restart_prometheus-elasticsearch-exporter-9600].orig
+++ File[/var/log/wmf_auto_restart_prometheus-elasticsearch-exporter-9600]

+    mode   => 0755
+    backup => False
+    force  => True
+    owner  => root
+    ensure => directory
+    group  => root

File[/lib/systemd/system/opensearch-production-search-psi-codfw-gc-log-cleanup.timer]

Parameters differences:

--- File[/lib/systemd/system/opensearch-production-search-psi-codfw-gc-log-cleanup.timer].orig
+++ File[/lib/systemd/system/opensearch-production-search-psi-codfw-gc-log-cleanup.timer]

+    mode   => 0444
+    notify => Exec[systemd daemon-reload for opensearch-production-search-psi-codfw-gc-log-cleanup.timer (opensearch-production-search-psi-codfw-gc-log-cleanup.timer)]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /lib/systemd/system/opensearch-production-search-psi-codfw-gc-log-cleanup.timer.orig
+++ /lib/systemd/system/opensearch-production-search-psi-codfw-gc-log-cleanup.timer
@@ -0,0 +1,12 @@
+[Unit]
+Description=Periodic execution of opensearch-production-search-psi-codfw-gc-log-cleanup.service
+
+[Timer]
+Unit=opensearch-production-search-psi-codfw-gc-log-cleanup.service
+# Accuracy sets the maximum time interval around the execution time we want to allow
+AccuracySec=15sec
+OnCalendar=*-*-* 02:12:00
+RandomizedDelaySec=0
+
+[Install]
+WantedBy=multi-user.target

Admin::Hashuser[ebernhardson]

Parameters differences:

--- Admin::Hashuser[ebernhardson].orig
+++ Admin::Hashuser[ebernhardson]

+    ensure_ssh_key => True

File[/etc/nginx]

Parameters differences:

--- File[/etc/nginx].orig
+++ File[/etc/nginx]

+    require => Package[nginx-light]
+    group   => root
+    before  => ['Service[nginx]']
+    owner   => root
+    ensure  => directory

Class[Profile::Opensearch::Monitoring::Base_checks]

Parameters differences:

--- Class[Profile::Opensearch::Monitoring::Base_checks].orig
+++ Class[Profile::Opensearch::Monitoring::Base_checks]

+    timeout                       => 4
+    enable_unassigned_shard_check => True
+    shard_size_warning            => 110
+    require                       => ['Class[Profile::Opensearch::Server]']
+    threshold                     => >=0.15
+    shard_size_critical           => 140
+    enable_shard_size_check       => True

Exec[ip link set up dev ipip0]

Parameters differences:

--- Exec[ip link set up dev ipip0].orig
+++ Exec[ip link set up dev ipip0]

+    unless  => ip link show ipip0 | grep -q UP
+    path    => /bin:/usr/bin
+    returns => [0, 2]

File[/etc/logrotate.d/opensearch-disable-readahead]

Parameters differences:

--- File[/etc/logrotate.d/opensearch-disable-readahead].orig
+++ File[/etc/logrotate.d/opensearch-disable-readahead]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => absent

Content differences:

--- /etc/logrotate.d/opensearch-disable-readahead.orig
+++ /etc/logrotate.d/opensearch-disable-readahead
@@ -0,0 +1,12 @@
+# logrotate(8) config for opensearch-disable-readahead
+
+/var/log/opensearch-disable-readahead/*.log {
+    daily
+    copytruncate
+    missingok
+    compress
+    delaycompress
+    notifempty
+    rotate 15
+    size 256M
+}

File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.pem]

Parameters differences:

--- File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.pem].orig
+++ File[/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.pem]

+    mode   => 0440
+    notify => ['Exec[nginx-reload]', 'Exec[nginx-reload]']
+    group  => root
+    owner  => root
+    ensure => file

Ferm::Rule[clamp-mss-ipv4]

Parameters differences:

--- Ferm::Rule[clamp-mss-ipv4].orig
+++ Ferm::Rule[clamp-mss-ipv4]

+    prio   => 10
+    desc   => 
+    table  => filter
+    rule   => outerface (eno12399np0 lo) saddr @ipfilter(10.2.1.30) proto tcp sport (9200 9243 9643) tcp-flags (SYN) SYN TCPMSS set-mss 1440;
+    chain  => OUTPUT
+    ensure => present
+    domain => (ip)

Monitoring::Exported_nagios_service[cirrussearch2089 ssh]

Parameters differences:

--- Monitoring::Exported_nagios_service[cirrussearch2089 ssh].orig
+++ Monitoring::Exported_nagios_service[cirrussearch2089 ssh]

@@
-    notifications_enabled => 0
+    notifications_enabled => 1
@@
-    servicegroups         => insetup_codfw
+    servicegroups         => elasticsearch_codfw
@@
-    contact_groups        => admins
+    contact_groups        => team-data-platform,team-discovery

Package[python3-poolcounter]

Parameters differences:

--- Package[python3-poolcounter].orig
+++ Package[python3-poolcounter]

+    provider => apt
+    ensure   => present

Exec[systemd daemon-reload for prometheus_ferm_mss.service (prometheus_ferm_mss.service)]

Parameters differences:

--- Exec[systemd daemon-reload for prometheus_ferm_mss.service (prometheus_ferm_mss.service)].orig
+++ Exec[systemd daemon-reload for prometheus_ferm_mss.service (prometheus_ferm_mss.service)]

+    refreshonly => True
+    command     => /bin/systemctl daemon-reload

File[/root/.etcdrc]

Parameters differences:

--- File[/root/.etcdrc].orig
+++ File[/root/.etcdrc]

+    mode      => 0440
+    show_diff => False
+    group     => root
+    owner     => root
+    ensure    => present

Content differences:

--- /root/.etcdrc.orig
+++ /root/.etcdrc
@@ -0,0 +1,3 @@
+password: another_secret
+username: conftool
+

Systemd::Unit[prometheus_ferm_mss.service]

Parameters differences:

--- Systemd::Unit[prometheus_ferm_mss.service].orig
+++ Systemd::Unit[prometheus_ferm_mss.service]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => prometheus_ferm_mss.service
+    ensure            => present

Exec[udev_reload]

Parameters differences:

--- Exec[udev_reload].orig
+++ Exec[udev_reload]

+    refreshonly => True
+    command     => /usr/bin/udevadm control --reload && /usr/bin/udevadm trigger

Motd::Script[cirrus::opensearch]

Parameters differences:

--- Motd::Script[cirrus::opensearch].orig
+++ Motd::Script[cirrus::opensearch]

+    priority => 5
+    ensure   => present

Logstash::Conf[output-udp-gelf_relay]

Parameters differences:

--- Logstash::Conf[output-udp-gelf_relay].orig
+++ Logstash::Conf[output-udp-gelf_relay]

+    priority => 90
+    ensure   => present

Prometheus::Wmf_elasticsearch_exporter[cirrussearch2089:9200]

Parameters differences:

--- Prometheus::Wmf_elasticsearch_exporter[cirrussearch2089:9200].orig
+++ Prometheus::Wmf_elasticsearch_exporter[cirrussearch2089:9200]

+    indices_to_monitor => ['commonswiki_file', 'wikidatawiki_content', 'enwiki_general', 'enwiki_content', 'viwiki_general', 'commonswiki_general', 'cebwiki_content', 'metawiki_general', 'dewiki_content', 'frwiki_content']
+    prometheus_port    => 9120
+    elasticsearch_port => 9200

Rsyslog::Conf[prometheus_lvs_realserver_mss]

Parameters differences:

--- Rsyslog::Conf[prometheus_lvs_realserver_mss].orig
+++ Rsyslog::Conf[prometheus_lvs_realserver_mss]

+    mode     => 0444
+    require  => File[/var/log/prometheus_lvs_realserver_mss]
+    priority => 40
+    ensure   => present

Ferm::Rule[ipip]

Parameters differences:

--- Ferm::Rule[ipip].orig
+++ Ferm::Rule[ipip]

+    prio   => 10
+    desc   => 
+    table  => filter
+    rule   => saddr 172.16.0.0/12 proto ipencap ACCEPT;
+    chain  => INPUT
+    ensure => present
+    domain => (ip)

Systemd::Unit[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer]

Parameters differences:

--- Systemd::Unit[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer].orig
+++ Systemd::Unit[wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer]

+    restart           => False
+    require           => ['Class[Systemd]']
+    override          => False
+    override_filename => puppet-override.conf
+    unit              => wmf_auto_restart_prometheus-wmf-elasticsearch-exporter-9600.timer
+    ensure            => present

Nrpe::Monitor_service[disk_space]

Parameters differences:

--- Nrpe::Monitor_service[disk_space].orig
+++ Nrpe::Monitor_service[disk_space]

@@
-    contact_group => admins
+    contact_group => team-data-platform,team-discovery
@@
-    nrpe_command  => /usr/lib/nagios/plugins/check_disk -w 6% -c 3% -W 6% -K 3% -l -e -A -i "/srv/sd[a-b][1-3]" -i "/srv/nvme[0-9]n[0-9]p[0-9]" --exclude-type=fuse --exclude-type=fuse.fuse_dfs --exclude-type=tracefs
+    nrpe_command  => /usr/lib/nagios/plugins/check_disk -w 15% -c 10% -W 6% -K 3% -l -e -A -i "/srv/sd[a-b][1-3]" --exclude-type=tracefs

Systemd::Timer::Job[prometheus_ferm_mss]

Parameters differences:

--- Systemd::Timer::Job[prometheus_ferm_mss].orig
+++ Systemd::Timer::Job[prometheus_ferm_mss]

+    description               => Regular job to collect MSS values of ferm-based hosts
+    syslog_match_startswith   => True
+    monitoring_contact_groups => admins
+    logging_enabled           => True
+    success_exit_status       => []
+    fixed_random_delay        => False
+    private_tmp               => False
+    logfile_basedir           => /var/log
+    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
+    send_mail_only_on_error   => True
+    ignore_errors             => False
+    logfile_group             => root
+    interval                  => {'start': 'OnCalendar', 'interval': 'minutely'}
+    send_mail                 => False
+    logfile_name              => syslog.log
+    user                      => root
+    ensure                    => present
+    send_mail_to              => root@cirrussearch2089.codfw.wmnet
+    logfile_perms             => all
+    syslog_force_stop         => True
+    environment               => {}
+    command                   => /usr/local/bin/prometheus-ferm-mss -o /var/lib/prometheus/node.d/ferm-mss.prom -e 10.2.1.30:9200 -e 10.2.1.30:9243 -e 10.2.1.30:9643
+    monitoring_enabled        => False

Opensearch::Log::Hot_threads_cluster[production-search-codfw]

Parameters differences:

--- Opensearch::Log::Hot_threads_cluster[production-search-codfw].orig
+++ Opensearch::Log::Hot_threads_cluster[production-search-codfw]

+    cluster_name => production-search-codfw
+    http_port    => 9200

File[/etc/logstash/jvm.options]

Parameters differences:

--- File[/etc/logstash/jvm.options].orig
+++ File[/etc/logstash/jvm.options]

+    mode    => 0444
+    require => Package[logstash]
+    notify  => Service[logstash]
+    group   => root
+    owner   => root

Content differences:

--- /etc/logstash/jvm.options.orig
+++ /etc/logstash/jvm.options
@@ -0,0 +1,85 @@
+#####################################################################
+## THSI FILE IS MANAGED BY PUPPET
+## puppet://modules/logstash/jvm.options.erb
+#####################################################################
+
+## JVM configuration
+
+# Xms represents the initial size of total heap space
+# Xmx represents the maximum size of total heap space
+
+-Xms192m
+-Xmx192m
+
+################################################################
+## Expert settings
+################################################################
+##
+## All settings below this section are considered
+## expert settings. Don't tamper with them unless
+## you understand what you are doing
+##
+################################################################
+
+## GC configuration
+-XX:CMSInitiatingOccupancyFraction=75
+-XX:+UseCMSInitiatingOccupancyOnly
+
+## optimizations
+
+# disable calls to System#gc
+-XX:+DisableExplicitGC
+
+## locale
+# Set the locale language
+#-Duser.language=en
+
+# Set the locale country
+#-Duser.country=US
+
+# Set the locale variant, if any
+#-Duser.variant=
+
+## basic
+
+# set the I/O temp directory
+#-Djava.io.tmpdir=$HOME
+
+# set to headless, just in case
+-Djava.awt.headless=true
+
+# ensure UTF-8 encoding by default (e.g. filenames)
+-Dfile.encoding=UTF-8
+
+# use our provided JNA always versus the system one
+#-Djna.nosys=true
+
+# log4j2
+-Dlog4j2.formatMsgNoLookups=true
+
+## heap dumps
+
+# generate a heap dump when an allocation from the Java heap fails
+# heap dumps are created in the working directory of the JVM
+-XX:+HeapDumpOnOutOfMemoryError
+
+# specify an alternative path for heap dumps
+# ensure the directory exists and has sufficient space
+#-XX:HeapDumpPath=${LOGSTASH_HOME}/heapdump.hprof
+
+## GC logging
+#-XX:+PrintGCDetails
+#-XX:+PrintGCTimeStamps
+#-XX:+PrintGCDateStamps
+#-XX:+PrintClassHistogram
+#-XX:+PrintTenuringDistribution
+#-XX:+PrintGCApplicationStoppedTime
+
+# log GC status to a file with time stamps
+# ensure the directory exists
+#-Xloggc:${LS_GC_LOG_FILE}
+
+
+# Custom flags for GC logging.
+-Xlog:gc*:file=/var/log/logstash/logstash_jvm_gc.%p.log::filecount=10,filesize=20000
+-Xlog:gc+age=trace

File[/etc/opensearch/production-search-codfw/opensearch.yml]

Parameters differences:

--- File[/etc/opensearch/production-search-codfw/opensearch.yml].orig
+++ File[/etc/opensearch/production-search-codfw/opensearch.yml]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => file

Content differences:

--- /etc/opensearch/production-search-codfw/opensearch.yml.orig
+++ /etc/opensearch/production-search-codfw/opensearch.yml
@@ -0,0 +1,195 @@
+# ======================== OpenSearch Configuration =========================
+#
+# NOTE: OpenSearch comes with reasonable defaults for most settings.
+#       Before you set out to tweak and tune the configuration, make sure you
+#       understand what are you trying to accomplish and the consequences.
+#
+# The primary way of configuring a node is via this file. This template lists
+# the most important settings you may want to configure for a production cluster.
+#
+# Please consult the documentation for further information on configuration options:
+# https://opensearch.org/docs/opensearch/configuration/
+#
+# ---------------------------------- Cluster -----------------------------------
+#
+# Use a descriptive name for your cluster:
+#
+cluster.name: production-search-codfw
+#
+# ------------------------------------ Node ------------------------------------
+#
+# Use a descriptive name for the node:
+#
+node.name: cirrussearch2089-production-search-codfw
+#
+# Add custom attributes to the node:
+#
+#node.attr.rack: r1
+#
+# Row/rack awareness attributes
+#
+cluster.routing.allocation.awareness.attributes: row
+node.attr.row: codfw-row-a
+node.attr.rack: A4
+#
+# Other attributes
+#
+node.attr.hostname: cirrussearch2089
+node.attr.fqdn: cirrussearch2089.codfw.wmnet
+#
+# Allow this node to be eligible as a master node (enabled by default):
+node.master: false
+# Allow this node to store data (enabled by default):
+node.data: true
+#
+# ----------------------------------- Paths ------------------------------------
+#
+# Path to directory where to store the data (separate multiple locations by comma):
+#
+path.data: /srv/opensearch/production-search-codfw
+#
+# Path to log files:
+#
+path.logs: /var/log/opensearch
+#
+# ----------------------------------- Memory -----------------------------------
+#
+# Lock the memory on startup:
+#
+bootstrap.memory_lock: false # we don't have swap anyway
+#
+# Make sure that the heap size is set to about half the memory available
+# on the system and that the owner of the process is allowed to use this
+# limit.
+#
+# OpenSearch performs poorly when the system is swapping the memory.
+#
+# ---------------------------------- Network -----------------------------------
+#
+# Set the bind address to a specific IP (IPv4 or IPv6):
+#
+#network.host: 192.168.0.1
+#
+# Set a custom port for HTTP:
+#
+http.port: 9200
+#
+# For more information, consult the network module documentation.
+#
+# Set the address other nodes will use to communicate with this node. If not
+# set, it is automatically derived. It must point to an actual IP address.
+#
+network.publish_host: 10.192.6.18
+
+# Set both 'bind_host' and 'publish_host':
+#
+network.host: [ _local_,_site_ ]
+
+# Set a custom port for the node to node communication (9300 by default):
+#
+transport.tcp.port: 9300
+#
+# --------------------------------- Discovery ----------------------------------
+#
+# Pass an initial list of hosts to perform discovery when this node is started:
+# The default list of hosts is ["127.0.0.1", "[::1]"]
+#
+#discovery.seed_hosts: ["host1", "host2"]
+#
+# Bootstrap the cluster using an initial set of master-eligible nodes:
+#
+#cluster.initial_master_nodes: ["node-1", "node-2"]
+#
+discovery.zen.ping.unicast.hosts: ["cirrussearch2093.codfw.wmnet", "cirrussearch2061.codfw.wmnet", "cirrussearch2081.codfw.wmnet", "cirrussearch2084.codfw.wmnet", "cirrussearch2111.codfw.wmnet"]
+#
+# For more information, consult the discovery and cluster formation module documentation.
+#
+# ---------------------------------- Gateway -----------------------------------
+#
+# Block initial recovery after a full cluster restart until N nodes are started:
+#
+gateway.recover_after_nodes: 24
+#
+# For more information, consult the gateway module documentation.
+#
+#
+gateway:
+    recover_after_time: 5m
+    expected_nodes: 36
+#
+# ---------------------------------- Various -----------------------------------
+#
+# Require explicit names when deleting indices:
+#
+#action.destructive_requires_name: true
+#
+# ---------------------------------- Custom settings ---------------------------
+#
+#path.plugins: /usr/share/opensearch/plugins
+#
+# If a plugin listed here is not installed for current node, the node will not start.
+#
+plugin.mandatory: analysis-hebrew,analysis-icu,analysis-smartcn,analysis-stempel,analysis-ukrainian,cirrus-highlighter,opensearch-analysis-stconvert,opensearch-extra,opensearch-extra-analysis-esperanto,opensearch-extra-analysis-serbian,opensearch-extra-analysis-slovak,ltr
+
+# Specify the pattern of index names allowed to be created automatically
+action.auto_create_index: +apifeatureusage-*,+glent_*,-*
+
+# Protect against accidental close/delete operations on all indices. You can
+# still close/delete individual indices.
+action.destructive_requires_name: true
+
+# Allow up to 5000 shards to be queried at a time. The default
+# 1k is too low to allow mwgrep to operate.
+action.search.shard_count.limit: 5000
+
+# Enable the disk space aware shard allocator
+cluster.routing.allocation.disk.threshold_enabled: true
+# Stop allocating new shards to nodes with 75% disk utilization
+cluster.routing.allocation.disk.watermark.low: 0.75
+# Move shards off of nodes with 80% disk utilization
+cluster.routing.allocation.disk.watermark.high: 0.80
+# Check disk utilization every 60 seconds
+cluster.info.update.interval: 60s
+
+# Set the number of concurrent recoveries happening on a node:
+#
+# 1. During the initial recovery
+cluster.routing.allocation.node_initial_primaries_recoveries: 3
+
+# 2. During adding/removing nodes, rebalancing, etc
+cluster.routing.allocation.node_concurrent_recoveries: 2
+
+# Set to throttle throughput when recovering (eg. 100mb, by default 20mb):
+indices.recovery.max_bytes_per_sec: 40mb
+
+##
+# OpenSearch cluster balance configuration
+##
+# This puts much more weight on distributing the shards per index then the defaults.  This will make sure that,
+# for example, each live node gets a copy of enwiki.
+cluster.routing.allocation.balance:
+    shard: 0.195
+    index: 0.8
+
+##
+# Filter cache size
+##
+indices.queries.cache.size: 20%
+
+##
+# Thread pool settings
+##
+thread_pool:
+    write:
+        size: 6
+        queue_size: 1000
+
+
+
+
+
+##
+# Size the LTR cache to prevent churn under normal and AB testing scenarios
+ltr.caches.max_mem: 100mb
+
+

File[/etc/nginx/sites-enabled/production-search-codfw]

Parameters differences:

--- File[/etc/nginx/sites-enabled/production-search-codfw].orig
+++ File[/etc/nginx/sites-enabled/production-search-codfw]

+    target => /etc/nginx/sites-available/production-search-codfw
+    group  => root
+    before => ['Service[nginx]']
+    owner  => root
+    ensure => link

File[/etc/opensearch]

Parameters differences:

--- File[/etc/opensearch].orig
+++ File[/etc/opensearch]

+    mode    => 0755
+    purge   => True
+    force   => True
+    owner   => root
+    recurse => True
+    ensure  => directory
+    group   => root

File[/usr/local/bin/pool-opensearch_1@production-search-codfw]

Parameters differences:

--- File[/usr/local/bin/pool-opensearch_1@production-search-codfw].orig
+++ File[/usr/local/bin/pool-opensearch_1@production-search-codfw]

+    mode   => 0555
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /usr/local/bin/pool-opensearch_1@production-search-codfw.orig
+++ /usr/local/bin/pool-opensearch_1@production-search-codfw
@@ -0,0 +1,2 @@
+#!/bin/bash
+/usr/local/bin/safe-service-restart --pools search search-https --pool --retries 10 --wait 5

Tlsproxy::Localssl[production-search-psi-codfw]

Parameters differences:

--- Tlsproxy::Localssl[production-search-psi-codfw].orig
+++ Tlsproxy::Localssl[production-search-psi-codfw]

+    acme_chief         => False
+    server_aliases     => ['search.discovery.wmnet', 'search-chi.discovery.wmnet', 'search-omega.discovery.wmnet', 'search-psi.discovery.wmnet', 'search.svc.codfw.wmnet', 'search-chi.svc.codfw.wmnet', 'search-omega.svc.codfw.wmnet', 'search-psi.svc.codfw.wmnet']
+    upstream_ip        => 10.192.6.18
+    upstream_ports     => [9600]
+    default_server     => True
+    keepalive_timeout  => 60
+    require            => ['Class[Profile::Tlsproxy::Instance]']
+    access_log         => False
+    acme_certname      => production-search-psi-codfw
+    tls_ports          => [9643]
+    enable_http2       => False
+    read_timeout       => 180
+    only_get_requests  => False
+    keepalive_requests => 100
+    server_name        => cirrussearch2089.codfw.wmnet
+    skip_private       => False
+    cfssl_paths        => {'cert': '/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.pem', 'key': '/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet-key.pem', 'chain': '/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chain.pem', 'chained': '/etc/cfssl/ssl/discovery__cirrussearch2089_codfw_wmnet/discovery__cirrussearch2089_codfw_wmnet.chained.pem'}

File[/etc/logstash/conf.d/README]

Parameters differences:

--- File[/etc/logstash/conf.d/README].orig
+++ File[/etc/logstash/conf.d/README]

+    require => Package[logstash]
+    source  => puppet:///modules/logstash/conf.d/README
+    group   => logstash
+    owner   => logstash
+    ensure  => present

File[/etc/logrotate.d/wmf_auto_restart_prometheus-elasticsearch-exporter-9600]

Parameters differences:

--- File[/etc/logrotate.d/wmf_auto_restart_prometheus-elasticsearch-exporter-9600].orig
+++ File[/etc/logrotate.d/wmf_auto_restart_prometheus-elasticsearch-exporter-9600]

+    mode   => 0444
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/logrotate.d/wmf_auto_restart_prometheus-elasticsearch-exporter-9600.orig
+++ /etc/logrotate.d/wmf_auto_restart_prometheus-elasticsearch-exporter-9600
@@ -0,0 +1,12 @@
+# logrotate(8) config for wmf_auto_restart_prometheus-elasticsearch-exporter-9600
+
+/var/log/wmf_auto_restart_prometheus-elasticsearch-exporter-9600/*.log {
+    daily
+    copytruncate
+    missingok
+    compress
+    delaycompress
+    notifempty
+    rotate 15
+    size 256M
+}

Logrotate::Conf[clean_up_dlq]

Parameters differences:

--- Logrotate::Conf[clean_up_dlq].orig
+++ Logrotate::Conf[clean_up_dlq]

+    ensure => absent

Systemd::Timer[nrpe2nodexp-check_tcp-mss-clamper_status]

Parameters differences:

--- Systemd::Timer[nrpe2nodexp-check_tcp-mss-clamper_status].orig
+++ Systemd::Timer[nrpe2nodexp-check_tcp-mss-clamper_status]

+    fixed_random_delay => False
+    unit_name          => nrpe2nodexp-check_tcp-mss-clamper_status.service
+    timer_intervals    => [{'start': 'OnUnitInactiveSec', 'interval': '10min'}, {'interval': '1s', 'start': 'OnActiveSec'}]
+    splay              => 0
+    accuracy           => 15sec
+    ensure             => absent

Exec[disable-rp-filter-ipip60]

Parameters differences:

--- Exec[disable-rp-filter-ipip60].orig
+++ Exec[disable-rp-filter-ipip60]

+    unless  => /usr/sbin/sysctl -n net.ipv4.conf.ipip60.rp_filter |grep -- '0'
+    command => /usr/sbin/sysctl -q net.ipv4.conf.ipip60.rp_filter=0
+    require => Interface::Ipip[ipip_ipv6]

File[/etc/rsyslog.d/40-opensearch-disable-readahead-production-search-psi-codfw.conf]

Parameters differences:

--- File[/etc/rsyslog.d/40-opensearch-disable-readahead-production-search-psi-codfw.conf].orig
+++ File[/etc/rsyslog.d/40-opensearch-disable-readahead-production-search-psi-codfw.conf]

+    mode   => 0444
+    notify => Service[rsyslog]
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/rsyslog.d/40-opensearch-disable-readahead-production-search-psi-codfw.conf.orig
+++ /etc/rsyslog.d/40-opensearch-disable-readahead-production-search-psi-codfw.conf
@@ -0,0 +1,10 @@
+# rsyslog.conf(5) configuration file for services.
+# This file is managed by Puppet.
+if $programname startswith "opensearch-disable-readahead-production-search-psi-codfw" then {
+    action(
+        type="omfile" file="/var/log/opensearch-disable-readahead-production-search-psi-codfw/syslog.log"
+        fileOwner="root" fileGroup="root"
+        fileCreateMode="0644"
+    )
+    & stop
+}

Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_shard_size_check_9600]

Parameters differences:

--- Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_shard_size_check_9600].orig
+++ Monitoring::Exported_nagios_service[cirrussearch2089 opensearch_shard_size_check_9600]

+    passive_checks_enabled => 1
+    is_volatile            => 0
+    contact_groups         => admins,team-discovery
+    notification_options   => c,r,f
+    service_description    => OpenSearch shard size check - 9600
+    max_check_attempts     => 3
+    active_checks_enabled  => 1
+    notification_interval  => 0
+    check_command          => nrpe_check!check_opensearch_shard_size_check_9600!10
+    retry_interval         => 180
+    check_freshness        => 0
+    notes_url              => https://wikitech.wikimedia.org/wiki/Search#If_it_has_been_indexed
+    notifications_enabled  => 1
+    check_period           => 24x7
+    ensure                 => present
+    check_interval         => 1440
+    notification_period    => 24x7
+    host_name              => cirrussearch2089
+    servicegroups          => elasticsearch_codfw

Apt::Repository[wikimedia-opensearch-plugins]

Parameters differences:

--- Apt::Repository[wikimedia-opensearch-plugins].orig
+++ Apt::Repository[wikimedia-opensearch-plugins]

+    uri                      => http://apt.wikimedia.org/wikimedia
+    allow_releaseinfo_change => False
+    trust_repo               => False
+    components               => component/opensearch13
+    source                   => True
+    bin                      => True
+    ensure                   => present
+    dist                     => bullseye-wikimedia

File[/etc/update-motd.d/05-cirrus--opensearch]

Parameters differences:

--- File[/etc/update-motd.d/05-cirrus--opensearch].orig
+++ File[/etc/update-motd.d/05-cirrus--opensearch]

+    mode   => 0555
+    group  => root
+    owner  => root
+    ensure => present

Content differences:

--- /etc/update-motd.d/05-cirrus--opensearch.orig
+++ /etc/update-motd.d/05-cirrus--opensearch
@@ -0,0 +1,2 @@
+#!/bin/sh
+printf "%s\n" "cirrussearch2089 is a OpenSearch cirrus (cirrus::opensearch)"

Systemd::Timer[wmf_auto_restart_prometheus-elasticsearch-exporter-9200]

Parameters differences:

--- Systemd::Timer[wmf_auto_restart_prometheus-elasticsearch-exporter-9200].orig
+++ Systemd::Timer[wmf_auto_restart_prometheus-elasticsearch-exporter-9200]

+    fixed_random_delay => False
+    unit_name          => wmf_auto_restart_prometheus-elasticsearch-exporter-9200.service
+    timer_intervals    => [{'start': 'OnCalendar', 'interval': 'Mon,Tue,Wed,Thu,Fri *-*-* 2:54:00'}]
+    splay              => 0
+    accuracy           => 15sec
+    ensure             => present

Prometheus::Alert::Rule[check_opensearch_unassigned_shard_check_9200_f98d7b74d9467ea0aa193a52c18e30ab]

Parameters differences:

--- Prometheus::Alert::Rule[check_opensearch_unassigned_shard_check_9200_f98d7b74d9467ea0aa193a52c18e30ab].orig
+++ Prometheus::Alert::Rule[check_opensearch_unassigned_shard_check_9200_f98d7b74d9467ea0aa193a52c18e30ab]

+    dashboard          => TODO
+    expr               => (nagios_nrpe_check_result{alert_rule_hash="f98d7b74d9467ea0aa193a52c18e30ab",check_name="check_opensearch_unassigned_shard_check_9200", status=~"(WARNING|CRITICAL)", severity=~"(warning|critical)"} > 0) * on (instance) group_left (team) role_owner
+    team               => observability
+    description        => OpenSearch unassigned shard check - 9200
+    runbook            => https://wikitech.wikimedia.org/wiki/Search#Administration
+    def_label_whitelst => ['team', 'severity']
+    ensure             => absent
+    logs               => https://logstash.wikimedia.org/app/dashboards#/view/2d343ac0-6df8-11f0-8e08-7fab0da52b33?_g=(filters:!((query:(match_phrase:(event.module:check_opensearch_unassigned_shard_check_9200))),(query:(match_phrase:(host.name:{{$labels.instance|stripPort}})))))
+    severity           => info
+    summary            => OpenSearch unassigned shard check - 9200
+    instance           => ops
+    group              => nrpechecks
+    alert_name         => check_opensearch_unassigned_shard_check_9200
+    for                => 86400m
+    site               => codfw

File_line[opensearch_bin_bashrc]

Parameters differences:

--- File_line[opensearch_bin_bashrc].orig
+++ File_line[opensearch_bin_bashrc]

+    path   => /root/.bashrc
+    ensure => present
+    line   => PATH=${PATH}:/usr/share/opensearch/bin  # Managed by puppet

Compilation results for cirrussearch2089.codfw.wmnet: System changes detected

Catalog differences

Summary

Resources only in the new catalog

Resources only in the old catalog

Resources modified

Relevant files