Compilation results for db2250.codfw.wmnet: System changes detected

Catalog differences

Summary

Resources only in the new catalog

• Exec[systemd daemon-reload for nrpe2nodexp-mysqld.timer (nrpe2nodexp-mysqld.timer)]
• Monitoring::Exported_nagios_service[db2250 mariadb_replica_sql_lag_s1]
• Motd::Script[db-section-s1]
• File[/etc/mysql/ssl/server-cert.pem]
• Prometheus::Alert::Rule[check_mysqld_ca81db2c9e98d408f9f65c79817da116]
• Systemd::Timer::Job[nrpe2nodexp-mariadb_disk_space]
• Monitoring::Exported_nagios_service[db2250 mariadb_memory]
• File_line[auto_restart_file_presence_prometheus-mysqld-exporter@s1]
• File[/var/lib/prometheus/node.d/check_mariadb_replica_sql_lag_s1.prom]
• Package[monitoring-plugins-contrib]
• File[/etc/ferm/conf.d/10_s1_mariadb_dba]
• File[/usr/local/bin/mbstream]
• Ferm::Service[s1_mariadb_dba]
• Systemd::Unit[wmf_auto_restart_prometheus-mysqld-exporter@s1.timer]
• Systemd::Timer[nrpe2nodexp-mariadb_replica_sql_state_s1]
• Service[nrpe2nodexp-mysqld.timer]
• Package[mariadb-backup]
• Monitoring::Exported_nagios_service[db2250 mysqld]
• File[/srv/sqldata.s1]
• Nrpe::Check[check_mysqld]
• User[mysql]
• Systemd::Unit[nrpe2nodexp-mariadb_replica_io_state_s1.service]
• Systemd::Unit[nrpe2nodexp-mariadb_disk_space.service]
• File[/etc/default/prometheus]
• Monitoring::Service[mariadb_replica_sql_state_s1]
• Nrpe::Monitor_service[mysqld]
• Nrpe::Check[check_mariadb_read_only_s1]
• File[/var/lib/prometheus/node.d/check_mariadb_replica_sql_state_s1.prom]
• Profile::Auto_restarts::Service[prometheus-mysqld-exporter@s1]
• Group[mysql]
• Nrpe::Monitor_service[mariadb_read_only_s1]
• Rsyslog::Conf[nrpe2nodexp-mariadb_replica_sql_state_s1]
• Exec[systemd daemon-reload for wmf_auto_restart_prometheus-mysqld-exporter@s1.timer (wmf_auto_restart_prometheus-mysqld-exporter@s1.timer)]
• File[/etc/mysql/ssl/client-cert.pem]
• File[/etc/ferm/conf.d/10_s1_mariadb_internal]
• Profile::Prometheus::Mysqld_exporter_instance[s1]
• Package[percona-toolkit]
• File[/lib/systemd/system/prometheus-mysqld-exporter@.service]
• Rsyslog::Conf[nrpe2nodexp-mariadb_read_only_s1]
• Class[Profile::Mariadb::Dbstore_multiinstance]
• Systemd::Service[nrpe2nodexp-mariadb_replica_sql_lag_s1]
• Systemd::Unit[nrpe2nodexp-mariadb_memory.timer]
• File[/etc/mysql/ssl]
• Nrpe::Monitor_service[mariadb_memory]
• Package[wmf-mariadb1011]
• Systemd::Timer::Job[nrpe2nodexp-mariadb_memory]
• Class[Mariadb::Monitor_process]
• Concat_fragment[/etc/mysql_puppet_ca_chain]
• Package[grc]
• File[/var/lib/prometheus/node.d/check_mysqld.prom]
• File[/etc/nagios/nrpe.d/check_mariadb_read_only_s1.cfg]
• File[/lib/systemd/system/nrpe2nodexp-mysqld.service]
• File[/lib/systemd/system/wmf_auto_restart_prometheus-mysqld-exporter@s1.timer]
• File[/usr/lib/nagios/plugins/check_mariadb.pl]
• File[/usr/local/bin/mysql.s1]
• File[/lib/systemd/system/nrpe2nodexp-mariadb_disk_space.timer]
• Rsyslog::Conf[nrpe2nodexp-mariadb_replica_sql_lag_s1]
• Sudo::User[nrpe-check_mariadb_disk_space]
• File[/etc/nagios/nrpe.d/check_mariadb_disk_space.cfg]
• Exec[systemd daemon-reload for wmf_auto_restart_prometheus-mysqld-exporter@s1.service (wmf_auto_restart_prometheus-mysqld-exporter@s1.service)]
• File[/etc/mysql/mysqld.conf.d/s1.cnf]
• Systemd::Timer::Job[nrpe2nodexp-mysqld]
• Monitoring::Exported_nagios_service[db2250 mariadb_disk_space]
• File[/etc/sudoers.d/nrpe-check_mariadb_read_only_s1]
• Class[Profile::Mariadb::Wmfmariadbpy]
• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_memory.timer (nrpe2nodexp-mariadb_memory.timer)]
• Rsyslog::Conf[nrpe2nodexp-mariadb_disk_space]
• Sudo::User[nrpe-check_mariadb_replica_sql_state_s1]
• File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-replica-sql-lag-s1.conf]
• Mariadb::Monitor_readonly[s1]
• Class[Mariadb::Config]
• Class[Wmfmariadbpy]
• File[/etc/wmfmariadbpy/section_ports.csv]
• File[/var/log/wmf_auto_restart_prometheus-mysqld-exporter_s1]
• File[/etc/update-motd.d/05-mariadb--backup-source]
• Nrpe::Check[check_mariadb_replica_sql_state_s1]
• File[/lib/systemd/system/nrpe2nodexp-mariadb_disk_space.service]
• Class[Mariadb::Packages_wmf]
• Systemd::Syslog[wmf_auto_restart_prometheus-mysqld-exporter_s1]
• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_state_s1.service (nrpe2nodexp-mariadb_replica_sql_state_s1.service)]
• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_state_s1.timer (nrpe2nodexp-mariadb_replica_sql_state_s1.timer)]
• Monitoring::Exported_nagios_service[db2250 mariadb_replica_sql_state_s1]
• File[/usr/local/lib/nagios/plugins/check_mariadb]
• Systemd::Timer[nrpe2nodexp-mariadb_memory]
• Monitoring::Service[mysqld]
• Systemd::Unit[nrpe2nodexp-mariadb_memory.service]
• File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-replica-sql-state-s1.conf]
• Exec[systemd daemon-reload for nrpe2nodexp-mysqld.service (nrpe2nodexp-mysqld.service)]
• File[/etc/mysql/my.cnf]
• Systemd::Unit[nrpe2nodexp-mariadb_replica_sql_state_s1.service]
• Sudo::User[nrpe-check_mariadb_read_only_s1]
• Systemd::Service[nrpe2nodexp-mysqld]
• File[/etc/sudoers.d/nrpe-check_mariadb_disk_space]
• Nrpe::Plugin[check_mariadb]
• File[/etc/nagios/nrpe.d/check_mariadb_replica_sql_state_s1.cfg]
• Monitoring::Service[mariadb_disk_space]
• File[/var/lib/prometheus/.my.s1.cnf]
• Service[nrpe2nodexp-mariadb_disk_space.timer]
• File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_state_s1.service]
• Systemd::Unit[prometheus-mysqld-exporter@]
• Ferm::Service[s1_mariadb_internal]
• Monitoring::Service[mariadb_replica_sql_lag_s1]
• File[/lib/systemd/system/nrpe2nodexp-mariadb_memory.service]
• Service[nrpe2nodexp-mariadb_read_only_s1.timer]
• File[/etc/mysql/ssl/server.p12]
• Service[nrpe2nodexp-mariadb_replica_sql_state_s1.timer]
• File[/etc/ferm/conf.d/10_s1_orchestrator]
• Profile::Mariadb::Section[s1]
• Logrotate::Conf[wmf_auto_restart_prometheus-mysqld-exporter_s1]
• File[/var/lib/prometheus/node.d/check_mariadb_memory.prom]
• Monitoring::Service[mariadb_read_only_s1]
• Rsyslog::Conf[nrpe2nodexp-mariadb_memory]
• File[/etc/systemd/system/mariadb.service.d/override.conf]
• File[/etc/sudoers.d/nrpe-check_mariadb_replica_sql_state_s1]
• Nrpe::Check[check_mariadb_replica_io_state_s1]
• File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_io_state_s1.service]
• Systemd::Timer::Job[nrpe2nodexp-mariadb_replica_sql_state_s1]
• Prometheus::Alert::Rule[check_mariadb_memory_b5a92c72542d5f0c031014e7a8db014e]
• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_read_only_s1.service (nrpe2nodexp-mariadb_read_only_s1.service)]
• Concat::Fragment[/etc/mysql_puppet_agent_cert]
• Systemd::Unit[nrpe2nodexp-mariadb_replica_io_state_s1.timer]
• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_io_state_s1.timer (nrpe2nodexp-mariadb_replica_io_state_s1.timer)]
• Prometheus::Alert::Rule[check_mariadb_read_only_s1_42baf529e0f3bfca391aca0a24c880e9]
• Sudo::User[nrpe-check_mariadb_replica_sql_lag_s1]
• Nrpe::Monitor_service[mariadb_replica_sql_lag_s1]
• File[/var/lib/prometheus/node.d/check_mariadb_read_only_s1.prom]
• Monitoring::Exported_nagios_service[db2250 mariadb_read_only_s1]
• Service[nrpe2nodexp-mariadb_replica_sql_lag_s1.timer]
• File[/etc/nagios/nrpe.d/check_mariadb_replica_io_state_s1.cfg]
• Concat_fragment[/etc/mysql_puppet_agent_cert]
• Package[python3-wmfmariadbpy]
• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_lag_s1.timer (nrpe2nodexp-mariadb_replica_sql_lag_s1.timer)]
• File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-read-only-s1.conf]
• File[/etc/nagios/nrpe.d/check_mariadb_replica_sql_lag_s1.cfg]
• Exec[systemctl try-restart prometheus-mysqld-exporter@s1]
• Sudo::User[nrpe-check_mariadb_replica_io_state_s1]
• File[/lib/systemd/system/nrpe2nodexp-mariadb_memory.timer]
• File[/srv/tmp.s1]
• Nrpe::Monitor_service[mariadb_replica_io_state_s1]
• File[/lib/systemd/system/wmf_auto_restart_prometheus-mysqld-exporter@s1.service]
• Rsyslog::Conf[wmf_auto_restart_prometheus-mysqld-exporter_s1]
• Systemd::Unit[nrpe2nodexp-mysqld.service]
• Systemd::Timer[nrpe2nodexp-mariadb_replica_io_state_s1]
• Motd::Message[mariadb::backup_source]
• Nrpe::Check[check_mariadb_disk_space]
• File[/lib/systemd/system/nrpe2nodexp-mariadb_read_only_s1.timer]
• Systemd::Timer[wmf_auto_restart_prometheus-mysqld-exporter@s1]
• File[/etc/mysql/ssl/server.key]
• Monitoring::Service[mariadb_memory]
• Service[nrpe2nodexp-mariadb_memory.timer]
• File[/etc/nagios/nrpe.d/check_mysqld.cfg]
• Systemd::Timer::Job[wmf_auto_restart_prometheus-mysqld-exporter@s1]
• Systemd::Unit[nrpe2nodexp-mariadb_replica_sql_lag_s1.timer]
• Nrpe::Monitor_service[mariadb_disk_space]
• Systemd::Timer[nrpe2nodexp-mariadb_disk_space]
• File[/etc/sudoers.d/nrpe-check_mariadb_replica_sql_lag_s1]
• Systemd::Service[nrpe2nodexp-mariadb_replica_io_state_s1]
• Node[__node_regexp__db2250.codfw.]
• File[/etc/rsyslog.d/25-nrpe2nodexp-mysqld.conf]
• File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_io_state_s1.timer]
• File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_state_s1.timer]
• Ferm::Service[s1_orchestrator]
• Prometheus::Alert::Rule[check_mariadb_replica_io_state_s1_0d5979f7abbb49256cfedd70a646784e]
• File[/etc/sudoers.d/nrpe-check_mariadb_replica_io_state_s1]
• Systemd::Unit[nrpe2nodexp-mariadb_replica_sql_state_s1.timer]
• File[/var/lib/prometheus/node.d/check_mariadb_disk_space.prom]
• Exec[systemctl-daemon-reload]
• File[/etc/mysql]
• Class[Mariadb::Service]
• Class[Prometheus::Mysqld_exporter::Common]
• Systemd::Service[nrpe2nodexp-mariadb_read_only_s1]
• File[/etc/sudoers.d/nrpe-check_mysqld]
• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_disk_space.timer (nrpe2nodexp-mariadb_disk_space.timer)]
• Puppet::Expose_agent_certs[/etc/mysql]
• Systemd::Timer[nrpe2nodexp-mariadb_replica_sql_lag_s1]
• Nrpe::Monitor_service[mariadb_replica_sql_state_s1]
• Systemd::Timer::Job[nrpe2nodexp-mariadb_read_only_s1]
• Monitoring::Service[mariadb_replica_io_state_s1]
• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_memory.service (nrpe2nodexp-mariadb_memory.service)]
• File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-replica-io-state-s1.conf]
• Profile::Mariadb::Ferm[s1]
• Systemd::Service[nrpe2nodexp-mariadb_replica_sql_state_s1]
• Systemd::Timer::Job[nrpe2nodexp-mariadb_replica_io_state_s1]
• File[/etc/sudoers.d/nrpe-check_mariadb_memory]
• Systemd::Timer::Job[nrpe2nodexp-mariadb_replica_sql_lag_s1]
• File[/lib/systemd/system/nrpe2nodexp-mariadb_read_only_s1.service]
• Sudo::User[nrpe-check_mysqld]
• Systemd::Unit[nrpe2nodexp-mariadb_read_only_s1.timer]
• Exec[systemd daemon-reload for prometheus-mysqld-exporter@.service (prometheus-mysqld-exporter@)]
• Prometheus::Alert::Rule[check_mariadb_disk_space_d2aae7bfec5549be1a8b9413215bd86c]
• File[/lib/systemd/system/nrpe2nodexp-mysqld.timer]
• Prometheus::Mysqld_exporter::Instance[s1]
• Class[Mariadb::Monitor_memory]
• File[/etc/my.cnf]
• File[/etc/mysql/ssl/server-keypair.pem]
• File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-disk-space.conf]
• Nrpe::Check[check_mariadb_memory]
• Class[Profile::Mariadb::Packages_wmf]
• File[/etc/wmfmariadbpy]
• Systemd::Unit[nrpe2nodexp-mysqld.timer]
• Nrpe::Check[check_mariadb_replica_sql_lag_s1]
• Sslcert::X509_to_pkcs12[puppet::expose_agent_cert: /etc/mysql]
• File[/usr/local/etc/my.cnf]
• Mariadb::Monitor_replication[s1]
• Concat_file[/etc/mysql/ssl/cert.pem]
• File[/etc/rsyslog.d/40-wmf-auto-restart-prometheus-mysqld-exporter-s1.conf]
• Rsyslog::Conf[nrpe2nodexp-mariadb_replica_io_state_s1]
• Class[Mariadb::Monitor_disk]
• Firewall::Service[s1_orchestrator]
• Concat::Fragment[/etc/mysql_puppet_ca_chain]
• File[/opt/wmf-mariadb1011/my.cnf]
• File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_lag_s1.timer]
• Sudo::User[nrpe-check_mariadb_memory]
• File[/var/lib/prometheus/node.d/check_mariadb_replica_io_state_s1.prom]
• File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_lag_s1.service]
• Systemd::Timer[nrpe2nodexp-mariadb_read_only_s1]
• File[/etc/mysql/ssl/server-key.pem]
• File[/etc/default/prometheus-mysqld-exporter@s1]
• Class[Passwords::Nagios::Mysql]
• File[/etc/nagios/nrpe.d/check_mariadb_memory.cfg]
• File[/etc/systemd/system/mariadb.service.d]
• File[/etc/update-motd.d/06-db-section-s1]
• File[/etc/mysql/grcat.config]
• Systemd::Service[nrpe2nodexp-mariadb_disk_space]
• File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-memory.conf]
• Class[Profile::Mariadb::Mysql_role]
• Mariadb::Instance[s1]
• Package[wmfmariadbpy-common]
• Class[Role::Mariadb::Backup_source]
• Package[prometheus-mysqld-exporter]
• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_lag_s1.service (nrpe2nodexp-mariadb_replica_sql_lag_s1.service)]
• File[/etc/mysql/mysqld.conf.d]
• Motd::Script[mariadb::backup_source]
• Concat[/etc/mysql/ssl/cert.pem]
• Rsyslog::Conf[nrpe2nodexp-mysqld]
• Service[nrpe2nodexp-mariadb_replica_io_state_s1.timer]
• File[/etc/mysql/ssl/cacert.pem]
• Systemd::Service[wmf_auto_restart_prometheus-mysqld-exporter@s1]
• Systemd::Unit[nrpe2nodexp-mariadb_read_only_s1.service]
• Prometheus::Alert::Rule[check_mariadb_replica_sql_state_s1_3f927914a18db09399b7fff42858d634]
• Systemd::Timer[nrpe2nodexp-mysqld]
• Systemd::Unit[wmf_auto_restart_prometheus-mysqld-exporter@s1.service]
• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_io_state_s1.service (nrpe2nodexp-mariadb_replica_io_state_s1.service)]
• Service[wmf_auto_restart_prometheus-mysqld-exporter@s1.timer]
• Systemd::Unit[nrpe2nodexp-mariadb_replica_sql_lag_s1.service]
• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_read_only_s1.timer (nrpe2nodexp-mariadb_read_only_s1.timer)]
• File[/etc/mysql/ssl/client-key.pem]
• Monitoring::Exported_nagios_service[db2250 mariadb_replica_io_state_s1]
• Systemd::Service[nrpe2nodexp-mariadb_memory]
• File[/etc/logrotate.d/wmf_auto_restart_prometheus-mysqld-exporter_s1]
• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_disk_space.service (nrpe2nodexp-mariadb_disk_space.service)]
• Prometheus::Alert::Rule[check_mariadb_replica_sql_lag_s1_8ec9d6c1155ae60699aaf9dfd1fa339b]
• Systemd::Unit[nrpe2nodexp-mariadb_disk_space.timer]

Resources only in the old catalog

• Motd::Message[insetup::data_persistence_ferm]
• Motd::Script[insetup::data_persistence_ferm]
• File[/etc/update-motd.d/05-insetup--data-persistence-ferm]
• Node[__node_regexp__db225053.codfw.]
• Class[Role::Insetup::Data_persistence_ferm]

Resources modified

• File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-replica-sql-lag-s1.conf]

  Parameters differences:

  --- File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-replica-sql-lag-s1.conf].orig
  +++ File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-replica-sql-lag-s1.conf]
  
  +    group  => root
  +    notify => Service[rsyslog]
  +    owner  => root
  +    ensure => absent
  +    mode   => 0444
  

  Content differences:

  --- /etc/rsyslog.d/25-nrpe2nodexp-mariadb-replica-sql-lag-s1.conf.orig
  +++ /etc/rsyslog.d/25-nrpe2nodexp-mariadb-replica-sql-lag-s1.conf
  @@ -0,0 +1,10 @@
  +# SPDX-License-Identifier: Apache-2.0
  +if $programname contains "nrpe2nodexp-mariadb_replica_sql_lag_s1" then {
  +    if ($msg contains "\"ecs.version\": \"1.7.0\"") then {
  +        # Send logs to kafka
  +        set $.log_outputs = "kafka ecs_170 local";
  +    } else {
  +        # Filter out non-relevant nrpe2nodexp messages
  +        stop
  +    }
  +}

• Systemd::Timer::Job[nrpe2nodexp-mariadb_replica_sql_state_s1]

  Parameters differences:

  --- Systemd::Timer::Job[nrpe2nodexp-mariadb_replica_sql_state_s1].orig
  +++ Systemd::Timer::Job[nrpe2nodexp-mariadb_replica_sql_state_s1]
  
  +    group                     => prometheus-node-exporter
  +    command                   => /usr/local/bin/nrpe2nodexp --alert-rule-hash "3f927914a18db09399b7fff42858d634" --timeout 10 --check-command "check_mariadb_replica_sql_state_s1"
  +    private_tmp               => False
  +    logfile_basedir           => /var/log
  +    user                      => nagios
  +    syslog_identifier         => nrpe2nodexp-mariadb_replica_sql_state_s1
  +    send_mail_to              => root@db2250.codfw.wmnet
  +    success_exit_status       => []
  +    fixed_random_delay        => True
  +    description               => execution of nrpe2nodexp for the check_mariadb_replica_sql_state_s1 command.
  +    ensure                    => absent
  +    logfile_group             => root
  +    monitoring_enabled        => False
  +    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
  +    syslog_match_startswith   => True
  +    interval                  => [{'start': 'OnUnitInactiveSec', 'interval': '1min'}]
  +    ignore_errors             => True
  +    send_mail_only_on_error   => True
  +    logfile_name              => syslog.log
  +    send_mail                 => False
  +    splay                     => 60
  +    syslog_force_stop         => True
  +    logfile_perms             => all
  +    environment               => {}
  +    logging_enabled           => False
  +    monitoring_contact_groups => admins
  

• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_read_only_s1.service (nrpe2nodexp-mariadb_read_only_s1.service)]

  Parameters differences:

  --- Exec[systemd daemon-reload for nrpe2nodexp-mariadb_read_only_s1.service (nrpe2nodexp-mariadb_read_only_s1.service)].orig
  +++ Exec[systemd daemon-reload for nrpe2nodexp-mariadb_read_only_s1.service (nrpe2nodexp-mariadb_read_only_s1.service)]
  
  +    refreshonly => True
  +    command     => /bin/systemctl daemon-reload
  

• Package[python3-wmfmariadbpy]

  Parameters differences:

  --- Package[python3-wmfmariadbpy].orig
  +++ Package[python3-wmfmariadbpy]
  
  +    provider => apt
  +    ensure   => installed
  

• File[/etc/nagios/nrpe.d/check_mariadb_replica_sql_lag_s1.cfg]

  Parameters differences:

  --- File[/etc/nagios/nrpe.d/check_mariadb_replica_sql_lag_s1.cfg].orig
  +++ File[/etc/nagios/nrpe.d/check_mariadb_replica_sql_lag_s1.cfg]
  
  +    group   => root
  +    notify  => Service[nagios-nrpe-server]
  +    owner   => root
  +    require => Package[nagios-nrpe-server]
  +    ensure  => present
  +    tag     => nrpe::check
  +    mode    => 0444
  

  Content differences:

  --- /etc/nagios/nrpe.d/check_mariadb_replica_sql_lag_s1.cfg.orig
  +++ /etc/nagios/nrpe.d/check_mariadb_replica_sql_lag_s1.cfg
  @@ -0,0 +1,2 @@
  +# File generated by puppet. DO NOT edit by hand
  +command[check_mariadb_replica_sql_lag_s1]=/usr/local/lib/nagios/plugins/check_mariadb --sock=/run/mysqld/mysqld.s1.sock --user=nagios --pass=  --warn-stopped --check=slave_sql_lag                            --shard=s1 --datacenter=eqiad                            --sql-lag-warn=60                            --sql-lag-crit=300

• File[/etc/rsyslog.d/25-nrpe2nodexp-mysqld.conf]

  Parameters differences:

  --- File[/etc/rsyslog.d/25-nrpe2nodexp-mysqld.conf].orig
  +++ File[/etc/rsyslog.d/25-nrpe2nodexp-mysqld.conf]
  
  +    group  => root
  +    notify => Service[rsyslog]
  +    owner  => root
  +    ensure => absent
  +    mode   => 0444
  

  Content differences:

  --- /etc/rsyslog.d/25-nrpe2nodexp-mysqld.conf.orig
  +++ /etc/rsyslog.d/25-nrpe2nodexp-mysqld.conf
  @@ -0,0 +1,10 @@
  +# SPDX-License-Identifier: Apache-2.0
  +if $programname contains "nrpe2nodexp-mysqld" then {
  +    if ($msg contains "\"ecs.version\": \"1.7.0\"") then {
  +        # Send logs to kafka
  +        set $.log_outputs = "kafka ecs_170 local";
  +    } else {
  +        # Filter out non-relevant nrpe2nodexp messages
  +        stop
  +    }
  +}

• Sudo::User[nrpe-check_mariadb_memory]

  Parameters differences:

  --- Sudo::User[nrpe-check_mariadb_memory].orig
  +++ Sudo::User[nrpe-check_mariadb_memory]
  
  +    privileges => []
  +    require    => ['Class[Sudo]']
  +    ensure     => absent
  +    tag        => nrpe::check
  +    user       => nagios
  

• File[/etc/update-motd.d/06-db-section-s1]

  Parameters differences:

  --- File[/etc/update-motd.d/06-db-section-s1].orig
  +++ File[/etc/update-motd.d/06-db-section-s1]
  
  +    group  => root
  +    owner  => root
  +    mode   => 0555
  +    ensure => present
  

  Content differences:

  --- /etc/update-motd.d/06-db-section-s1.orig
  +++ /etc/update-motd.d/06-db-section-s1
  @@ -0,0 +1,2 @@
  +#!/bin/sh
  +echo 'DB section s1 (alias: mysql.s1)'

• Systemd::Service[nrpe2nodexp-mariadb_disk_space]

  Parameters differences:

  --- Systemd::Service[nrpe2nodexp-mariadb_disk_space].orig
  +++ Systemd::Service[nrpe2nodexp-mariadb_disk_space]
  
  +    monitoring_enabled       => False
  +    require                  => Systemd::Unit[nrpe2nodexp-mariadb_disk_space.service]
  +    restart                  => False
  +    unit_type                => timer
  +    service_params           => {}
  +    monitoring_contact_group => admins
  +    monitoring_critical      => False
  +    migration_task           => T407130
  +    ensure                   => absent
  +    override                 => False
  

• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_read_only_s1.timer (nrpe2nodexp-mariadb_read_only_s1.timer)]

  Parameters differences:

  --- Exec[systemd daemon-reload for nrpe2nodexp-mariadb_read_only_s1.timer (nrpe2nodexp-mariadb_read_only_s1.timer)].orig
  +++ Exec[systemd daemon-reload for nrpe2nodexp-mariadb_read_only_s1.timer (nrpe2nodexp-mariadb_read_only_s1.timer)]
  
  +    refreshonly => True
  +    command     => /bin/systemctl daemon-reload
  

• Nrpe::Monitor_service[mariadb_memory]

  Parameters differences:

  --- Nrpe::Monitor_service[mariadb_memory].orig
  +++ Nrpe::Monitor_service[mariadb_memory]
  
  +    enable_nrpe2nodexp          => False
  +    alertmanager_team           => observability
  +    critical                    => False
  +    enable_icinga_check         => True
  +    description                 => MariaDB memory
  +    notes_url                   => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting
  +    nrpe2nodexp_parse_perf_data => False
  +    migration_task              => T315866
  +    ensure                      => present
  +    retry_interval              => 1
  +    timeout                     => 10
  +    require                     => Package[monitoring-plugins-contrib]
  +    contact_group               => admins
  +    check_interval              => 1
  +    nrpe_command                => /usr/lib/nagios/plugins/pmp-check-unix-memory -c 95 -w 90
  +    retries                     => 3
  

• Systemd::Timer::Job[wmf_auto_restart_prometheus-mysqld-exporter@s1]

  Parameters differences:

  --- Systemd::Timer::Job[wmf_auto_restart_prometheus-mysqld-exporter@s1].orig
  +++ Systemd::Timer::Job[wmf_auto_restart_prometheus-mysqld-exporter@s1]
  
  +    command                   => /usr/local/sbin/wmf-auto-restart -s prometheus-mysqld-exporter@s1
  +    private_tmp               => False
  +    logfile_basedir           => /var/log
  +    user                      => root
  +    send_mail_to              => root@db2250.codfw.wmnet
  +    success_exit_status       => []
  +    fixed_random_delay        => False
  +    description               => Auto restart job: prometheus-mysqld-exporter@s1
  +    ensure                    => present
  +    logfile_group             => root
  +    monitoring_enabled        => False
  +    require                   => File[/usr/local/sbin/wmf-auto-restart]
  +    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
  +    syslog_match_startswith   => True
  +    interval                  => {'start': 'OnCalendar', 'interval': 'Mon,Tue,Wed,Thu,Fri *-*-* 0:55:00'}
  +    ignore_errors             => False
  +    send_mail_only_on_error   => True
  +    send_mail                 => False
  +    syslog_force_stop         => True
  +    logfile_name              => syslog.log
  +    logfile_perms             => all
  +    environment               => {}
  +    logging_enabled           => True
  +    monitoring_contact_groups => admins
  

• Nrpe::Check[check_mariadb_memory]

  Parameters differences:

  --- Nrpe::Check[check_mariadb_memory].orig
  +++ Nrpe::Check[check_mariadb_memory]
  
  +    before  => Monitoring::Service[mariadb_memory]
  +    command => /usr/lib/nagios/plugins/pmp-check-unix-memory -c 95 -w 90
  +    ensure  => present
  

• Concat_file[/etc/mysql/ssl/cert.pem]

  Parameters differences:

  --- Concat_file[/etc/mysql/ssl/cert.pem].orig
  +++ Concat_file[/etc/mysql/ssl/cert.pem]
  
  +    replace        => True
  +    mode           => 0644
  +    show_diff      => True
  +    order          => alpha
  +    force          => False
  +    ensure_newline => False
  +    tag            => _etc_mysql_ssl_cert.pem
  +    format         => plain
  +    backup         => puppet
  

• File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_lag_s1.timer]

  Parameters differences:

  --- File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_lag_s1.timer].orig
  +++ File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_lag_s1.timer]
  
  +    group  => root
  +    notify => Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_lag_s1.timer (nrpe2nodexp-mariadb_replica_sql_lag_s1.timer)]
  +    owner  => root
  +    ensure => absent
  +    mode   => 0444
  

  Content differences:

  --- /lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_lag_s1.timer.orig
  +++ /lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_lag_s1.timer
  @@ -0,0 +1,14 @@
  +[Unit]
  +Description=Periodic execution of nrpe2nodexp-mariadb_replica_sql_lag_s1.service
  +
  +[Timer]
  +Unit=nrpe2nodexp-mariadb_replica_sql_lag_s1.service
  +# Accuracy sets the maximum time interval around the execution time we want to allow
  +AccuracySec=15sec
  +OnUnitInactiveSec=1min
  +OnActiveSec=1s
  +RandomizedDelaySec=60
  +FixedRandomDelay=true
  +
  +[Install]
  +WantedBy=multi-user.target

• File[/etc/mysql/mysqld.conf.d]

  Parameters differences:

  --- File[/etc/mysql/mysqld.conf.d].orig
  +++ File[/etc/mysql/mysqld.conf.d]
  
  +    group  => root
  +    owner  => root
  +    mode   => 0755
  +    ensure => directory
  

• Exec[systemd daemon-reload for nrpe2nodexp-mysqld.timer (nrpe2nodexp-mysqld.timer)]

  Parameters differences:

  --- Exec[systemd daemon-reload for nrpe2nodexp-mysqld.timer (nrpe2nodexp-mysqld.timer)].orig
  +++ Exec[systemd daemon-reload for nrpe2nodexp-mysqld.timer (nrpe2nodexp-mysqld.timer)]
  
  +    refreshonly => True
  +    command     => /bin/systemctl daemon-reload
  

• Concat::Fragment[main contacts]

• File[/etc/mysql/ssl/client-cert.pem]

  Parameters differences:

  --- File[/etc/mysql/ssl/client-cert.pem].orig
  +++ File[/etc/mysql/ssl/client-cert.pem]
  
  +    group  => root
  +    owner  => root
  +    ensure => absent
  

• File[/etc/nagios/nrpe.d/check_mariadb_read_only_s1.cfg]

  Parameters differences:

  --- File[/etc/nagios/nrpe.d/check_mariadb_read_only_s1.cfg].orig
  +++ File[/etc/nagios/nrpe.d/check_mariadb_read_only_s1.cfg]
  
  +    group   => root
  +    notify  => Service[nagios-nrpe-server]
  +    owner   => root
  +    require => Package[nagios-nrpe-server]
  +    ensure  => present
  +    tag     => nrpe::check
  +    mode    => 0444
  

  Content differences:

  --- /etc/nagios/nrpe.d/check_mariadb_read_only_s1.cfg.orig
  +++ /etc/nagios/nrpe.d/check_mariadb_read_only_s1.cfg
  @@ -0,0 +1,2 @@
  +# File generated by puppet. DO NOT edit by hand
  +command[check_mariadb_read_only_s1]=db-check-health --port=3311 --icinga --check_read_only=1 --process

• Exec[systemd daemon-reload for prometheus-mysqld-exporter@.service (prometheus-mysqld-exporter@)]

  Parameters differences:

  --- Exec[systemd daemon-reload for prometheus-mysqld-exporter@.service (prometheus-mysqld-exporter@)].orig
  +++ Exec[systemd daemon-reload for prometheus-mysqld-exporter@.service (prometheus-mysqld-exporter@)]
  
  +    refreshonly => True
  +    command     => /bin/systemctl daemon-reload
  

• File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-disk-space.conf]

  Parameters differences:

  --- File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-disk-space.conf].orig
  +++ File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-disk-space.conf]
  
  +    group  => root
  +    notify => Service[rsyslog]
  +    owner  => root
  +    ensure => absent
  +    mode   => 0444
  

  Content differences:

  --- /etc/rsyslog.d/25-nrpe2nodexp-mariadb-disk-space.conf.orig
  +++ /etc/rsyslog.d/25-nrpe2nodexp-mariadb-disk-space.conf
  @@ -0,0 +1,10 @@
  +# SPDX-License-Identifier: Apache-2.0
  +if $programname contains "nrpe2nodexp-mariadb_disk_space" then {
  +    if ($msg contains "\"ecs.version\": \"1.7.0\"") then {
  +        # Send logs to kafka
  +        set $.log_outputs = "kafka ecs_170 local";
  +    } else {
  +        # Filter out non-relevant nrpe2nodexp messages
  +        stop
  +    }
  +}

• Motd::Message[insetup::data_persistence_ferm]

  Parameters differences:

  --- Motd::Message[insetup::data_persistence_ferm].orig
  +++ Motd::Message[insetup::data_persistence_ferm]
  
  -    message  => db2250 is a Host being setup by Data Persistence SREs (insetup::data_persistence_ferm)
  -    priority => 5
  -    ensure   => present
  

• Systemd::Unit[nrpe2nodexp-mariadb_replica_io_state_s1.service]

  Parameters differences:

  --- Systemd::Unit[nrpe2nodexp-mariadb_replica_io_state_s1.service].orig
  +++ Systemd::Unit[nrpe2nodexp-mariadb_replica_io_state_s1.service]
  
  +    unit              => nrpe2nodexp-mariadb_replica_io_state_s1.service
  +    require           => ['Class[Systemd]']
  +    restart           => False
  +    ensure            => absent
  +    override          => False
  +    override_filename => puppet-override.conf
  

• Profile::Auto_restarts::Service[prometheus-mysqld-exporter@s1]

  Parameters differences:

  --- Profile::Auto_restarts::Service[prometheus-mysqld-exporter@s1].orig
  +++ Profile::Auto_restarts::Service[prometheus-mysqld-exporter@s1]
  
  +    ensure => present
  

• Class[Mariadb::Config]

  Parameters differences:

  --- Class[Mariadb::Config].orig
  +++ Class[Mariadb::Config]
  
  +    sql_mode                => 
  +    semi_sync               => off
  +    config                  => profile/mariadb/mysqld_config/dbstore_multiinstance.my.cnf.erb
  +    sync_binlog             => 1
  +    datadir                 => False
  +    innodb_change_buffering => none
  +    ssl_ca                  => 
  +    replication_role        => standalone
  +    password                => undefined
  +    ssl                     => puppet-cert
  +    p_s                     => on
  +    binlog_format           => ROW
  +    ssl_key                 => 
  +    expire_logs_days        => 30
  +    prompt                  => \u@\h(\d)>\_
  +    tmpdir                  => /srv/tmp
  +    socket                  => /run/mysqld/mysqld.sock
  +    event_scheduler         => 1
  +    ssl_cert                => 
  +    flush_log_at_trx_commit => 1
  +    max_allowed_packet      => 16M
  +    port                    => 3306
  +    ssl_verify_server_cert  => True
  +    read_only               => 1
  +    basedir                 => /opt/wmf-mariadb1011
  

• Systemd::Syslog[wmf_auto_restart_prometheus-mysqld-exporter_s1]

  Parameters differences:

  --- Systemd::Syslog[wmf_auto_restart_prometheus-mysqld-exporter_s1].orig
  +++ Systemd::Syslog[wmf_auto_restart_prometheus-mysqld-exporter_s1]
  
  +    group                  => root
  +    owner                  => root
  +    readable_by            => all
  +    log_filename           => syslog.log
  +    force_stop             => True
  +    programname_comparison => startswith
  +    ensure                 => present
  +    base_dir               => /var/log
  

• File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-replica-sql-state-s1.conf]

  Parameters differences:

  --- File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-replica-sql-state-s1.conf].orig
  +++ File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-replica-sql-state-s1.conf]
  
  +    group  => root
  +    notify => Service[rsyslog]
  +    owner  => root
  +    ensure => absent
  +    mode   => 0444
  

  Content differences:

  --- /etc/rsyslog.d/25-nrpe2nodexp-mariadb-replica-sql-state-s1.conf.orig
  +++ /etc/rsyslog.d/25-nrpe2nodexp-mariadb-replica-sql-state-s1.conf
  @@ -0,0 +1,10 @@
  +# SPDX-License-Identifier: Apache-2.0
  +if $programname contains "nrpe2nodexp-mariadb_replica_sql_state_s1" then {
  +    if ($msg contains "\"ecs.version\": \"1.7.0\"") then {
  +        # Send logs to kafka
  +        set $.log_outputs = "kafka ecs_170 local";
  +    } else {
  +        # Filter out non-relevant nrpe2nodexp messages
  +        stop
  +    }
  +}

• File[/var/lib/prometheus/node.d/check_mariadb_memory.prom]

  Parameters differences:

  --- File[/var/lib/prometheus/node.d/check_mariadb_memory.prom].orig
  +++ File[/var/lib/prometheus/node.d/check_mariadb_memory.prom]
  
  +    group  => root
  +    owner  => root
  +    ensure => absent
  

• File[/srv/tmp.s1]

  Parameters differences:

  --- File[/srv/tmp.s1].orig
  +++ File[/srv/tmp.s1]
  
  +    group  => mysql
  +    owner  => mysql
  +    mode   => 0755
  +    ensure => directory
  

• Motd::Message[mariadb::backup_source]

  Parameters differences:

  --- Motd::Message[mariadb::backup_source].orig
  +++ Motd::Message[mariadb::backup_source]
  
  +    message  => db2250 is a MariaDB server containing replicas of mediawiki databases used to generate backups (mariadb::backup_source)
  +    priority => 5
  +    ensure   => present
  

• File[/etc/mysql/ssl/cacert.pem]

  Parameters differences:

  --- File[/etc/mysql/ssl/cacert.pem].orig
  +++ File[/etc/mysql/ssl/cacert.pem]
  
  +    group     => mysql
  +    owner     => root
  +    show_diff => False
  +    ensure    => file
  +    backup    => False
  +    mode      => 0440
  

• Monitoring::Exported_nagios_service[db2250 mariadb_replica_io_state_s1]

  Parameters differences:

  --- Monitoring::Exported_nagios_service[db2250 mariadb_replica_io_state_s1].orig
  +++ Monitoring::Exported_nagios_service[db2250 mariadb_replica_io_state_s1]
  
  +    check_command          => nrpe_check!check_mariadb_replica_io_state_s1!10
  +    notification_period    => 24x7
  +    notification_interval  => 0
  +    check_period           => 24x7
  +    service_description    => MariaDB Replica IO: s1
  +    max_check_attempts     => 3
  +    notes_url              => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica
  +    notifications_enabled  => 0
  +    is_volatile            => 0
  +    retry_interval         => 1
  +    ensure                 => present
  +    active_checks_enabled  => 1
  +    notification_options   => c,r,f
  +    contact_groups         => admins
  +    check_interval         => 1
  +    servicegroups          => mysql_codfw
  +    passive_checks_enabled => 1
  +    host_name              => db2250
  +    check_freshness        => 0
  

• Nrpe::Monitor_service[mysqld]

  Parameters differences:

  --- Nrpe::Monitor_service[mysqld].orig
  +++ Nrpe::Monitor_service[mysqld]
  
  +    enable_nrpe2nodexp          => False
  +    alertmanager_team           => observability
  +    critical                    => False
  +    contact_group               => admins
  +    enable_icinga_check         => True
  +    check_interval              => 1
  +    description                 => mysqld processes
  +    notes_url                   => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting
  +    nrpe_command                => /usr/lib/nagios/plugins/check_procs -c 1:1 -C mysqld
  +    nrpe2nodexp_parse_perf_data => False
  +    migration_task              => T315866
  +    ensure                      => present
  +    retry_interval              => 1
  +    retries                     => 3
  +    timeout                     => 10
  

• File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_io_state_s1.service]

  Parameters differences:

  --- File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_io_state_s1.service].orig
  +++ File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_io_state_s1.service]
  
  +    group  => root
  +    notify => Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_io_state_s1.service (nrpe2nodexp-mariadb_replica_io_state_s1.service)]
  +    owner  => root
  +    ensure => absent
  +    mode   => 0444
  

  Content differences:

  --- /lib/systemd/system/nrpe2nodexp-mariadb_replica_io_state_s1.service.orig
  +++ /lib/systemd/system/nrpe2nodexp-mariadb_replica_io_state_s1.service
  @@ -0,0 +1,11 @@
  +[Unit]
  +Description=execution of nrpe2nodexp for the check_mariadb_replica_io_state_s1 command.
  +Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
  +
  +[Service]
  +Type=oneshot
  +User=nagios
  +
  +Group=prometheus-node-exporter
  +SyslogIdentifier=nrpe2nodexp-mariadb_replica_io_state_s1
  +ExecStart=-/usr/local/bin/nrpe2nodexp --alert-rule-hash "0d5979f7abbb49256cfedd70a646784e" --timeout 10 --check-command "check_mariadb_replica_io_state_s1"

• Systemd::Service[nrpe2nodexp-mariadb_replica_sql_state_s1]

  Parameters differences:

  --- Systemd::Service[nrpe2nodexp-mariadb_replica_sql_state_s1].orig
  +++ Systemd::Service[nrpe2nodexp-mariadb_replica_sql_state_s1]
  
  +    monitoring_enabled       => False
  +    require                  => Systemd::Unit[nrpe2nodexp-mariadb_replica_sql_state_s1.service]
  +    restart                  => False
  +    unit_type                => timer
  +    service_params           => {}
  +    monitoring_contact_group => admins
  +    monitoring_critical      => False
  +    migration_task           => T407130
  +    ensure                   => absent
  +    override                 => False
  

• File[/lib/systemd/system/nrpe2nodexp-mariadb_read_only_s1.service]

  Parameters differences:

  --- File[/lib/systemd/system/nrpe2nodexp-mariadb_read_only_s1.service].orig
  +++ File[/lib/systemd/system/nrpe2nodexp-mariadb_read_only_s1.service]
  
  +    group  => root
  +    notify => Exec[systemd daemon-reload for nrpe2nodexp-mariadb_read_only_s1.service (nrpe2nodexp-mariadb_read_only_s1.service)]
  +    owner  => root
  +    ensure => absent
  +    mode   => 0444
  

  Content differences:

  --- /lib/systemd/system/nrpe2nodexp-mariadb_read_only_s1.service.orig
  +++ /lib/systemd/system/nrpe2nodexp-mariadb_read_only_s1.service
  @@ -0,0 +1,11 @@
  +[Unit]
  +Description=execution of nrpe2nodexp for the check_mariadb_read_only_s1 command.
  +Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
  +
  +[Service]
  +Type=oneshot
  +User=nagios
  +
  +Group=prometheus-node-exporter
  +SyslogIdentifier=nrpe2nodexp-mariadb_read_only_s1
  +ExecStart=-/usr/local/bin/nrpe2nodexp --alert-rule-hash "42baf529e0f3bfca391aca0a24c880e9" --timeout 10 --check-command "check_mariadb_read_only_s1"

• File[/etc/default/prometheus-mysqld-exporter@s1]

  Parameters differences:

  --- File[/etc/default/prometheus-mysqld-exporter@s1].orig
  +++ File[/etc/default/prometheus-mysqld-exporter@s1]
  
  +    group  => root
  +    notify => Exec[systemctl try-restart prometheus-mysqld-exporter@s1]
  +    owner  => root
  +    ensure => present
  +    mode   => 0444
  

  Content differences:

  --- /etc/default/prometheus-mysqld-exporter@s1.orig
  +++ /etc/default/prometheus-mysqld-exporter@s1
  @@ -0,0 +1 @@
  +ARGS='--web.listen-address ":13311" --config.my-cnf "/var/lib/prometheus/.my.s1.cnf" --collect.global_status --collect.global_variables --collect.info_schema.processlist --collect.slave_status --collect.heartbeat --collect.heartbeat.utc --no-collect.info_schema.tables'

• Motd::Script[mariadb::backup_source]

  Parameters differences:

  --- Motd::Script[mariadb::backup_source].orig
  +++ Motd::Script[mariadb::backup_source]
  
  +    ensure   => present
  +    priority => 5
  

• Prometheus::Alert::Rule[check_mysqld_ca81db2c9e98d408f9f65c79817da116]

  Parameters differences:

  --- Prometheus::Alert::Rule[check_mysqld_ca81db2c9e98d408f9f65c79817da116].orig
  +++ Prometheus::Alert::Rule[check_mysqld_ca81db2c9e98d408f9f65c79817da116]
  
  +    group              => nrpechecks
  +    team               => observability
  +    for                => 3m
  +    site               => codfw
  +    severity           => info
  +    dashboard          => TODO
  +    summary            => NRPE CHECK: mysqld processes
  +    logs               => https://logstash.wikimedia.org/app/dashboards#/view/2d343ac0-6df8-11f0-8e08-7fab0da52b33?_g=(filters:!((query:(match_phrase:(event.module:check_mysqld))),(query:(match_phrase:(host.name:{{$labels.instance|stripPort}})))))
  +    instance           => ops
  +    alert_name         => nrpe_mysqld_processes
  +    description        => NRPE CHECK: mysqld processes
  +    runbook            => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting
  +    ensure             => absent
  +    def_label_whitelst => ['team', 'severity']
  +    expr               => (nagios_nrpe_check_result{alert_rule_hash="ca81db2c9e98d408f9f65c79817da116",check_name="check_mysqld", status=~"(WARNING|CRITICAL)", severity=~"(warning|critical)"} > 0) * on (instance) group_left (team) role_owner
  

• Concat_fragment[/etc/mysql_puppet_ca_chain]

  Parameters differences:

  --- Concat_fragment[/etc/mysql_puppet_ca_chain].orig
  +++ Concat_fragment[/etc/mysql_puppet_ca_chain]
  
  +    tag    => _etc_mysql_ssl_cert.pem
  +    order  => 02
  +    source => /var/lib/puppet/ssl/certs/ca.pem
  +    target => /etc/mysql/ssl/cert.pem
  

• Class[Profile::Mariadb::Wmfmariadbpy]

  Parameters differences:

  --- Class[Profile::Mariadb::Wmfmariadbpy].orig
  +++ Class[Profile::Mariadb::Wmfmariadbpy]
  
  +    role          => db
  +    section_ports => {'s1': 3311, 's2': 3312, 's3': 3313, 's4': 3314, 's5': 3315, 's6': 3316, 's7': 3317, 's8': 3318, 'x1': 3320, 'x3': 3363, 'm1': 3321, 'm2': 3322, 'm3': 3323, 'm5': 3325, 'staging': 3350, 'matomo': 3351, 'analytics_meta': 3352, 'backup1-eqiad': 3360, 'backup1-codfw': 3361}
  

• Systemd::Unit[nrpe2nodexp-mariadb_memory.service]

  Parameters differences:

  --- Systemd::Unit[nrpe2nodexp-mariadb_memory.service].orig
  +++ Systemd::Unit[nrpe2nodexp-mariadb_memory.service]
  
  +    unit              => nrpe2nodexp-mariadb_memory.service
  +    require           => ['Class[Systemd]']
  +    restart           => False
  +    ensure            => absent
  +    override          => False
  +    override_filename => puppet-override.conf
  

• Monitoring::Service[mariadb_disk_space]

  Parameters differences:

  --- Monitoring::Service[mariadb_disk_space].orig
  +++ Monitoring::Service[mariadb_disk_space]
  
  +    freshness      => 36000
  +    config_dir     => /etc/nagios
  +    passive        => False
  +    check_command  => nrpe_check!check_mariadb_disk_space!10
  +    host           => db2250
  +    critical       => False
  +    contact_group  => admins
  +    check_interval => 1
  +    description    => MariaDB disk space
  +    notes_url      => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting
  +    retry_interval => 1
  +    ensure         => present
  +    migration_task => T315866
  +    retries        => 3
  

• File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_state_s1.timer]

  Parameters differences:

  --- File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_state_s1.timer].orig
  +++ File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_state_s1.timer]
  
  +    group  => root
  +    notify => Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_state_s1.timer (nrpe2nodexp-mariadb_replica_sql_state_s1.timer)]
  +    owner  => root
  +    ensure => absent
  +    mode   => 0444
  

  Content differences:

  --- /lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_state_s1.timer.orig
  +++ /lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_state_s1.timer
  @@ -0,0 +1,14 @@
  +[Unit]
  +Description=Periodic execution of nrpe2nodexp-mariadb_replica_sql_state_s1.service
  +
  +[Timer]
  +Unit=nrpe2nodexp-mariadb_replica_sql_state_s1.service
  +# Accuracy sets the maximum time interval around the execution time we want to allow
  +AccuracySec=15sec
  +OnUnitInactiveSec=1min
  +OnActiveSec=1s
  +RandomizedDelaySec=60
  +FixedRandomDelay=true
  +
  +[Install]
  +WantedBy=multi-user.target

• Prometheus::Alert::Rule[check_mariadb_disk_space_d2aae7bfec5549be1a8b9413215bd86c]

  Parameters differences:

  --- Prometheus::Alert::Rule[check_mariadb_disk_space_d2aae7bfec5549be1a8b9413215bd86c].orig
  +++ Prometheus::Alert::Rule[check_mariadb_disk_space_d2aae7bfec5549be1a8b9413215bd86c]
  
  +    group              => nrpechecks
  +    team               => observability
  +    for                => 3m
  +    site               => codfw
  +    severity           => info
  +    dashboard          => TODO
  +    summary            => NRPE CHECK: MariaDB disk space
  +    logs               => https://logstash.wikimedia.org/app/dashboards#/view/2d343ac0-6df8-11f0-8e08-7fab0da52b33?_g=(filters:!((query:(match_phrase:(event.module:check_mariadb_disk_space))),(query:(match_phrase:(host.name:{{$labels.instance|stripPort}})))))
  +    instance           => ops
  +    alert_name         => nrpe_MariaDB_disk_space
  +    description        => NRPE CHECK: MariaDB disk space
  +    runbook            => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting
  +    ensure             => absent
  +    def_label_whitelst => ['team', 'severity']
  +    expr               => (nagios_nrpe_check_result{alert_rule_hash="d2aae7bfec5549be1a8b9413215bd86c",check_name="check_mariadb_disk_space", status=~"(WARNING|CRITICAL)", severity=~"(warning|critical)"} > 0) * on (instance) group_left (team) role_owner
  

• File[/lib/systemd/system/nrpe2nodexp-mysqld.timer]

  Parameters differences:

  --- File[/lib/systemd/system/nrpe2nodexp-mysqld.timer].orig
  +++ File[/lib/systemd/system/nrpe2nodexp-mysqld.timer]
  
  +    group  => root
  +    notify => Exec[systemd daemon-reload for nrpe2nodexp-mysqld.timer (nrpe2nodexp-mysqld.timer)]
  +    owner  => root
  +    ensure => absent
  +    mode   => 0444
  

  Content differences:

  --- /lib/systemd/system/nrpe2nodexp-mysqld.timer.orig
  +++ /lib/systemd/system/nrpe2nodexp-mysqld.timer
  @@ -0,0 +1,14 @@
  +[Unit]
  +Description=Periodic execution of nrpe2nodexp-mysqld.service
  +
  +[Timer]
  +Unit=nrpe2nodexp-mysqld.service
  +# Accuracy sets the maximum time interval around the execution time we want to allow
  +AccuracySec=15sec
  +OnUnitInactiveSec=1min
  +OnActiveSec=1s
  +RandomizedDelaySec=60
  +FixedRandomDelay=true
  +
  +[Install]
  +WantedBy=multi-user.target

• File[/etc/rsyslog.d/40-wmf-auto-restart-prometheus-mysqld-exporter-s1.conf]

  Parameters differences:

  --- File[/etc/rsyslog.d/40-wmf-auto-restart-prometheus-mysqld-exporter-s1.conf].orig
  +++ File[/etc/rsyslog.d/40-wmf-auto-restart-prometheus-mysqld-exporter-s1.conf]
  
  +    group  => root
  +    notify => Service[rsyslog]
  +    owner  => root
  +    ensure => present
  +    mode   => 0444
  

  Content differences:

  --- /etc/rsyslog.d/40-wmf-auto-restart-prometheus-mysqld-exporter-s1.conf.orig
  +++ /etc/rsyslog.d/40-wmf-auto-restart-prometheus-mysqld-exporter-s1.conf
  @@ -0,0 +1,10 @@
  +# rsyslog.conf(5) configuration file for services.
  +# This file is managed by Puppet.
  +if $programname startswith "wmf_auto_restart_prometheus-mysqld-exporter_s1" then {
  +    action(
  +        type="omfile" file="/var/log/wmf_auto_restart_prometheus-mysqld-exporter_s1/syslog.log"
  +        fileOwner="root" fileGroup="root"
  +        fileCreateMode="0644"
  +    )
  +    & stop
  +}

• Concat::Fragment[/etc/mysql_puppet_ca_chain]

  Parameters differences:

  --- Concat::Fragment[/etc/mysql_puppet_ca_chain].orig
  +++ Concat::Fragment[/etc/mysql_puppet_ca_chain]
  
  +    order  => 02
  +    source => /var/lib/puppet/ssl/certs/ca.pem
  +    target => /etc/mysql/ssl/cert.pem
  

• Systemd::Unit[nrpe2nodexp-mariadb_replica_sql_lag_s1.service]

  Parameters differences:

  --- Systemd::Unit[nrpe2nodexp-mariadb_replica_sql_lag_s1.service].orig
  +++ Systemd::Unit[nrpe2nodexp-mariadb_replica_sql_lag_s1.service]
  
  +    unit              => nrpe2nodexp-mariadb_replica_sql_lag_s1.service
  +    require           => ['Class[Systemd]']
  +    restart           => False
  +    ensure            => absent
  +    override          => False
  +    override_filename => puppet-override.conf
  

• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_disk_space.service (nrpe2nodexp-mariadb_disk_space.service)]

  Parameters differences:

  --- Exec[systemd daemon-reload for nrpe2nodexp-mariadb_disk_space.service (nrpe2nodexp-mariadb_disk_space.service)].orig
  +++ Exec[systemd daemon-reload for nrpe2nodexp-mariadb_disk_space.service (nrpe2nodexp-mariadb_disk_space.service)]
  
  +    refreshonly => True
  +    command     => /bin/systemctl daemon-reload
  

• Package[wmf-mariadb1011]

  Parameters differences:

  --- Package[wmf-mariadb1011].orig
  +++ Package[wmf-mariadb1011]
  
  +    provider => apt
  +    ensure   => installed
  

• Class[Mariadb::Monitor_process]

  Parameters differences:

  --- Class[Mariadb::Monitor_process].orig
  +++ Class[Mariadb::Monitor_process]
  
  +    process_count => 1
  +    is_critical   => False
  +    contact_group => admins
  +    process_name  => mysqld
  

• Systemd::Timer::Job[nrpe2nodexp-mysqld]

  Parameters differences:

  --- Systemd::Timer::Job[nrpe2nodexp-mysqld].orig
  +++ Systemd::Timer::Job[nrpe2nodexp-mysqld]
  
  +    group                     => prometheus-node-exporter
  +    command                   => /usr/local/bin/nrpe2nodexp --alert-rule-hash "ca81db2c9e98d408f9f65c79817da116" --timeout 10 --check-command "check_mysqld"
  +    private_tmp               => False
  +    logfile_basedir           => /var/log
  +    user                      => nagios
  +    syslog_identifier         => nrpe2nodexp-mysqld
  +    send_mail_to              => root@db2250.codfw.wmnet
  +    success_exit_status       => []
  +    fixed_random_delay        => True
  +    description               => execution of nrpe2nodexp for the check_mysqld command.
  +    ensure                    => absent
  +    logfile_group             => root
  +    monitoring_enabled        => False
  +    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
  +    syslog_match_startswith   => True
  +    interval                  => [{'start': 'OnUnitInactiveSec', 'interval': '1min'}]
  +    ignore_errors             => True
  +    send_mail_only_on_error   => True
  +    logfile_name              => syslog.log
  +    send_mail                 => False
  +    splay                     => 60
  +    syslog_force_stop         => True
  +    logfile_perms             => all
  +    environment               => {}
  +    logging_enabled           => False
  +    monitoring_contact_groups => admins
  

• Service[nrpe2nodexp-mariadb_disk_space.timer]

  Parameters differences:

  --- Service[nrpe2nodexp-mariadb_disk_space.timer].orig
  +++ Service[nrpe2nodexp-mariadb_disk_space.timer]
  
  +    before   => ['Exec[systemd daemon-reload for nrpe2nodexp-mariadb_disk_space.timer (nrpe2nodexp-mariadb_disk_space.timer)]']
  +    provider => systemd
  +    enable   => False
  +    ensure   => stopped
  

• Systemd::Unit[prometheus-mysqld-exporter@]

  Parameters differences:

  --- Systemd::Unit[prometheus-mysqld-exporter@].orig
  +++ Systemd::Unit[prometheus-mysqld-exporter@]
  
  +    unit              => prometheus-mysqld-exporter@
  +    require           => ['Package[prometheus-mysqld-exporter]', 'Class[Systemd]']
  +    restart           => False
  +    ensure            => present
  +    override          => False
  +    override_filename => puppet-override.conf
  

• Ferm::Service[s1_mariadb_internal]

  Parameters differences:

  --- Ferm::Service[s1_mariadb_internal].orig
  +++ Ferm::Service[s1_mariadb_internal]
  
  +    srange  => $INTERNAL
  +    prio    => 10
  +    proto   => tcp
  +    ensure  => present
  +    port    => 3311
  +    notrack => True
  +    desc    => 
  

• Rsyslog::Conf[nrpe2nodexp-mariadb_memory]

  Parameters differences:

  --- Rsyslog::Conf[nrpe2nodexp-mariadb_memory].orig
  +++ Rsyslog::Conf[nrpe2nodexp-mariadb_memory]
  
  +    ensure   => absent
  +    mode     => 0444
  +    priority => 25
  

• File[/lib/systemd/system/nrpe2nodexp-mariadb_read_only_s1.timer]

  Parameters differences:

  --- File[/lib/systemd/system/nrpe2nodexp-mariadb_read_only_s1.timer].orig
  +++ File[/lib/systemd/system/nrpe2nodexp-mariadb_read_only_s1.timer]
  
  +    group  => root
  +    notify => Exec[systemd daemon-reload for nrpe2nodexp-mariadb_read_only_s1.timer (nrpe2nodexp-mariadb_read_only_s1.timer)]
  +    owner  => root
  +    ensure => absent
  +    mode   => 0444
  

  Content differences:

  --- /lib/systemd/system/nrpe2nodexp-mariadb_read_only_s1.timer.orig
  +++ /lib/systemd/system/nrpe2nodexp-mariadb_read_only_s1.timer
  @@ -0,0 +1,14 @@
  +[Unit]
  +Description=Periodic execution of nrpe2nodexp-mariadb_read_only_s1.service
  +
  +[Timer]
  +Unit=nrpe2nodexp-mariadb_read_only_s1.service
  +# Accuracy sets the maximum time interval around the execution time we want to allow
  +AccuracySec=15sec
  +OnUnitInactiveSec=1min
  +OnActiveSec=1s
  +RandomizedDelaySec=60
  +FixedRandomDelay=true
  +
  +[Install]
  +WantedBy=multi-user.target

• File[/var/lib/prometheus/node.d/check_mariadb_replica_io_state_s1.prom]

  Parameters differences:

  --- File[/var/lib/prometheus/node.d/check_mariadb_replica_io_state_s1.prom].orig
  +++ File[/var/lib/prometheus/node.d/check_mariadb_replica_io_state_s1.prom]
  
  +    group  => root
  +    owner  => root
  +    ensure => absent
  

• File[/var/lib/prometheus/node.d/check_mariadb_replica_sql_state_s1.prom]

  Parameters differences:

  --- File[/var/lib/prometheus/node.d/check_mariadb_replica_sql_state_s1.prom].orig
  +++ File[/var/lib/prometheus/node.d/check_mariadb_replica_sql_state_s1.prom]
  
  +    group  => root
  +    owner  => root
  +    ensure => absent
  

• Nrpe::Monitor_service[mariadb_read_only_s1]

  Parameters differences:

  --- Nrpe::Monitor_service[mariadb_read_only_s1].orig
  +++ Nrpe::Monitor_service[mariadb_read_only_s1]
  
  +    enable_nrpe2nodexp          => False
  +    alertmanager_team           => observability
  +    critical                    => False
  +    enable_icinga_check         => True
  +    description                 => MariaDB read only s1
  +    notes_url                   => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Master_comes_back_in_read_only
  +    nrpe2nodexp_parse_perf_data => False
  +    migration_task              => T315866
  +    ensure                      => present
  +    retry_interval              => 1
  +    timeout                     => 10
  +    require                     => Package[wmfmariadbpy-common]
  +    contact_group               => admins
  +    check_interval              => 1
  +    nrpe_command                => db-check-health --port=3311 --icinga --check_read_only=1 --process
  +    retries                     => 3
  

• File[/etc/ferm/conf.d/10_s1_mariadb_internal]

  Parameters differences:

  --- File[/etc/ferm/conf.d/10_s1_mariadb_internal].orig
  +++ File[/etc/ferm/conf.d/10_s1_mariadb_internal]
  
  +    group   => root
  +    notify  => Service[ferm]
  +    owner   => root
  +    require => File[/etc/ferm/conf.d]
  +    ensure  => present
  +    tag     => ferm
  +    mode    => 0400
  

  Content differences:

  --- /etc/ferm/conf.d/10_s1_mariadb_internal.orig
  +++ /etc/ferm/conf.d/10_s1_mariadb_internal
  @@ -0,0 +1,8 @@
  +# Autogenerated by puppet. DO NOT EDIT BY HAND!
  +#
  +# 
  +&R_SERVICE(tcp, 3311, $INTERNAL);
  +
  +
  +
  +&NO_TRACK(tcp, 3311);

• Class[Profile::Mariadb::Dbstore_multiinstance]

  Parameters differences:

  --- Class[Profile::Mariadb::Dbstore_multiinstance].orig
  +++ Class[Profile::Mariadb::Dbstore_multiinstance]
  
  +    instances     => {'s1': '192G'}
  +    require       => ['Class[Profile::Mariadb::Packages_wmf]']
  +    section_ports => {'s1': 3311, 's2': 3312, 's3': 3313, 's4': 3314, 's5': 3315, 's6': 3316, 's7': 3317, 's8': 3318, 'x1': 3320, 'x3': 3363, 'm1': 3321, 'm2': 3322, 'm3': 3323, 'm5': 3325, 'staging': 3350, 'matomo': 3351, 'analytics_meta': 3352, 'backup1-eqiad': 3360, 'backup1-codfw': 3361}
  

• Systemd::Unit[nrpe2nodexp-mariadb_memory.timer]

  Parameters differences:

  --- Systemd::Unit[nrpe2nodexp-mariadb_memory.timer].orig
  +++ Systemd::Unit[nrpe2nodexp-mariadb_memory.timer]
  
  +    unit              => nrpe2nodexp-mariadb_memory.timer
  +    require           => ['Class[Systemd]']
  +    restart           => False
  +    ensure            => absent
  +    override          => False
  +    override_filename => puppet-override.conf
  

• File[/usr/local/bin/mysql.s1]

  Parameters differences:

  --- File[/usr/local/bin/mysql.s1].orig
  +++ File[/usr/local/bin/mysql.s1]
  
  +    group  => root
  +    owner  => root
  +    mode   => 0555
  +    ensure => present
  

  Content differences:

  --- /usr/local/bin/mysql.s1.orig
  +++ /usr/local/bin/mysql.s1
  @@ -0,0 +1,3 @@
  +#!/bin/bash
  +
  +exec sudo mysql -S /run/mysqld/mysqld.s1.sock --prompt="\\u@$(hostname):s1[\\d]> "

• Exec[systemd daemon-reload for wmf_auto_restart_prometheus-mysqld-exporter@s1.service (wmf_auto_restart_prometheus-mysqld-exporter@s1.service)]

  Parameters differences:

  --- Exec[systemd daemon-reload for wmf_auto_restart_prometheus-mysqld-exporter@s1.service (wmf_auto_restart_prometheus-mysqld-exporter@s1.service)].orig
  +++ Exec[systemd daemon-reload for wmf_auto_restart_prometheus-mysqld-exporter@s1.service (wmf_auto_restart_prometheus-mysqld-exporter@s1.service)]
  
  +    refreshonly => True
  +    command     => /bin/systemctl daemon-reload
  

• Mariadb::Monitor_readonly[s1]

  Parameters differences:

  --- Mariadb::Monitor_readonly[s1].orig
  +++ Mariadb::Monitor_readonly[s1]
  
  +    port          => 3311
  +    is_critical   => False
  +    contact_group => admins
  +    read_only     => 1
  

• Monitoring::Exported_nagios_service[db2250 mariadb_replica_sql_state_s1]

  Parameters differences:

  --- Monitoring::Exported_nagios_service[db2250 mariadb_replica_sql_state_s1].orig
  +++ Monitoring::Exported_nagios_service[db2250 mariadb_replica_sql_state_s1]
  
  +    check_command          => nrpe_check!check_mariadb_replica_sql_state_s1!10
  +    notification_period    => 24x7
  +    notification_interval  => 0
  +    check_period           => 24x7
  +    service_description    => MariaDB Replica SQL: s1
  +    max_check_attempts     => 3
  +    notes_url              => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica
  +    notifications_enabled  => 0
  +    is_volatile            => 0
  +    retry_interval         => 1
  +    ensure                 => present
  +    active_checks_enabled  => 1
  +    notification_options   => c,r,f
  +    contact_groups         => admins
  +    check_interval         => 1
  +    servicegroups          => mysql_codfw
  +    passive_checks_enabled => 1
  +    host_name              => db2250
  +    check_freshness        => 0
  

• Systemd::Unit[nrpe2nodexp-mariadb_replica_sql_state_s1.service]

  Parameters differences:

  --- Systemd::Unit[nrpe2nodexp-mariadb_replica_sql_state_s1.service].orig
  +++ Systemd::Unit[nrpe2nodexp-mariadb_replica_sql_state_s1.service]
  
  +    unit              => nrpe2nodexp-mariadb_replica_sql_state_s1.service
  +    require           => ['Class[Systemd]']
  +    restart           => False
  +    ensure            => absent
  +    override          => False
  +    override_filename => puppet-override.conf
  

• File[/var/lib/prometheus/.my.s1.cnf]

  Parameters differences:

  --- File[/var/lib/prometheus/.my.s1.cnf].orig
  +++ File[/var/lib/prometheus/.my.s1.cnf]
  
  +    group  => prometheus
  +    notify => Exec[systemctl try-restart prometheus-mysqld-exporter@s1]
  +    owner  => prometheus
  +    ensure => present
  +    mode   => 0400
  

  Content differences:

  --- /var/lib/prometheus/.my.s1.cnf.orig
  +++ /var/lib/prometheus/.my.s1.cnf
  @@ -0,0 +1,5 @@
  +[client]
  +socket = /run/mysqld/mysqld.s1.sock
  +user = prometheus
  +password = 'This is a fake passsword, but cannot be empty due to Debian #953040'
  +ssl = false

• Concat::Fragment[/etc/mysql_puppet_agent_cert]

  Parameters differences:

  --- Concat::Fragment[/etc/mysql_puppet_agent_cert].orig
  +++ Concat::Fragment[/etc/mysql_puppet_agent_cert]
  
  +    order  => 01
  +    source => /var/lib/puppet/ssl/certs/db2250.codfw.wmnet.pem
  +    target => /etc/mysql/ssl/cert.pem
  

• Puppet::Expose_agent_certs[/etc/mysql]

  Parameters differences:

  --- Puppet::Expose_agent_certs[/etc/mysql].orig
  +++ Puppet::Expose_agent_certs[/etc/mysql]
  
  +    group           => mysql
  +    provide_keypair => False
  +    provide_private => True
  +    provide_p12     => False
  +    ssldir          => /var/lib/puppet/ssl
  +    user            => mysql
  +    ensure          => present
  +    provide_pem     => True
  

• Class[Mariadb::Packages_wmf]

  Parameters differences:

  --- Class[Mariadb::Packages_wmf].orig
  +++ Class[Mariadb::Packages_wmf]
  
  +    package => wmf-mariadb1011
  

• Systemd::Service[nrpe2nodexp-mysqld]

  Parameters differences:

  --- Systemd::Service[nrpe2nodexp-mysqld].orig
  +++ Systemd::Service[nrpe2nodexp-mysqld]
  
  +    monitoring_enabled       => False
  +    require                  => Systemd::Unit[nrpe2nodexp-mysqld.service]
  +    restart                  => False
  +    unit_type                => timer
  +    service_params           => {}
  +    monitoring_contact_group => admins
  +    monitoring_critical      => False
  +    migration_task           => T407130
  +    ensure                   => absent
  +    override                 => False
  

• File[/etc/update-motd.d/05-insetup--data-persistence-ferm]

  Parameters differences:

  --- File[/etc/update-motd.d/05-insetup--data-persistence-ferm].orig
  +++ File[/etc/update-motd.d/05-insetup--data-persistence-ferm]
  
  -    group  => root
  -    owner  => root
  -    mode   => 0555
  -    ensure => present
  

  Content differences:

  --- /etc/update-motd.d/05-insetup--data-persistence-ferm.orig
  +++ /etc/update-motd.d/05-insetup--data-persistence-ferm
  @@ -1,2 +0,0 @@
  -#!/bin/sh
  -printf "%s\n" "db2250 is a Host being setup by Data Persistence SREs (insetup::data_persistence_ferm)"

• Service[nrpe2nodexp-mariadb_read_only_s1.timer]

  Parameters differences:

  --- Service[nrpe2nodexp-mariadb_read_only_s1.timer].orig
  +++ Service[nrpe2nodexp-mariadb_read_only_s1.timer]
  
  +    before   => ['Exec[systemd daemon-reload for nrpe2nodexp-mariadb_read_only_s1.timer (nrpe2nodexp-mariadb_read_only_s1.timer)]']
  +    provider => systemd
  +    enable   => False
  +    ensure   => stopped
  

• File[/lib/systemd/system/nrpe2nodexp-mariadb_memory.timer]

  Parameters differences:

  --- File[/lib/systemd/system/nrpe2nodexp-mariadb_memory.timer].orig
  +++ File[/lib/systemd/system/nrpe2nodexp-mariadb_memory.timer]
  
  +    group  => root
  +    notify => Exec[systemd daemon-reload for nrpe2nodexp-mariadb_memory.timer (nrpe2nodexp-mariadb_memory.timer)]
  +    owner  => root
  +    ensure => absent
  +    mode   => 0444
  

  Content differences:

  --- /lib/systemd/system/nrpe2nodexp-mariadb_memory.timer.orig
  +++ /lib/systemd/system/nrpe2nodexp-mariadb_memory.timer
  @@ -0,0 +1,14 @@
  +[Unit]
  +Description=Periodic execution of nrpe2nodexp-mariadb_memory.service
  +
  +[Timer]
  +Unit=nrpe2nodexp-mariadb_memory.service
  +# Accuracy sets the maximum time interval around the execution time we want to allow
  +AccuracySec=15sec
  +OnUnitInactiveSec=1min
  +OnActiveSec=1s
  +RandomizedDelaySec=60
  +FixedRandomDelay=true
  +
  +[Install]
  +WantedBy=multi-user.target

• Systemd::Timer::Job[nrpe2nodexp-mariadb_read_only_s1]

  Parameters differences:

  --- Systemd::Timer::Job[nrpe2nodexp-mariadb_read_only_s1].orig
  +++ Systemd::Timer::Job[nrpe2nodexp-mariadb_read_only_s1]
  
  +    group                     => prometheus-node-exporter
  +    command                   => /usr/local/bin/nrpe2nodexp --alert-rule-hash "42baf529e0f3bfca391aca0a24c880e9" --timeout 10 --check-command "check_mariadb_read_only_s1"
  +    private_tmp               => False
  +    logfile_basedir           => /var/log
  +    user                      => nagios
  +    syslog_identifier         => nrpe2nodexp-mariadb_read_only_s1
  +    send_mail_to              => root@db2250.codfw.wmnet
  +    success_exit_status       => []
  +    fixed_random_delay        => True
  +    description               => execution of nrpe2nodexp for the check_mariadb_read_only_s1 command.
  +    ensure                    => absent
  +    logfile_group             => root
  +    monitoring_enabled        => False
  +    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
  +    syslog_match_startswith   => True
  +    interval                  => [{'start': 'OnUnitInactiveSec', 'interval': '1min'}]
  +    ignore_errors             => True
  +    send_mail_only_on_error   => True
  +    logfile_name              => syslog.log
  +    send_mail                 => False
  +    splay                     => 60
  +    syslog_force_stop         => True
  +    logfile_perms             => all
  +    environment               => {}
  +    logging_enabled           => False
  +    monitoring_contact_groups => admins
  

• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_memory.service (nrpe2nodexp-mariadb_memory.service)]

  Parameters differences:

  --- Exec[systemd daemon-reload for nrpe2nodexp-mariadb_memory.service (nrpe2nodexp-mariadb_memory.service)].orig
  +++ Exec[systemd daemon-reload for nrpe2nodexp-mariadb_memory.service (nrpe2nodexp-mariadb_memory.service)]
  
  +    refreshonly => True
  +    command     => /bin/systemctl daemon-reload
  

• File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-memory.conf]

  Parameters differences:

  --- File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-memory.conf].orig
  +++ File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-memory.conf]
  
  +    group  => root
  +    notify => Service[rsyslog]
  +    owner  => root
  +    ensure => absent
  +    mode   => 0444
  

  Content differences:

  --- /etc/rsyslog.d/25-nrpe2nodexp-mariadb-memory.conf.orig
  +++ /etc/rsyslog.d/25-nrpe2nodexp-mariadb-memory.conf
  @@ -0,0 +1,10 @@
  +# SPDX-License-Identifier: Apache-2.0
  +if $programname contains "nrpe2nodexp-mariadb_memory" then {
  +    if ($msg contains "\"ecs.version\": \"1.7.0\"") then {
  +        # Send logs to kafka
  +        set $.log_outputs = "kafka ecs_170 local";
  +    } else {
  +        # Filter out non-relevant nrpe2nodexp messages
  +        stop
  +    }
  +}

• Package[prometheus-mysqld-exporter]

  Parameters differences:

  --- Package[prometheus-mysqld-exporter].orig
  +++ Package[prometheus-mysqld-exporter]
  
  +    provider => apt
  +    ensure   => installed
  

• Concat_fragment[main contacts]

  Content differences:

  --- main contacts.orig
  +++ main contacts
  @@ -1,3 +1,3 @@
   ---
  -role::insetup::data_persistence_ferm:
  +role::mariadb::backup_source:
   - Data Persistence

• Monitoring::Exported_nagios_service[db2250 mariadb_memory]

  Parameters differences:

  --- Monitoring::Exported_nagios_service[db2250 mariadb_memory].orig
  +++ Monitoring::Exported_nagios_service[db2250 mariadb_memory]
  
  +    check_command          => nrpe_check!check_mariadb_memory!10
  +    notification_period    => 24x7
  +    notification_interval  => 0
  +    check_period           => 24x7
  +    service_description    => MariaDB memory
  +    max_check_attempts     => 3
  +    notes_url              => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting
  +    notifications_enabled  => 0
  +    is_volatile            => 0
  +    retry_interval         => 1
  +    ensure                 => present
  +    active_checks_enabled  => 1
  +    notification_options   => c,r,f
  +    contact_groups         => admins
  +    check_interval         => 1
  +    servicegroups          => mysql_codfw
  +    passive_checks_enabled => 1
  +    host_name              => db2250
  +    check_freshness        => 0
  

• Rsyslog::Conf[nrpe2nodexp-mariadb_replica_sql_state_s1]

  Parameters differences:

  --- Rsyslog::Conf[nrpe2nodexp-mariadb_replica_sql_state_s1].orig
  +++ Rsyslog::Conf[nrpe2nodexp-mariadb_replica_sql_state_s1]
  
  +    ensure   => absent
  +    mode     => 0444
  +    priority => 25
  

• Exec[systemd daemon-reload for wmf_auto_restart_prometheus-mysqld-exporter@s1.timer (wmf_auto_restart_prometheus-mysqld-exporter@s1.timer)]

  Parameters differences:

  --- Exec[systemd daemon-reload for wmf_auto_restart_prometheus-mysqld-exporter@s1.timer (wmf_auto_restart_prometheus-mysqld-exporter@s1.timer)].orig
  +++ Exec[systemd daemon-reload for wmf_auto_restart_prometheus-mysqld-exporter@s1.timer (wmf_auto_restart_prometheus-mysqld-exporter@s1.timer)]
  
  +    before      => ['Service[wmf_auto_restart_prometheus-mysqld-exporter@s1.timer]']
  +    refreshonly => True
  +    command     => /bin/systemctl daemon-reload
  

• File[/etc/ferm/conf.d/10_s1_orchestrator]

  Parameters differences:

  --- File[/etc/ferm/conf.d/10_s1_orchestrator].orig
  +++ File[/etc/ferm/conf.d/10_s1_orchestrator]
  
  +    group   => root
  +    notify  => Service[ferm]
  +    owner   => root
  +    require => File[/etc/ferm/conf.d]
  +    ensure  => present
  +    tag     => ferm
  +    mode    => 0400
  

  Content differences:

  --- /etc/ferm/conf.d/10_s1_orchestrator.orig
  +++ /etc/ferm/conf.d/10_s1_orchestrator
  @@ -0,0 +1,8 @@
  +# Autogenerated by puppet. DO NOT EDIT BY HAND!
  +#
  +# 
  +&R_SERVICE(tcp, 3311, (10.64.0.20 208.80.154.9 208.80.155.103 2620:0:861:4:208:80:155:103));
  +
  +
  +
  +&NO_TRACK(tcp, 3311);

• File[/etc/sudoers.d/nrpe-check_mariadb_replica_io_state_s1]

  Parameters differences:

  --- File[/etc/sudoers.d/nrpe-check_mariadb_replica_io_state_s1].orig
  +++ File[/etc/sudoers.d/nrpe-check_mariadb_replica_io_state_s1]
  
  +    group   => root
  +    owner   => root
  +    require => Package[nagios-nrpe-server]
  +    ensure  => absent
  

• File[/var/lib/prometheus/node.d/check_mariadb_disk_space.prom]

  Parameters differences:

  --- File[/var/lib/prometheus/node.d/check_mariadb_disk_space.prom].orig
  +++ File[/var/lib/prometheus/node.d/check_mariadb_disk_space.prom]
  
  +    group  => root
  +    owner  => root
  +    ensure => absent
  

• Class[Mariadb::Service]

  Parameters differences:

  --- Class[Mariadb::Service].orig
  +++ Class[Mariadb::Service]
  
  +    override => [Service]
  ExecStartPre=/bin/sh -c "echo 'mariadb main service is disabled, use mariadb@<instance_name> instead'; exit 1"
  +    enable   => False
  +    manage   => False
  +    ensure   => stopped
  

• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_disk_space.timer (nrpe2nodexp-mariadb_disk_space.timer)]

  Parameters differences:

  --- Exec[systemd daemon-reload for nrpe2nodexp-mariadb_disk_space.timer (nrpe2nodexp-mariadb_disk_space.timer)].orig
  +++ Exec[systemd daemon-reload for nrpe2nodexp-mariadb_disk_space.timer (nrpe2nodexp-mariadb_disk_space.timer)]
  
  +    refreshonly => True
  +    command     => /bin/systemctl daemon-reload
  

• Systemd::Timer::Job[nrpe2nodexp-mariadb_replica_io_state_s1]

  Parameters differences:

  --- Systemd::Timer::Job[nrpe2nodexp-mariadb_replica_io_state_s1].orig
  +++ Systemd::Timer::Job[nrpe2nodexp-mariadb_replica_io_state_s1]
  
  +    group                     => prometheus-node-exporter
  +    command                   => /usr/local/bin/nrpe2nodexp --alert-rule-hash "0d5979f7abbb49256cfedd70a646784e" --timeout 10 --check-command "check_mariadb_replica_io_state_s1"
  +    private_tmp               => False
  +    logfile_basedir           => /var/log
  +    user                      => nagios
  +    syslog_identifier         => nrpe2nodexp-mariadb_replica_io_state_s1
  +    send_mail_to              => root@db2250.codfw.wmnet
  +    success_exit_status       => []
  +    fixed_random_delay        => True
  +    description               => execution of nrpe2nodexp for the check_mariadb_replica_io_state_s1 command.
  +    ensure                    => absent
  +    logfile_group             => root
  +    monitoring_enabled        => False
  +    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
  +    syslog_match_startswith   => True
  +    interval                  => [{'start': 'OnUnitInactiveSec', 'interval': '1min'}]
  +    ignore_errors             => True
  +    send_mail_only_on_error   => True
  +    logfile_name              => syslog.log
  +    send_mail                 => False
  +    splay                     => 60
  +    syslog_force_stop         => True
  +    logfile_perms             => all
  +    environment               => {}
  +    logging_enabled           => False
  +    monitoring_contact_groups => admins
  

• Class[Profile::Mariadb::Packages_wmf]

  Parameters differences:

  --- Class[Profile::Mariadb::Packages_wmf].orig
  +++ Class[Profile::Mariadb::Packages_wmf]
  
  +    package => wmf-mariadb1011
  

• File[/etc/mysql/ssl/server-key.pem]

  Parameters differences:

  --- File[/etc/mysql/ssl/server-key.pem].orig
  +++ File[/etc/mysql/ssl/server-key.pem]
  
  +    group     => mysql
  +    owner     => root
  +    show_diff => False
  +    ensure    => file
  +    backup    => False
  +    mode      => 0440
  

• Systemd::Unit[nrpe2nodexp-mariadb_disk_space.service]

  Parameters differences:

  --- Systemd::Unit[nrpe2nodexp-mariadb_disk_space.service].orig
  +++ Systemd::Unit[nrpe2nodexp-mariadb_disk_space.service]
  
  +    unit              => nrpe2nodexp-mariadb_disk_space.service
  +    require           => ['Class[Systemd]']
  +    restart           => False
  +    ensure            => absent
  +    override          => False
  +    override_filename => puppet-override.conf
  

• File[/lib/systemd/system/nrpe2nodexp-mariadb_disk_space.timer]

  Parameters differences:

  --- File[/lib/systemd/system/nrpe2nodexp-mariadb_disk_space.timer].orig
  +++ File[/lib/systemd/system/nrpe2nodexp-mariadb_disk_space.timer]
  
  +    group  => root
  +    notify => Exec[systemd daemon-reload for nrpe2nodexp-mariadb_disk_space.timer (nrpe2nodexp-mariadb_disk_space.timer)]
  +    owner  => root
  +    ensure => absent
  +    mode   => 0444
  

  Content differences:

  --- /lib/systemd/system/nrpe2nodexp-mariadb_disk_space.timer.orig
  +++ /lib/systemd/system/nrpe2nodexp-mariadb_disk_space.timer
  @@ -0,0 +1,14 @@
  +[Unit]
  +Description=Periodic execution of nrpe2nodexp-mariadb_disk_space.service
  +
  +[Timer]
  +Unit=nrpe2nodexp-mariadb_disk_space.service
  +# Accuracy sets the maximum time interval around the execution time we want to allow
  +AccuracySec=15sec
  +OnUnitInactiveSec=1min
  +OnActiveSec=1s
  +RandomizedDelaySec=60
  +FixedRandomDelay=true
  +
  +[Install]
  +WantedBy=multi-user.target

• Systemd::Timer[nrpe2nodexp-mariadb_memory]

  Parameters differences:

  --- Systemd::Timer[nrpe2nodexp-mariadb_memory].orig
  +++ Systemd::Timer[nrpe2nodexp-mariadb_memory]
  
  +    fixed_random_delay => True
  +    splay              => 60
  +    unit_name          => nrpe2nodexp-mariadb_memory.service
  +    accuracy           => 15sec
  +    ensure             => absent
  +    timer_intervals    => [{'start': 'OnUnitInactiveSec', 'interval': '1min'}, {'interval': '1s', 'start': 'OnActiveSec'}]
  

• Class[Adduser]

  Parameters differences:

  --- Class[Adduser].orig
  +++ Class[Adduser]
  
  @@
  -    before => ['Package[puppet]', 'Package[facter]', 'Package[augeas-tools]', 'Package[virt-what]', 'Package[puppet-module-puppetlabs-augeas-core]', 'Package[python3-prometheus-client]', 'Package[python3-yaml]', 'Package[ruby-net-ssh]', 'Package[openssl]', 'Package[ssl-cert]', 'Package[ca-certificates]', 'Package[wmf-certificates]', 'Package[ntp]', 'Package[systemd-timesyncd]', 'Package[exim4-config]', 'Package[exim4-daemon-light]', 'Package[logrotate]', 'Package[prometheus-node-exporter]', 'Package[bsdutils]', 'Package[smartmontools]', 'Package[rsyslog]', 'Package[rsyslog-openssl]', 'Package[cadvisor]', 'Package[prometheus-ethtool-exporter]', 'Package[acct]', 'Package[byobu]', 'Package[colordiff]', 'Package[curl]', 'Package[debian-goodies]', 'Package[ethtool]', 'Package[gdb]', 'Package[gdisk]', 'Package[git]', 'Package[htop]', 'Package[httpry]', 'Package[iotop]', 'Package[iperf]', 'Package[jq]', 'Package[libtemplate-perl]', 'Package[lldpd]', 'Package[lshw]', 'Package[molly-guard]', 'Package[moreutils]', 'Package[net-tools]', 'Package[numactl]', 'Package[ncdu]', 'Package[ngrep]', 'Package[pigz]', 'Package[psmisc]', 'Package[pv]', 'Package[python3]', 'Package[screen]', 'Package[strace]', 'Package[sysstat]', 'Package[tcpdump]', 'Package[tmux]', 'Package[tree]', 'Package[vim]', 'Package[vim-addon-manager]', 'Package[vim-scripts]', 'Package[wipe]', 'Package[xfsprogs]', 'Package[zsh]', 'Package[icdiff]', 'Package[linux-perf]', 'Package[bsd-mailx]', 'Package[ack]', 'Package[netcat-openbsd]', 'Package[tshark]', 'Package[fzf]', 'Package[ripgrep]', 'Package[fd-find]', 'Package[kitty-terminfo]', 'Package[mtr-tiny]', 'Package[bat]', 'Package[efibootmgr]', 'Package[bind9-dnsutils]', 'Package[tzdata]', 'Package[python3-wmflib]', 'Package[ruby-sorted-set]', 'Package[btop]', 'Package[dstat]', 'Package[apport]', 'Package[command-not-found]', 'Package[command-not-found-data]', 'Package[ecryptfs-utils]', 'Package[mlocate]', 'Package[os-prober]', 'Package[python3-apport]', 'Package[wpasupplicant]', 'Package[atop]', 'Package[apt-listchanges]', 'Package[isc-dhcp-client]', 'Package[rasdaemon]', 'Package[libicu67]', 'Package[libwsutil12]', 'Package[libwireshark14]', 'Package[libopencsd0]', 'Package[libwiretap11]', 'Package[ruby2.7]', 'Package[python3.9-minimal]', 'Package[python3.9]', 'Package[perl-modules-5.32]', 'Package[libpython3.9]', 'Package[libperl5.32]', 'Package[libpython3.9-minimal]', 'Package[libpython3.9-stdlib]', 'Package[libidn11]', 'Package[libldap-2.4-2]', 'Package[liburing1]', 'Package[libwebp6]', 'Package[libcbor0]', 'Package[libusb-0.1-4]', 'Package[telnet]', 'Package[libruby2.7]', 'Package[libdns-export1110]', 'Package[libisc-export1105]', 'Package[libbpf0]', 'Package[openssh-client]', 'Package[openssh-server]', 'Package[debdeploy-client]', 'Package[python3-dateutil]', 'Package[sudo]', 'Package[golang-cfssl]', 'Package[debmonitor-client]', 'Package[perccli]', 'Package[nagios-nrpe-server]', 'Package[monitoring-plugins]', 'Package[monitoring-plugins-basic]', 'Package[monitoring-plugins-standard]', 'Package[liburiparser1]', 'Package[python3-attr]', 'Package[iucode-tool]', 'Package[freeipmi-tools]', 'Package[freeipmi-ipmiseld]', 'Package[rsyslog-kafka]', 'Package[emacs-nox]', 'Package[prometheus-ipmi-exporter]', 'Package[libnet-dns-perl]', 'Package[iptables]', 'Package[ferm]', 'Package[ulogd2]', 'Package[conntrack]', 'Package[prometheus-rsyslog-exporter]', 'Package[initramfs-tools]', 'Package[python3-click]', 'Package[python3-box]', 'Package[confd]', 'Package[python3-toml]']
  +    before => ['Package[puppet]', 'Package[facter]', 'Package[augeas-tools]', 'Package[virt-what]', 'Package[puppet-module-puppetlabs-augeas-core]', 'Package[python3-prometheus-client]', 'Package[python3-yaml]', 'Package[ruby-net-ssh]', 'Package[openssl]', 'Package[ssl-cert]', 'Package[ca-certificates]', 'Package[wmf-certificates]', 'Package[ntp]', 'Package[systemd-timesyncd]', 'Package[exim4-config]', 'Package[exim4-daemon-light]', 'Package[logrotate]', 'Package[prometheus-node-exporter]', 'Package[bsdutils]', 'Package[smartmontools]', 'Package[rsyslog]', 'Package[rsyslog-openssl]', 'Package[cadvisor]', 'Package[prometheus-ethtool-exporter]', 'Package[acct]', 'Package[byobu]', 'Package[colordiff]', 'Package[curl]', 'Package[debian-goodies]', 'Package[ethtool]', 'Package[gdb]', 'Package[gdisk]', 'Package[git]', 'Package[htop]', 'Package[httpry]', 'Package[iotop]', 'Package[iperf]', 'Package[jq]', 'Package[libtemplate-perl]', 'Package[lldpd]', 'Package[lshw]', 'Package[molly-guard]', 'Package[moreutils]', 'Package[net-tools]', 'Package[numactl]', 'Package[ncdu]', 'Package[ngrep]', 'Package[pigz]', 'Package[psmisc]', 'Package[pv]', 'Package[python3]', 'Package[screen]', 'Package[strace]', 'Package[sysstat]', 'Package[tcpdump]', 'Package[tmux]', 'Package[tree]', 'Package[vim]', 'Package[vim-addon-manager]', 'Package[vim-scripts]', 'Package[wipe]', 'Package[xfsprogs]', 'Package[zsh]', 'Package[icdiff]', 'Package[linux-perf]', 'Package[bsd-mailx]', 'Package[ack]', 'Package[netcat-openbsd]', 'Package[tshark]', 'Package[fzf]', 'Package[ripgrep]', 'Package[fd-find]', 'Package[kitty-terminfo]', 'Package[mtr-tiny]', 'Package[bat]', 'Package[efibootmgr]', 'Package[bind9-dnsutils]', 'Package[tzdata]', 'Package[python3-wmflib]', 'Package[ruby-sorted-set]', 'Package[btop]', 'Package[dstat]', 'Package[apport]', 'Package[command-not-found]', 'Package[command-not-found-data]', 'Package[ecryptfs-utils]', 'Package[mlocate]', 'Package[os-prober]', 'Package[python3-apport]', 'Package[wpasupplicant]', 'Package[atop]', 'Package[apt-listchanges]', 'Package[isc-dhcp-client]', 'Package[rasdaemon]', 'Package[libicu67]', 'Package[libwsutil12]', 'Package[libwireshark14]', 'Package[libopencsd0]', 'Package[libwiretap11]', 'Package[ruby2.7]', 'Package[python3.9-minimal]', 'Package[python3.9]', 'Package[perl-modules-5.32]', 'Package[libpython3.9]', 'Package[libperl5.32]', 'Package[libpython3.9-minimal]', 'Package[libpython3.9-stdlib]', 'Package[libidn11]', 'Package[libldap-2.4-2]', 'Package[liburing1]', 'Package[libwebp6]', 'Package[libcbor0]', 'Package[libusb-0.1-4]', 'Package[telnet]', 'Package[libruby2.7]', 'Package[libdns-export1110]', 'Package[libisc-export1105]', 'Package[libbpf0]', 'Package[openssh-client]', 'Package[openssh-server]', 'Package[debdeploy-client]', 'Package[python3-dateutil]', 'Package[sudo]', 'Package[golang-cfssl]', 'Package[debmonitor-client]', 'Package[perccli]', 'Package[nagios-nrpe-server]', 'Package[monitoring-plugins]', 'Package[monitoring-plugins-basic]', 'Package[monitoring-plugins-standard]', 'Package[liburiparser1]', 'Package[python3-attr]', 'Package[iucode-tool]', 'Package[freeipmi-tools]', 'Package[freeipmi-ipmiseld]', 'Package[rsyslog-kafka]', 'Package[emacs-nox]', 'Package[prometheus-ipmi-exporter]', 'Package[libnet-dns-perl]', 'Package[iptables]', 'Package[ferm]', 'Package[ulogd2]', 'Package[conntrack]', 'Package[wmf-mariadb1011]', 'Package[percona-toolkit]', 'Package[grc]', 'Package[mariadb-backup]', 'Package[python3-wmfmariadbpy]', 'Package[wmfmariadbpy-common]', 'Package[monitoring-plugins-contrib]', 'Package[prometheus-rsyslog-exporter]', 'Package[initramfs-tools]', 'Package[python3-click]', 'Package[python3-box]', 'Package[confd]', 'Package[python3-toml]', 'Package[prometheus-mysqld-exporter]']
  

• Service[nrpe2nodexp-mariadb_replica_io_state_s1.timer]

  Parameters differences:

  --- Service[nrpe2nodexp-mariadb_replica_io_state_s1.timer].orig
  +++ Service[nrpe2nodexp-mariadb_replica_io_state_s1.timer]
  
  +    before   => ['Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_io_state_s1.timer (nrpe2nodexp-mariadb_replica_io_state_s1.timer)]']
  +    provider => systemd
  +    enable   => False
  +    ensure   => stopped
  

• Nrpe::Check[check_mariadb_replica_sql_state_s1]

  Parameters differences:

  --- Nrpe::Check[check_mariadb_replica_sql_state_s1].orig
  +++ Nrpe::Check[check_mariadb_replica_sql_state_s1]
  
  +    before  => Monitoring::Service[mariadb_replica_sql_state_s1]
  +    command => /usr/local/lib/nagios/plugins/check_mariadb --sock=/run/mysqld/mysqld.s1.sock --user=nagios --pass=  --warn-stopped --check=slave_sql_state
  +    ensure  => present
  

• Profile::Mariadb::Section[s1]

  Parameters differences:

  --- Profile::Mariadb::Section[s1].orig
  +++ Profile::Mariadb::Section[s1]
  
  +    section       => s1
  +    mention_alias => True
  

• File[/etc/systemd/system/mariadb.service.d/override.conf]

  Parameters differences:

  --- File[/etc/systemd/system/mariadb.service.d/override.conf].orig
  +++ File[/etc/systemd/system/mariadb.service.d/override.conf]
  
  +    group  => root
  +    notify => Exec[systemctl-daemon-reload]
  +    owner  => root
  +    ensure => present
  +    mode   => 0644
  

  Content differences:

  --- /etc/systemd/system/mariadb.service.d/override.conf.orig
  +++ /etc/systemd/system/mariadb.service.d/override.conf
  @@ -0,0 +1,2 @@
  +[Service]
  +ExecStartPre=/bin/sh -c "echo 'mariadb main service is disabled, use mariadb@<instance_name> instead'; exit 1"

• Systemd::Unit[nrpe2nodexp-mariadb_replica_io_state_s1.timer]

  Parameters differences:

  --- Systemd::Unit[nrpe2nodexp-mariadb_replica_io_state_s1.timer].orig
  +++ Systemd::Unit[nrpe2nodexp-mariadb_replica_io_state_s1.timer]
  
  +    unit              => nrpe2nodexp-mariadb_replica_io_state_s1.timer
  +    require           => ['Class[Systemd]']
  +    restart           => False
  +    ensure            => absent
  +    override          => False
  +    override_filename => puppet-override.conf
  

• File[/etc/nagios/nrpe.d/check_mariadb_replica_io_state_s1.cfg]

  Parameters differences:

  --- File[/etc/nagios/nrpe.d/check_mariadb_replica_io_state_s1.cfg].orig
  +++ File[/etc/nagios/nrpe.d/check_mariadb_replica_io_state_s1.cfg]
  
  +    group   => root
  +    notify  => Service[nagios-nrpe-server]
  +    owner   => root
  +    require => Package[nagios-nrpe-server]
  +    ensure  => present
  +    tag     => nrpe::check
  +    mode    => 0444
  

  Content differences:

  --- /etc/nagios/nrpe.d/check_mariadb_replica_io_state_s1.cfg.orig
  +++ /etc/nagios/nrpe.d/check_mariadb_replica_io_state_s1.cfg
  @@ -0,0 +1,2 @@
  +# File generated by puppet. DO NOT edit by hand
  +command[check_mariadb_replica_io_state_s1]=/usr/local/lib/nagios/plugins/check_mariadb --sock=/run/mysqld/mysqld.s1.sock --user=nagios --pass=  --warn-stopped --check=slave_io_state

• Monitoring::Service[mariadb_memory]

  Parameters differences:

  --- Monitoring::Service[mariadb_memory].orig
  +++ Monitoring::Service[mariadb_memory]
  
  +    freshness      => 36000
  +    config_dir     => /etc/nagios
  +    passive        => False
  +    check_command  => nrpe_check!check_mariadb_memory!10
  +    host           => db2250
  +    critical       => False
  +    contact_group  => admins
  +    check_interval => 1
  +    description    => MariaDB memory
  +    notes_url      => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting
  +    retry_interval => 1
  +    ensure         => present
  +    migration_task => T315866
  +    retries        => 3
  

• Firewall::Service[s1_orchestrator]

  Parameters differences:

  --- Firewall::Service[s1_orchestrator].orig
  +++ Firewall::Service[s1_orchestrator]
  
  +    srange  => ['dborch1001.wikimedia.org', 'dborch1002.wikimedia.org', 'dborch1003.eqiad.wmnet']
  +    prio    => 10
  +    proto   => tcp
  +    ensure  => present
  +    port    => [3311]
  +    notrack => True
  +    desc    => 
  

• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_lag_s1.service (nrpe2nodexp-mariadb_replica_sql_lag_s1.service)]

  Parameters differences:

  --- Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_lag_s1.service (nrpe2nodexp-mariadb_replica_sql_lag_s1.service)].orig
  +++ Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_lag_s1.service (nrpe2nodexp-mariadb_replica_sql_lag_s1.service)]
  
  +    refreshonly => True
  +    command     => /bin/systemctl daemon-reload
  

• Profile::Prometheus::Mysqld_exporter_instance[s1]

  Parameters differences:

  --- Profile::Prometheus::Mysqld_exporter_instance[s1].orig
  +++ Profile::Prometheus::Mysqld_exporter_instance[s1]
  
  +    port   => 13311
  +    socket => /run/mysqld/mysqld.s1.sock
  

• File[/lib/systemd/system/prometheus-mysqld-exporter@.service]

  Parameters differences:

  --- File[/lib/systemd/system/prometheus-mysqld-exporter@.service].orig
  +++ File[/lib/systemd/system/prometheus-mysqld-exporter@.service]
  
  +    group  => root
  +    notify => Exec[systemd daemon-reload for prometheus-mysqld-exporter@.service (prometheus-mysqld-exporter@)]
  +    owner  => root
  +    ensure => present
  +    mode   => 0444
  

  Content differences:

  --- /lib/systemd/system/prometheus-mysqld-exporter@.service.orig
  +++ /lib/systemd/system/prometheus-mysqld-exporter@.service
  @@ -0,0 +1,18 @@
  +[Unit]
  +Description=Prometheus exporter for MySQL (instance %i)
  +Documentation=https://prometheus.io/docs/introduction/overview/
  +ConditionPathExists=/etc/default/prometheus-mysqld-exporter@%i
  +#Ensure the exporter is (re-)started and stopped with the relevant
  +#mariadb instance
  +After=mariadb@%i.service
  +Requisite=mariadb@%i.service
  +
  +[Service]
  +Restart=always
  +User=prometheus
  +Group=prometheus
  +EnvironmentFile=/etc/default/prometheus-mysqld-exporter@%i
  +ExecStart=/usr/bin/prometheus-mysqld-exporter $ARGS
  +
  +[Install]
  +WantedBy=multi-user.target

• File[/etc/mysql/ssl]

  Parameters differences:

  --- File[/etc/mysql/ssl].orig
  +++ File[/etc/mysql/ssl]
  
  +    group  => mysql
  +    owner  => mysql
  +    mode   => 0555
  +    ensure => directory
  

• File[/lib/systemd/system/nrpe2nodexp-mysqld.service]

  Parameters differences:

  --- File[/lib/systemd/system/nrpe2nodexp-mysqld.service].orig
  +++ File[/lib/systemd/system/nrpe2nodexp-mysqld.service]
  
  +    group  => root
  +    notify => Exec[systemd daemon-reload for nrpe2nodexp-mysqld.service (nrpe2nodexp-mysqld.service)]
  +    owner  => root
  +    ensure => absent
  +    mode   => 0444
  

  Content differences:

  --- /lib/systemd/system/nrpe2nodexp-mysqld.service.orig
  +++ /lib/systemd/system/nrpe2nodexp-mysqld.service
  @@ -0,0 +1,11 @@
  +[Unit]
  +Description=execution of nrpe2nodexp for the check_mysqld command.
  +Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
  +
  +[Service]
  +Type=oneshot
  +User=nagios
  +
  +Group=prometheus-node-exporter
  +SyslogIdentifier=nrpe2nodexp-mysqld
  +ExecStart=-/usr/local/bin/nrpe2nodexp --alert-rule-hash "ca81db2c9e98d408f9f65c79817da116" --timeout 10 --check-command "check_mysqld"

• Rsyslog::Conf[nrpe2nodexp-mariadb_replica_sql_lag_s1]

  Parameters differences:

  --- Rsyslog::Conf[nrpe2nodexp-mariadb_replica_sql_lag_s1].orig
  +++ Rsyslog::Conf[nrpe2nodexp-mariadb_replica_sql_lag_s1]
  
  +    ensure   => absent
  +    mode     => 0444
  +    priority => 25
  

• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_state_s1.timer (nrpe2nodexp-mariadb_replica_sql_state_s1.timer)]

  Parameters differences:

  --- Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_state_s1.timer (nrpe2nodexp-mariadb_replica_sql_state_s1.timer)].orig
  +++ Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_state_s1.timer (nrpe2nodexp-mariadb_replica_sql_state_s1.timer)]
  
  +    refreshonly => True
  +    command     => /bin/systemctl daemon-reload
  

• File[/etc/sudoers.d/nrpe-check_mariadb_disk_space]

  Parameters differences:

  --- File[/etc/sudoers.d/nrpe-check_mariadb_disk_space].orig
  +++ File[/etc/sudoers.d/nrpe-check_mariadb_disk_space]
  
  +    group   => root
  +    owner   => root
  +    require => Package[nagios-nrpe-server]
  +    ensure  => absent
  

• Monitoring::Service[mariadb_replica_sql_lag_s1]

  Parameters differences:

  --- Monitoring::Service[mariadb_replica_sql_lag_s1].orig
  +++ Monitoring::Service[mariadb_replica_sql_lag_s1]
  
  +    freshness      => 36000
  +    config_dir     => /etc/nagios
  +    passive        => False
  +    check_command  => nrpe_check!check_mariadb_replica_sql_lag_s1!10
  +    host           => db2250
  +    critical       => False
  +    contact_group  => admins
  +    check_interval => 1
  +    description    => MariaDB Replica Lag: s1
  +    notes_url      => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica
  +    retry_interval => 1
  +    ensure         => present
  +    migration_task => T315866
  +    retries        => 10
  

• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_io_state_s1.timer (nrpe2nodexp-mariadb_replica_io_state_s1.timer)]

  Parameters differences:

  --- Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_io_state_s1.timer (nrpe2nodexp-mariadb_replica_io_state_s1.timer)].orig
  +++ Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_io_state_s1.timer (nrpe2nodexp-mariadb_replica_io_state_s1.timer)]
  
  +    refreshonly => True
  +    command     => /bin/systemctl daemon-reload
  

• Systemd::Timer[nrpe2nodexp-mariadb_replica_io_state_s1]

  Parameters differences:

  --- Systemd::Timer[nrpe2nodexp-mariadb_replica_io_state_s1].orig
  +++ Systemd::Timer[nrpe2nodexp-mariadb_replica_io_state_s1]
  
  +    fixed_random_delay => True
  +    splay              => 60
  +    unit_name          => nrpe2nodexp-mariadb_replica_io_state_s1.service
  +    accuracy           => 15sec
  +    ensure             => absent
  +    timer_intervals    => [{'start': 'OnUnitInactiveSec', 'interval': '1min'}, {'interval': '1s', 'start': 'OnActiveSec'}]
  

• File[/etc/nagios/nrpe.d/check_mysqld.cfg]

  Parameters differences:

  --- File[/etc/nagios/nrpe.d/check_mysqld.cfg].orig
  +++ File[/etc/nagios/nrpe.d/check_mysqld.cfg]
  
  +    group   => root
  +    notify  => Service[nagios-nrpe-server]
  +    owner   => root
  +    require => Package[nagios-nrpe-server]
  +    ensure  => present
  +    tag     => nrpe::check
  +    mode    => 0444
  

  Content differences:

  --- /etc/nagios/nrpe.d/check_mysqld.cfg.orig
  +++ /etc/nagios/nrpe.d/check_mysqld.cfg
  @@ -0,0 +1,2 @@
  +# File generated by puppet. DO NOT edit by hand
  +command[check_mysqld]=/usr/lib/nagios/plugins/check_procs -c 1:1 -C mysqld

• File[/etc/sudoers.d/nrpe-check_mysqld]

  Parameters differences:

  --- File[/etc/sudoers.d/nrpe-check_mysqld].orig
  +++ File[/etc/sudoers.d/nrpe-check_mysqld]
  
  +    group   => root
  +    owner   => root
  +    require => Package[nagios-nrpe-server]
  +    ensure  => absent
  

• Ferm::Service[s1_mariadb_dba]

  Parameters differences:

  --- Ferm::Service[s1_mariadb_dba].orig
  +++ Ferm::Service[s1_mariadb_dba]
  
  +    prio    => 10
  +    proto   => tcp
  +    ensure  => present
  +    port    => 3331
  +    notrack => False
  +    desc    => 
  +    srange  => $MYSQL_ROOT_CLIENTS
  

• Nrpe::Check[check_mysqld]

  Parameters differences:

  --- Nrpe::Check[check_mysqld].orig
  +++ Nrpe::Check[check_mysqld]
  
  +    before  => Monitoring::Service[mysqld]
  +    command => /usr/lib/nagios/plugins/check_procs -c 1:1 -C mysqld
  +    ensure  => present
  

• File[/etc/nagios/nrpe.d/check_mariadb_disk_space.cfg]

  Parameters differences:

  --- File[/etc/nagios/nrpe.d/check_mariadb_disk_space.cfg].orig
  +++ File[/etc/nagios/nrpe.d/check_mariadb_disk_space.cfg]
  
  +    group   => root
  +    notify  => Service[nagios-nrpe-server]
  +    owner   => root
  +    require => Package[nagios-nrpe-server]
  +    ensure  => present
  +    tag     => nrpe::check
  +    mode    => 0444
  

  Content differences:

  --- /etc/nagios/nrpe.d/check_mariadb_disk_space.cfg.orig
  +++ /etc/nagios/nrpe.d/check_mariadb_disk_space.cfg
  @@ -0,0 +1,2 @@
  +# File generated by puppet. DO NOT edit by hand
  +command[check_mariadb_disk_space]=/usr/lib/nagios/plugins/check_disk -w 10% -c 5% -l -e --exclude-type=tracefs --exclude-type=tmpfs --exclude-type=ramfs --exclude-type=devtmpfs

• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_memory.timer (nrpe2nodexp-mariadb_memory.timer)]

  Parameters differences:

  --- Exec[systemd daemon-reload for nrpe2nodexp-mariadb_memory.timer (nrpe2nodexp-mariadb_memory.timer)].orig
  +++ Exec[systemd daemon-reload for nrpe2nodexp-mariadb_memory.timer (nrpe2nodexp-mariadb_memory.timer)]
  
  +    refreshonly => True
  +    command     => /bin/systemctl daemon-reload
  

• File[/etc/mysql/ssl/server.p12]

  Parameters differences:

  --- File[/etc/mysql/ssl/server.p12].orig
  +++ File[/etc/mysql/ssl/server.p12]
  
  +    group  => mysql
  +    owner  => mysql
  +    mode   => 0440
  +    ensure => absent
  

• Nrpe::Monitor_service[mariadb_replica_sql_lag_s1]

  Parameters differences:

  --- Nrpe::Monitor_service[mariadb_replica_sql_lag_s1].orig
  +++ Nrpe::Monitor_service[mariadb_replica_sql_lag_s1]
  
  +    enable_nrpe2nodexp          => False
  +    alertmanager_team           => observability
  +    critical                    => False
  +    contact_group               => admins
  +    enable_icinga_check         => True
  +    check_interval              => 1
  +    description                 => MariaDB Replica Lag: s1
  +    notes_url                   => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica
  +    nrpe_command                => /usr/local/lib/nagios/plugins/check_mariadb --sock=/run/mysqld/mysqld.s1.sock --user=nagios --pass=  --warn-stopped --check=slave_sql_lag                            --shard=s1 --datacenter=eqiad                            --sql-lag-warn=60                            --sql-lag-crit=300
  +    nrpe2nodexp_parse_perf_data => False
  +    migration_task              => T315866
  +    ensure                      => present
  +    retry_interval              => 1
  +    retries                     => 10
  +    timeout                     => 10
  

• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_lag_s1.timer (nrpe2nodexp-mariadb_replica_sql_lag_s1.timer)]

  Parameters differences:

  --- Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_lag_s1.timer (nrpe2nodexp-mariadb_replica_sql_lag_s1.timer)].orig
  +++ Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_lag_s1.timer (nrpe2nodexp-mariadb_replica_sql_lag_s1.timer)]
  
  +    refreshonly => True
  +    command     => /bin/systemctl daemon-reload
  

• Nrpe::Monitor_service[mariadb_disk_space]

  Parameters differences:

  --- Nrpe::Monitor_service[mariadb_disk_space].orig
  +++ Nrpe::Monitor_service[mariadb_disk_space]
  
  +    enable_nrpe2nodexp          => False
  +    alertmanager_team           => observability
  +    critical                    => False
  +    contact_group               => admins
  +    enable_icinga_check         => True
  +    check_interval              => 1
  +    description                 => MariaDB disk space
  +    notes_url                   => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting
  +    nrpe_command                => /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -l -e --exclude-type=tracefs --exclude-type=tmpfs --exclude-type=ramfs --exclude-type=devtmpfs
  +    nrpe2nodexp_parse_perf_data => False
  +    migration_task              => T315866
  +    ensure                      => present
  +    retry_interval              => 1
  +    retries                     => 3
  +    timeout                     => 10
  

• Monitoring::Service[mariadb_replica_io_state_s1]

  Parameters differences:

  --- Monitoring::Service[mariadb_replica_io_state_s1].orig
  +++ Monitoring::Service[mariadb_replica_io_state_s1]
  
  +    freshness      => 36000
  +    config_dir     => /etc/nagios
  +    passive        => False
  +    check_command  => nrpe_check!check_mariadb_replica_io_state_s1!10
  +    host           => db2250
  +    critical       => False
  +    contact_group  => admins
  +    check_interval => 1
  +    description    => MariaDB Replica IO: s1
  +    notes_url      => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica
  +    retry_interval => 1
  +    ensure         => present
  +    migration_task => T315866
  +    retries        => 3
  

• Nrpe::Monitor_service[mariadb_replica_sql_state_s1]

  Parameters differences:

  --- Nrpe::Monitor_service[mariadb_replica_sql_state_s1].orig
  +++ Nrpe::Monitor_service[mariadb_replica_sql_state_s1]
  
  +    enable_nrpe2nodexp          => False
  +    alertmanager_team           => observability
  +    critical                    => False
  +    contact_group               => admins
  +    enable_icinga_check         => True
  +    check_interval              => 1
  +    description                 => MariaDB Replica SQL: s1
  +    notes_url                   => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica
  +    nrpe_command                => /usr/local/lib/nagios/plugins/check_mariadb --sock=/run/mysqld/mysqld.s1.sock --user=nagios --pass=  --warn-stopped --check=slave_sql_state
  +    nrpe2nodexp_parse_perf_data => False
  +    migration_task              => T315866
  +    ensure                      => present
  +    retry_interval              => 1
  +    retries                     => 3
  +    timeout                     => 10
  

• Mariadb::Monitor_replication[s1]

  Parameters differences:

  --- Mariadb::Monitor_replication[s1].orig
  +++ Mariadb::Monitor_replication[s1]
  
  +    lag_warn      => 60
  +    socket        => /run/mysqld/mysqld.s1.sock
  +    lag_crit      => 300
  +    warn_stopped  => True
  +    contact_group => admins
  +    multisource   => False
  +    is_critical   => False
  +    source_dc     => eqiad
  

• File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_lag_s1.service]

  Parameters differences:

  --- File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_lag_s1.service].orig
  +++ File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_lag_s1.service]
  
  +    group  => root
  +    notify => Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_lag_s1.service (nrpe2nodexp-mariadb_replica_sql_lag_s1.service)]
  +    owner  => root
  +    ensure => absent
  +    mode   => 0444
  

  Content differences:

  --- /lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_lag_s1.service.orig
  +++ /lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_lag_s1.service
  @@ -0,0 +1,11 @@
  +[Unit]
  +Description=execution of nrpe2nodexp for the check_mariadb_replica_sql_lag_s1 command.
  +Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
  +
  +[Service]
  +Type=oneshot
  +User=nagios
  +
  +Group=prometheus-node-exporter
  +SyslogIdentifier=nrpe2nodexp-mariadb_replica_sql_lag_s1
  +ExecStart=-/usr/local/bin/nrpe2nodexp --alert-rule-hash "8ec9d6c1155ae60699aaf9dfd1fa339b" --timeout 10 --check-command "check_mariadb_replica_sql_lag_s1"

• Prometheus::Alert::Rule[check_mariadb_replica_sql_state_s1_3f927914a18db09399b7fff42858d634]

  Parameters differences:

  --- Prometheus::Alert::Rule[check_mariadb_replica_sql_state_s1_3f927914a18db09399b7fff42858d634].orig
  +++ Prometheus::Alert::Rule[check_mariadb_replica_sql_state_s1_3f927914a18db09399b7fff42858d634]
  
  +    group              => nrpechecks
  +    team               => observability
  +    for                => 3m
  +    site               => codfw
  +    severity           => info
  +    dashboard          => TODO
  +    summary            => NRPE CHECK: MariaDB Replica SQL: s1
  +    logs               => https://logstash.wikimedia.org/app/dashboards#/view/2d343ac0-6df8-11f0-8e08-7fab0da52b33?_g=(filters:!((query:(match_phrase:(event.module:check_mariadb_replica_sql_state_s1))),(query:(match_phrase:(host.name:{{$labels.instance|stripPort}})))))
  +    instance           => ops
  +    alert_name         => nrpe_MariaDB_Replica_SQL__s1
  +    description        => NRPE CHECK: MariaDB Replica SQL: s1
  +    runbook            => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica
  +    ensure             => absent
  +    def_label_whitelst => ['team', 'severity']
  +    expr               => (nagios_nrpe_check_result{alert_rule_hash="3f927914a18db09399b7fff42858d634",check_name="check_mariadb_replica_sql_state_s1", status=~"(WARNING|CRITICAL)", severity=~"(warning|critical)"} > 0) * on (instance) group_left (team) role_owner
  

• Systemd::Timer::Job[nrpe2nodexp-mariadb_memory]

  Parameters differences:

  --- Systemd::Timer::Job[nrpe2nodexp-mariadb_memory].orig
  +++ Systemd::Timer::Job[nrpe2nodexp-mariadb_memory]
  
  +    group                     => prometheus-node-exporter
  +    command                   => /usr/local/bin/nrpe2nodexp --alert-rule-hash "b5a92c72542d5f0c031014e7a8db014e" --timeout 10 --check-command "check_mariadb_memory"
  +    private_tmp               => False
  +    logfile_basedir           => /var/log
  +    user                      => nagios
  +    syslog_identifier         => nrpe2nodexp-mariadb_memory
  +    send_mail_to              => root@db2250.codfw.wmnet
  +    success_exit_status       => []
  +    fixed_random_delay        => True
  +    description               => execution of nrpe2nodexp for the check_mariadb_memory command.
  +    ensure                    => absent
  +    logfile_group             => root
  +    monitoring_enabled        => False
  +    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
  +    syslog_match_startswith   => True
  +    interval                  => [{'start': 'OnUnitInactiveSec', 'interval': '1min'}]
  +    ignore_errors             => True
  +    send_mail_only_on_error   => True
  +    logfile_name              => syslog.log
  +    send_mail                 => False
  +    splay                     => 60
  +    syslog_force_stop         => True
  +    logfile_perms             => all
  +    environment               => {}
  +    logging_enabled           => False
  +    monitoring_contact_groups => admins
  

• Logrotate::Conf[wmf_auto_restart_prometheus-mysqld-exporter_s1]

  Parameters differences:

  --- Logrotate::Conf[wmf_auto_restart_prometheus-mysqld-exporter_s1].orig
  +++ Logrotate::Conf[wmf_auto_restart_prometheus-mysqld-exporter_s1]
  
  +    ensure => present
  

• File[/etc/sudoers.d/nrpe-check_mariadb_replica_sql_state_s1]

  Parameters differences:

  --- File[/etc/sudoers.d/nrpe-check_mariadb_replica_sql_state_s1].orig
  +++ File[/etc/sudoers.d/nrpe-check_mariadb_replica_sql_state_s1]
  
  +    group   => root
  +    owner   => root
  +    require => Package[nagios-nrpe-server]
  +    ensure  => absent
  

• Concat_fragment[/etc/mysql_puppet_agent_cert]

  Parameters differences:

  --- Concat_fragment[/etc/mysql_puppet_agent_cert].orig
  +++ Concat_fragment[/etc/mysql_puppet_agent_cert]
  
  +    tag    => _etc_mysql_ssl_cert.pem
  +    order  => 01
  +    source => /var/lib/puppet/ssl/certs/db2250.codfw.wmnet.pem
  +    target => /etc/mysql/ssl/cert.pem
  

• Class[Profile::Apt]

  Parameters differences:

  --- Class[Profile::Apt].orig
  +++ Class[Profile::Apt]
  
  @@
  -    before => ['Package[puppet]', 'Package[facter]', 'Package[augeas-tools]', 'Package[virt-what]', 'Package[puppet-module-puppetlabs-augeas-core]', 'Package[python3-prometheus-client]', 'Package[python3-yaml]', 'Package[ruby-net-ssh]', 'Package[openssl]', 'Package[ssl-cert]', 'Package[ca-certificates]', 'Package[wmf-certificates]', 'Package[ntp]', 'Package[systemd-timesyncd]', 'Package[exim4-config]', 'Package[exim4-daemon-light]', 'Package[logrotate]', 'Package[prometheus-node-exporter]', 'Package[bsdutils]', 'Package[smartmontools]', 'Package[rsyslog]', 'Package[rsyslog-openssl]', 'Package[cadvisor]', 'Package[prometheus-ethtool-exporter]', 'Package[acct]', 'Package[byobu]', 'Package[colordiff]', 'Package[curl]', 'Package[debian-goodies]', 'Package[ethtool]', 'Package[gdb]', 'Package[gdisk]', 'Package[git]', 'Package[htop]', 'Package[httpry]', 'Package[iotop]', 'Package[iperf]', 'Package[jq]', 'Package[libtemplate-perl]', 'Package[lldpd]', 'Package[lshw]', 'Package[molly-guard]', 'Package[moreutils]', 'Package[net-tools]', 'Package[numactl]', 'Package[ncdu]', 'Package[ngrep]', 'Package[pigz]', 'Package[psmisc]', 'Package[pv]', 'Package[python3]', 'Package[screen]', 'Package[strace]', 'Package[sysstat]', 'Package[tcpdump]', 'Package[tmux]', 'Package[tree]', 'Package[vim]', 'Package[vim-addon-manager]', 'Package[vim-scripts]', 'Package[wipe]', 'Package[xfsprogs]', 'Package[zsh]', 'Package[icdiff]', 'Package[linux-perf]', 'Package[bsd-mailx]', 'Package[ack]', 'Package[netcat-openbsd]', 'Package[tshark]', 'Package[fzf]', 'Package[ripgrep]', 'Package[fd-find]', 'Package[kitty-terminfo]', 'Package[mtr-tiny]', 'Package[bat]', 'Package[efibootmgr]', 'Package[bind9-dnsutils]', 'Package[tzdata]', 'Package[python3-wmflib]', 'Package[ruby-sorted-set]', 'Package[btop]', 'Package[dstat]', 'Package[apport]', 'Package[command-not-found]', 'Package[command-not-found-data]', 'Package[ecryptfs-utils]', 'Package[mlocate]', 'Package[os-prober]', 'Package[python3-apport]', 'Package[wpasupplicant]', 'Package[atop]', 'Package[apt-listchanges]', 'Package[isc-dhcp-client]', 'Package[rasdaemon]', 'Package[libicu67]', 'Package[libwsutil12]', 'Package[libwireshark14]', 'Package[libopencsd0]', 'Package[libwiretap11]', 'Package[ruby2.7]', 'Package[python3.9-minimal]', 'Package[python3.9]', 'Package[perl-modules-5.32]', 'Package[libpython3.9]', 'Package[libperl5.32]', 'Package[libpython3.9-minimal]', 'Package[libpython3.9-stdlib]', 'Package[libidn11]', 'Package[libldap-2.4-2]', 'Package[liburing1]', 'Package[libwebp6]', 'Package[libcbor0]', 'Package[libusb-0.1-4]', 'Package[telnet]', 'Package[libruby2.7]', 'Package[libdns-export1110]', 'Package[libisc-export1105]', 'Package[libbpf0]', 'Package[openssh-client]', 'Package[openssh-server]', 'Package[debdeploy-client]', 'Package[python3-dateutil]', 'Package[sudo]', 'Package[golang-cfssl]', 'Package[debmonitor-client]', 'Package[perccli]', 'Package[nagios-nrpe-server]', 'Package[monitoring-plugins]', 'Package[monitoring-plugins-basic]', 'Package[monitoring-plugins-standard]', 'Package[liburiparser1]', 'Package[python3-attr]', 'Package[iucode-tool]', 'Package[freeipmi-tools]', 'Package[freeipmi-ipmiseld]', 'Package[rsyslog-kafka]', 'Package[emacs-nox]', 'Package[prometheus-ipmi-exporter]', 'Package[libnet-dns-perl]', 'Package[iptables]', 'Package[ferm]', 'Package[ulogd2]', 'Package[conntrack]', 'Package[prometheus-rsyslog-exporter]', 'Package[initramfs-tools]', 'Package[python3-click]', 'Package[python3-box]', 'Package[confd]', 'Package[python3-toml]']
  +    before => ['Package[puppet]', 'Package[facter]', 'Package[augeas-tools]', 'Package[virt-what]', 'Package[puppet-module-puppetlabs-augeas-core]', 'Package[python3-prometheus-client]', 'Package[python3-yaml]', 'Package[ruby-net-ssh]', 'Package[openssl]', 'Package[ssl-cert]', 'Package[ca-certificates]', 'Package[wmf-certificates]', 'Package[ntp]', 'Package[systemd-timesyncd]', 'Package[exim4-config]', 'Package[exim4-daemon-light]', 'Package[logrotate]', 'Package[prometheus-node-exporter]', 'Package[bsdutils]', 'Package[smartmontools]', 'Package[rsyslog]', 'Package[rsyslog-openssl]', 'Package[cadvisor]', 'Package[prometheus-ethtool-exporter]', 'Package[acct]', 'Package[byobu]', 'Package[colordiff]', 'Package[curl]', 'Package[debian-goodies]', 'Package[ethtool]', 'Package[gdb]', 'Package[gdisk]', 'Package[git]', 'Package[htop]', 'Package[httpry]', 'Package[iotop]', 'Package[iperf]', 'Package[jq]', 'Package[libtemplate-perl]', 'Package[lldpd]', 'Package[lshw]', 'Package[molly-guard]', 'Package[moreutils]', 'Package[net-tools]', 'Package[numactl]', 'Package[ncdu]', 'Package[ngrep]', 'Package[pigz]', 'Package[psmisc]', 'Package[pv]', 'Package[python3]', 'Package[screen]', 'Package[strace]', 'Package[sysstat]', 'Package[tcpdump]', 'Package[tmux]', 'Package[tree]', 'Package[vim]', 'Package[vim-addon-manager]', 'Package[vim-scripts]', 'Package[wipe]', 'Package[xfsprogs]', 'Package[zsh]', 'Package[icdiff]', 'Package[linux-perf]', 'Package[bsd-mailx]', 'Package[ack]', 'Package[netcat-openbsd]', 'Package[tshark]', 'Package[fzf]', 'Package[ripgrep]', 'Package[fd-find]', 'Package[kitty-terminfo]', 'Package[mtr-tiny]', 'Package[bat]', 'Package[efibootmgr]', 'Package[bind9-dnsutils]', 'Package[tzdata]', 'Package[python3-wmflib]', 'Package[ruby-sorted-set]', 'Package[btop]', 'Package[dstat]', 'Package[apport]', 'Package[command-not-found]', 'Package[command-not-found-data]', 'Package[ecryptfs-utils]', 'Package[mlocate]', 'Package[os-prober]', 'Package[python3-apport]', 'Package[wpasupplicant]', 'Package[atop]', 'Package[apt-listchanges]', 'Package[isc-dhcp-client]', 'Package[rasdaemon]', 'Package[libicu67]', 'Package[libwsutil12]', 'Package[libwireshark14]', 'Package[libopencsd0]', 'Package[libwiretap11]', 'Package[ruby2.7]', 'Package[python3.9-minimal]', 'Package[python3.9]', 'Package[perl-modules-5.32]', 'Package[libpython3.9]', 'Package[libperl5.32]', 'Package[libpython3.9-minimal]', 'Package[libpython3.9-stdlib]', 'Package[libidn11]', 'Package[libldap-2.4-2]', 'Package[liburing1]', 'Package[libwebp6]', 'Package[libcbor0]', 'Package[libusb-0.1-4]', 'Package[telnet]', 'Package[libruby2.7]', 'Package[libdns-export1110]', 'Package[libisc-export1105]', 'Package[libbpf0]', 'Package[openssh-client]', 'Package[openssh-server]', 'Package[debdeploy-client]', 'Package[python3-dateutil]', 'Package[sudo]', 'Package[golang-cfssl]', 'Package[debmonitor-client]', 'Package[perccli]', 'Package[nagios-nrpe-server]', 'Package[monitoring-plugins]', 'Package[monitoring-plugins-basic]', 'Package[monitoring-plugins-standard]', 'Package[liburiparser1]', 'Package[python3-attr]', 'Package[iucode-tool]', 'Package[freeipmi-tools]', 'Package[freeipmi-ipmiseld]', 'Package[rsyslog-kafka]', 'Package[emacs-nox]', 'Package[prometheus-ipmi-exporter]', 'Package[libnet-dns-perl]', 'Package[iptables]', 'Package[ferm]', 'Package[ulogd2]', 'Package[conntrack]', 'Package[wmf-mariadb1011]', 'Package[percona-toolkit]', 'Package[grc]', 'Package[mariadb-backup]', 'Package[python3-wmfmariadbpy]', 'Package[wmfmariadbpy-common]', 'Package[monitoring-plugins-contrib]', 'Package[prometheus-rsyslog-exporter]', 'Package[initramfs-tools]', 'Package[python3-click]', 'Package[python3-box]', 'Package[confd]', 'Package[python3-toml]', 'Package[prometheus-mysqld-exporter]']
  

• Systemd::Service[nrpe2nodexp-mariadb_read_only_s1]

  Parameters differences:

  --- Systemd::Service[nrpe2nodexp-mariadb_read_only_s1].orig
  +++ Systemd::Service[nrpe2nodexp-mariadb_read_only_s1]
  
  +    monitoring_enabled       => False
  +    require                  => Systemd::Unit[nrpe2nodexp-mariadb_read_only_s1.service]
  +    restart                  => False
  +    unit_type                => timer
  +    service_params           => {}
  +    monitoring_contact_group => admins
  +    monitoring_critical      => False
  +    migration_task           => T407130
  +    ensure                   => absent
  +    override                 => False
  

• File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-replica-io-state-s1.conf]

  Parameters differences:

  --- File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-replica-io-state-s1.conf].orig
  +++ File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-replica-io-state-s1.conf]
  
  +    group  => root
  +    notify => Service[rsyslog]
  +    owner  => root
  +    ensure => absent
  +    mode   => 0444
  

  Content differences:

  --- /etc/rsyslog.d/25-nrpe2nodexp-mariadb-replica-io-state-s1.conf.orig
  +++ /etc/rsyslog.d/25-nrpe2nodexp-mariadb-replica-io-state-s1.conf
  @@ -0,0 +1,10 @@
  +# SPDX-License-Identifier: Apache-2.0
  +if $programname contains "nrpe2nodexp-mariadb_replica_io_state_s1" then {
  +    if ($msg contains "\"ecs.version\": \"1.7.0\"") then {
  +        # Send logs to kafka
  +        set $.log_outputs = "kafka ecs_170 local";
  +    } else {
  +        # Filter out non-relevant nrpe2nodexp messages
  +        stop
  +    }
  +}

• Rsyslog::Conf[nrpe2nodexp-mariadb_replica_io_state_s1]

  Parameters differences:

  --- Rsyslog::Conf[nrpe2nodexp-mariadb_replica_io_state_s1].orig
  +++ Rsyslog::Conf[nrpe2nodexp-mariadb_replica_io_state_s1]
  
  +    ensure   => absent
  +    mode     => 0444
  +    priority => 25
  

• File_line[auto_restart_file_presence_prometheus-mysqld-exporter@s1]

  Parameters differences:

  --- File_line[auto_restart_file_presence_prometheus-mysqld-exporter@s1].orig
  +++ File_line[auto_restart_file_presence_prometheus-mysqld-exporter@s1]
  
  +    line    => prometheus-mysqld-exporter@s1
  +    path    => /etc/debdeploy-client/autorestarts.conf
  +    require => File[/etc/debdeploy-client/autorestarts.conf]
  +    ensure  => present
  

• Package[percona-toolkit]

  Parameters differences:

  --- Package[percona-toolkit].orig
  +++ Package[percona-toolkit]
  
  +    provider => apt
  +    ensure   => installed
  

• File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_state_s1.service]

  Parameters differences:

  --- File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_state_s1.service].orig
  +++ File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_state_s1.service]
  
  +    group  => root
  +    notify => Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_state_s1.service (nrpe2nodexp-mariadb_replica_sql_state_s1.service)]
  +    owner  => root
  +    ensure => absent
  +    mode   => 0444
  

  Content differences:

  --- /lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_state_s1.service.orig
  +++ /lib/systemd/system/nrpe2nodexp-mariadb_replica_sql_state_s1.service
  @@ -0,0 +1,11 @@
  +[Unit]
  +Description=execution of nrpe2nodexp for the check_mariadb_replica_sql_state_s1 command.
  +Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
  +
  +[Service]
  +Type=oneshot
  +User=nagios
  +
  +Group=prometheus-node-exporter
  +SyslogIdentifier=nrpe2nodexp-mariadb_replica_sql_state_s1
  +ExecStart=-/usr/local/bin/nrpe2nodexp --alert-rule-hash "3f927914a18db09399b7fff42858d634" --timeout 10 --check-command "check_mariadb_replica_sql_state_s1"

• Sudo::User[nrpe-check_mariadb_disk_space]

  Parameters differences:

  --- Sudo::User[nrpe-check_mariadb_disk_space].orig
  +++ Sudo::User[nrpe-check_mariadb_disk_space]
  
  +    privileges => []
  +    require    => ['Class[Sudo]']
  +    ensure     => absent
  +    tag        => nrpe::check
  +    user       => nagios
  

• File[/etc/mysql/mysqld.conf.d/s1.cnf]

  Parameters differences:

  --- File[/etc/mysql/mysqld.conf.d/s1.cnf].orig
  +++ File[/etc/mysql/mysqld.conf.d/s1.cnf]
  
  +    group => root
  +    owner => root
  +    mode  => 0644
  

  Content differences:

  --- /etc/mysql/mysqld.conf.d/s1.cnf.orig
  +++ /etc/mysql/mysqld.conf.d/s1.cnf
  @@ -0,0 +1,9 @@
  +[mysqld@s1]
  +datadir = /srv/sqldata.s1
  +tmpdir = /srv/tmp.s1
  +socket = /run/mysqld/mysqld.s1.sock
  +port = 3311
  +extra-port = 3331
  +innodb_buffer_pool_size = 192G
  +read-only = 1
  +

• Class[Wmfmariadbpy]

  Parameters differences:

  --- Class[Wmfmariadbpy].orig
  +++ Class[Wmfmariadbpy]
  
  +    role          => db
  +    section_ports => {'s1': 3311, 's2': 3312, 's3': 3313, 's4': 3314, 's5': 3315, 's6': 3316, 's7': 3317, 's8': 3318, 'x1': 3320, 'x3': 3363, 'm1': 3321, 'm2': 3322, 'm3': 3323, 'm5': 3325, 'staging': 3350, 'matomo': 3351, 'analytics_meta': 3352, 'backup1-eqiad': 3360, 'backup1-codfw': 3361}
  

• File[/etc/update-motd.d/05-mariadb--backup-source]

  Parameters differences:

  --- File[/etc/update-motd.d/05-mariadb--backup-source].orig
  +++ File[/etc/update-motd.d/05-mariadb--backup-source]
  
  +    group  => root
  +    owner  => root
  +    mode   => 0555
  +    ensure => present
  

  Content differences:

  --- /etc/update-motd.d/05-mariadb--backup-source.orig
  +++ /etc/update-motd.d/05-mariadb--backup-source
  @@ -0,0 +1,2 @@
  +#!/bin/sh
  +printf "%s\n" "db2250 is a MariaDB server containing replicas of mediawiki databases used to generate backups (mariadb::backup_source)"

• Sudo::User[nrpe-check_mariadb_read_only_s1]

  Parameters differences:

  --- Sudo::User[nrpe-check_mariadb_read_only_s1].orig
  +++ Sudo::User[nrpe-check_mariadb_read_only_s1]
  
  +    privileges => []
  +    require    => ['Class[Sudo]']
  +    ensure     => absent
  +    tag        => nrpe::check
  +    user       => nagios
  

• Nrpe::Check[check_mariadb_replica_io_state_s1]

  Parameters differences:

  --- Nrpe::Check[check_mariadb_replica_io_state_s1].orig
  +++ Nrpe::Check[check_mariadb_replica_io_state_s1]
  
  +    before  => Monitoring::Service[mariadb_replica_io_state_s1]
  +    command => /usr/local/lib/nagios/plugins/check_mariadb --sock=/run/mysqld/mysqld.s1.sock --user=nagios --pass=  --warn-stopped --check=slave_io_state
  +    ensure  => present
  

• Systemd::Unit[nrpe2nodexp-mariadb_replica_sql_state_s1.timer]

  Parameters differences:

  --- Systemd::Unit[nrpe2nodexp-mariadb_replica_sql_state_s1.timer].orig
  +++ Systemd::Unit[nrpe2nodexp-mariadb_replica_sql_state_s1.timer]
  
  +    unit              => nrpe2nodexp-mariadb_replica_sql_state_s1.timer
  +    require           => ['Class[Systemd]']
  +    restart           => False
  +    ensure            => absent
  +    override          => False
  +    override_filename => puppet-override.conf
  

• Sudo::User[nrpe-check_mysqld]

  Parameters differences:

  --- Sudo::User[nrpe-check_mysqld].orig
  +++ Sudo::User[nrpe-check_mysqld]
  
  +    privileges => []
  +    require    => ['Class[Sudo]']
  +    ensure     => absent
  +    tag        => nrpe::check
  +    user       => nagios
  

• Class[Mariadb::Monitor_disk]

  Parameters differences:

  --- Class[Mariadb::Monitor_disk].orig
  +++ Class[Mariadb::Monitor_disk]
  
  +    pct_warning   => 10
  +    is_critical   => False
  +    contact_group => admins
  +    pct_critical  => 5
  

• File[/etc/nagios/nrpe.d/check_mariadb_memory.cfg]

  Parameters differences:

  --- File[/etc/nagios/nrpe.d/check_mariadb_memory.cfg].orig
  +++ File[/etc/nagios/nrpe.d/check_mariadb_memory.cfg]
  
  +    group   => root
  +    notify  => Service[nagios-nrpe-server]
  +    owner   => root
  +    require => Package[nagios-nrpe-server]
  +    ensure  => present
  +    tag     => nrpe::check
  +    mode    => 0444
  

  Content differences:

  --- /etc/nagios/nrpe.d/check_mariadb_memory.cfg.orig
  +++ /etc/nagios/nrpe.d/check_mariadb_memory.cfg
  @@ -0,0 +1,2 @@
  +# File generated by puppet. DO NOT edit by hand
  +command[check_mariadb_memory]=/usr/lib/nagios/plugins/pmp-check-unix-memory -c 95 -w 90

• File[/etc/systemd/system/mariadb.service.d]

  Parameters differences:

  --- File[/etc/systemd/system/mariadb.service.d].orig
  +++ File[/etc/systemd/system/mariadb.service.d]
  
  +    group  => root
  +    owner  => root
  +    mode   => 0755
  +    ensure => directory
  

• Systemd::Timer[nrpe2nodexp-mysqld]

  Parameters differences:

  --- Systemd::Timer[nrpe2nodexp-mysqld].orig
  +++ Systemd::Timer[nrpe2nodexp-mysqld]
  
  +    fixed_random_delay => True
  +    splay              => 60
  +    unit_name          => nrpe2nodexp-mysqld.service
  +    accuracy           => 15sec
  +    ensure             => absent
  +    timer_intervals    => [{'start': 'OnUnitInactiveSec', 'interval': '1min'}, {'interval': '1s', 'start': 'OnActiveSec'}]
  

• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_io_state_s1.service (nrpe2nodexp-mariadb_replica_io_state_s1.service)]

  Parameters differences:

  --- Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_io_state_s1.service (nrpe2nodexp-mariadb_replica_io_state_s1.service)].orig
  +++ Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_io_state_s1.service (nrpe2nodexp-mariadb_replica_io_state_s1.service)]
  
  +    refreshonly => True
  +    command     => /bin/systemctl daemon-reload
  

• Package[grc]

  Parameters differences:

  --- Package[grc].orig
  +++ Package[grc]
  
  +    provider => apt
  +    ensure   => installed
  

• Sudo::User[nrpe-check_mariadb_replica_sql_state_s1]

  Parameters differences:

  --- Sudo::User[nrpe-check_mariadb_replica_sql_state_s1].orig
  +++ Sudo::User[nrpe-check_mariadb_replica_sql_state_s1]
  
  +    privileges => []
  +    require    => ['Class[Sudo]']
  +    ensure     => absent
  +    tag        => nrpe::check
  +    user       => nagios
  

• File[/usr/local/lib/nagios/plugins/check_mariadb]

  Parameters differences:

  --- File[/usr/local/lib/nagios/plugins/check_mariadb].orig
  +++ File[/usr/local/lib/nagios/plugins/check_mariadb]
  
  +    group   => root
  +    owner   => root
  +    require => File[/usr/local/lib/nagios/plugins/]
  +    ensure  => file
  +    tag     => nrpe::plugin
  +    source  => puppet:///modules/icinga/check_mariadb.pl
  +    mode    => 0555
  

• File[/etc/nagios/nrpe.d/check_mariadb_replica_sql_state_s1.cfg]

  Parameters differences:

  --- File[/etc/nagios/nrpe.d/check_mariadb_replica_sql_state_s1.cfg].orig
  +++ File[/etc/nagios/nrpe.d/check_mariadb_replica_sql_state_s1.cfg]
  
  +    group   => root
  +    notify  => Service[nagios-nrpe-server]
  +    owner   => root
  +    require => Package[nagios-nrpe-server]
  +    ensure  => present
  +    tag     => nrpe::check
  +    mode    => 0444
  

  Content differences:

  --- /etc/nagios/nrpe.d/check_mariadb_replica_sql_state_s1.cfg.orig
  +++ /etc/nagios/nrpe.d/check_mariadb_replica_sql_state_s1.cfg
  @@ -0,0 +1,2 @@
  +# File generated by puppet. DO NOT edit by hand
  +command[check_mariadb_replica_sql_state_s1]=/usr/local/lib/nagios/plugins/check_mariadb --sock=/run/mysqld/mysqld.s1.sock --user=nagios --pass=  --warn-stopped --check=slave_sql_state

• Monitoring::Service[mariadb_read_only_s1]

  Parameters differences:

  --- Monitoring::Service[mariadb_read_only_s1].orig
  +++ Monitoring::Service[mariadb_read_only_s1]
  
  +    freshness      => 36000
  +    config_dir     => /etc/nagios
  +    passive        => False
  +    check_command  => nrpe_check!check_mariadb_read_only_s1!10
  +    host           => db2250
  +    critical       => False
  +    contact_group  => admins
  +    check_interval => 1
  +    description    => MariaDB read only s1
  +    notes_url      => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Master_comes_back_in_read_only
  +    retry_interval => 1
  +    ensure         => present
  +    migration_task => T315866
  +    retries        => 3
  

• Systemd::Unit[nrpe2nodexp-mariadb_replica_sql_lag_s1.timer]

  Parameters differences:

  --- Systemd::Unit[nrpe2nodexp-mariadb_replica_sql_lag_s1.timer].orig
  +++ Systemd::Unit[nrpe2nodexp-mariadb_replica_sql_lag_s1.timer]
  
  +    unit              => nrpe2nodexp-mariadb_replica_sql_lag_s1.timer
  +    require           => ['Class[Systemd]']
  +    restart           => False
  +    ensure            => absent
  +    override          => False
  +    override_filename => puppet-override.conf
  

• File[/etc/sudoers.d/nrpe-check_mariadb_memory]

  Parameters differences:

  --- File[/etc/sudoers.d/nrpe-check_mariadb_memory].orig
  +++ File[/etc/sudoers.d/nrpe-check_mariadb_memory]
  
  +    group   => root
  +    owner   => root
  +    require => Package[nagios-nrpe-server]
  +    ensure  => absent
  

• File[/opt/wmf-mariadb1011/my.cnf]

  Parameters differences:

  --- File[/opt/wmf-mariadb1011/my.cnf].orig
  +++ File[/opt/wmf-mariadb1011/my.cnf]
  
  +    group  => root
  +    owner  => root
  +    ensure => absent
  

• Class[Profile::Mariadb::Mysql_role]

  Parameters differences:

  --- Class[Profile::Mariadb::Mysql_role].orig
  +++ Class[Profile::Mariadb::Mysql_role]
  
  +    role => slave
  

• Package[mariadb-backup]

  Parameters differences:

  --- Package[mariadb-backup].orig
  +++ Package[mariadb-backup]
  
  +    provider => apt
  +    ensure   => installed
  

• Monitoring::Exported_nagios_service[db2250 mariadb_disk_space]

  Parameters differences:

  --- Monitoring::Exported_nagios_service[db2250 mariadb_disk_space].orig
  +++ Monitoring::Exported_nagios_service[db2250 mariadb_disk_space]
  
  +    check_command          => nrpe_check!check_mariadb_disk_space!10
  +    notification_period    => 24x7
  +    notification_interval  => 0
  +    check_period           => 24x7
  +    service_description    => MariaDB disk space
  +    max_check_attempts     => 3
  +    notes_url              => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting
  +    notifications_enabled  => 0
  +    is_volatile            => 0
  +    retry_interval         => 1
  +    ensure                 => present
  +    active_checks_enabled  => 1
  +    notification_options   => c,r,f
  +    contact_groups         => admins
  +    check_interval         => 1
  +    servicegroups          => mysql_codfw
  +    passive_checks_enabled => 1
  +    host_name              => db2250
  +    check_freshness        => 0
  

• File[/etc/wmfmariadbpy/section_ports.csv]

  Parameters differences:

  --- File[/etc/wmfmariadbpy/section_ports.csv].orig
  +++ File[/etc/wmfmariadbpy/section_ports.csv]
  
  +    group  => root
  +    owner  => root
  +    mode   => 0644
  +    ensure => file
  

  Content differences:

  --- /etc/wmfmariadbpy/section_ports.csv.orig
  +++ /etc/wmfmariadbpy/section_ports.csv
  @@ -0,0 +1,19 @@
  +s1, 3311
  +s2, 3312
  +s3, 3313
  +s4, 3314
  +s5, 3315
  +s6, 3316
  +s7, 3317
  +s8, 3318
  +x1, 3320
  +x3, 3363
  +m1, 3321
  +m2, 3322
  +m3, 3323
  +m5, 3325
  +staging, 3350
  +matomo, 3351
  +analytics_meta, 3352
  +backup1-eqiad, 3360
  +backup1-codfw, 3361

• File[/var/lib/prometheus/node.d/check_mariadb_read_only_s1.prom]

  Parameters differences:

  --- File[/var/lib/prometheus/node.d/check_mariadb_read_only_s1.prom].orig
  +++ File[/var/lib/prometheus/node.d/check_mariadb_read_only_s1.prom]
  
  +    group  => root
  +    owner  => root
  +    ensure => absent
  

• Sudo::User[nrpe-check_mariadb_replica_io_state_s1]

  Parameters differences:

  --- Sudo::User[nrpe-check_mariadb_replica_io_state_s1].orig
  +++ Sudo::User[nrpe-check_mariadb_replica_io_state_s1]
  
  +    privileges => []
  +    require    => ['Class[Sudo]']
  +    ensure     => absent
  +    tag        => nrpe::check
  +    user       => nagios
  

• File[/etc/my.cnf]

  Parameters differences:

  --- File[/etc/my.cnf].orig
  +++ File[/etc/my.cnf]
  
  +    group => root
  +    owner => root
  +    mode  => 0644
  

  Content differences:

  --- /etc/my.cnf.orig
  +++ /etc/my.cnf
  @@ -0,0 +1,128 @@
  +# replicas for backup sources and analytics
  +
  +# Please use separate .cnf templates for each type of server.
  +
  +[client]
  +
  +[mysqld]
  +
  +skip-external-locking
  +skip-name-resolve
  +temp-pool
  +
  +user               = mysql
  +basedir            = /opt/wmf-mariadb1011
  +server_id          = 180356362
  +report_host        = db2250.codfw.wmnet
  +wsrep_cluster_name = codfw # For orchestrator. T266635
  +
  +# gtid_domain_id flag is needed for multisource replication and GTID.
  +# Strictly it is only needed on masters or servers that can potentially be
  +# masters but for consistency it should be set in all of them.
  +# https://mariadb.com/kb/en/mariadb/gtid/
  +gtid_domain_id  = 180356362
  +
  +read_only       = 1
  +key_buffer_size = 1048576
  +
  +secure_file_priv   = /dev/null
  +max_connections    = 250
  +max_allowed_packet = 32M
  +connect_timeout    = 3
  +query_cache_size   = 0
  +query_cache_type   = 0
  +event_scheduler    = 1
  +
  +table_open_cache       = 50000
  +table_definition_cache = 50000
  +
  +# binary log and replication options
  +# We are temporarily enabling the binlogs on these servers to evaluate
  +# CDC functionality with paimon/flink.
  +#
  +# We will revert this change once the evaluation is complete.
  +# See #T258511 for a discussion about the long-term aim and #T358373.
  +log-bin
  +log-slave-updates
  +binlog-format                  = ROW
  +sync-binlog                    = 1
  +binlog_cache_size              = 10M
  +max-binlog_size                = 1000M
  +expire-logs-days               = 7
  +
  +# InnoDB file-per-table + TokuDB love those file descriptors
  +open-files-limit = 200000
  +
  +character_set_server     = binary
  +character_set_filesystem = binary
  +collation_server         = binary
  +
  +innodb_file_per_table          = 1
  +# this can get overiden by per-instance configuration
  +innodb_buffer_pool_size        = 15G
  +innodb_log_file_size           = 4G
  +innodb_flush_log_at_trx_commit = 0
  +innodb_flush_method            = O_DIRECT
  +innodb_io_capacity             = 1000
  +innodb_stats_method            = nulls_unequal
  +innodb_purge_threads           = 1
  +# Changed because of https://jira.mariadb.org/browse/MDEV-22373
  +innodb_change_buffering        = none
  +
  +innodb_strict_mode             = 1
  +
  +# T287244
  +innodb_checksum_algorithm = full_crc32
  +
  +# T268869
  +innodb_adaptive_hash_index=OFF
  +
  +# In 10.6 the default is 90%, 75% worked fine for us.
  +innodb_max_dirty_pages_pct = 75
  +
  +# don't dump and load innodb buffer at start and stop
  +# these servers will have the buffer pool in a bad state due to dumps
  +# so not sure if these will be useful, but we leave them here for now
  +innodb_buffer_pool_load_at_startup  = 1
  +innodb_buffer_pool_dump_at_shutdown = 1
  +
  +optimizer_switch='engine_condition_pushdown=on,optimize_join_buffer_size=on'
  +join_cache_level = 8
  +
  +skip-slave-start
  +slave_transaction_retries = 4294967295
  +
  +
  +# ssl
  +ssl-ca=/etc/ssl/certs/wmf-ca-certificates.crt
  +ssl-cert=/etc/mysql/ssl/cert.pem
  +ssl-key=/etc/mysql/ssl/server.key
  +ssl-cipher=TLSv1.2
  +
  +
  +# Enabling performance_schema (disabled by default in MariaDB10)
  +performance_schema                                     = 1
  +# downsizing performance schema memory usage: T99485
  +performance_schema_digests_size                        = -1
  +performance_schema_max_thread_instances                = 500
  +performance_schema_max_cond_instances                  = 1000
  +performance_schema_accounts_size                       = 300
  +performance_schema_hosts_size                          = 300
  +performance_schema_events_statements_history_size      = 10
  +performance_schema_events_statements_history_long_size = 1000
  +performance_schema_events_waits_history_size           = 10
  +performance_schema_events_waits_history_long_size      = 1000
  +performance_schema_events_stages_history_size          = 10
  +performance_schema_events_stages_history_long_size     = 1000
  +performance_schema_max_mutex_instances                 = 5000
  +performance_schema_max_rwlock_instances                = 2000
  +performance_schema_max_socket_instances                = 500
  +performance_schema_max_table_instances                 = 1000
  +
  +
  +[mysqldump]
  +
  +quick
  +max_allowed_packet = 32M
  +
  +!includedir /etc/mysql/mysqld.conf.d/

• Nrpe::Check[check_mariadb_replica_sql_lag_s1]

  Parameters differences:

  --- Nrpe::Check[check_mariadb_replica_sql_lag_s1].orig
  +++ Nrpe::Check[check_mariadb_replica_sql_lag_s1]
  
  +    before  => Monitoring::Service[mariadb_replica_sql_lag_s1]
  +    command => /usr/local/lib/nagios/plugins/check_mariadb --sock=/run/mysqld/mysqld.s1.sock --user=nagios --pass=  --warn-stopped --check=slave_sql_lag                            --shard=s1 --datacenter=eqiad                            --sql-lag-warn=60                            --sql-lag-crit=300
  +    ensure  => present
  

• File[/etc/mysql/ssl/client-key.pem]

  Parameters differences:

  --- File[/etc/mysql/ssl/client-key.pem].orig
  +++ File[/etc/mysql/ssl/client-key.pem]
  
  +    group  => root
  +    owner  => root
  +    ensure => absent
  

• Systemd::Unit[wmf_auto_restart_prometheus-mysqld-exporter@s1.timer]

  Parameters differences:

  --- Systemd::Unit[wmf_auto_restart_prometheus-mysqld-exporter@s1.timer].orig
  +++ Systemd::Unit[wmf_auto_restart_prometheus-mysqld-exporter@s1.timer]
  
  +    unit              => wmf_auto_restart_prometheus-mysqld-exporter@s1.timer
  +    require           => ['Class[Systemd]']
  +    restart           => False
  +    ensure            => present
  +    override          => False
  +    override_filename => puppet-override.conf
  

• File[/lib/systemd/system/nrpe2nodexp-mariadb_memory.service]

  Parameters differences:

  --- File[/lib/systemd/system/nrpe2nodexp-mariadb_memory.service].orig
  +++ File[/lib/systemd/system/nrpe2nodexp-mariadb_memory.service]
  
  +    group  => root
  +    notify => Exec[systemd daemon-reload for nrpe2nodexp-mariadb_memory.service (nrpe2nodexp-mariadb_memory.service)]
  +    owner  => root
  +    ensure => absent
  +    mode   => 0444
  

  Content differences:

  --- /lib/systemd/system/nrpe2nodexp-mariadb_memory.service.orig
  +++ /lib/systemd/system/nrpe2nodexp-mariadb_memory.service
  @@ -0,0 +1,11 @@
  +[Unit]
  +Description=execution of nrpe2nodexp for the check_mariadb_memory command.
  +Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
  +
  +[Service]
  +Type=oneshot
  +User=nagios
  +
  +Group=prometheus-node-exporter
  +SyslogIdentifier=nrpe2nodexp-mariadb_memory
  +ExecStart=-/usr/local/bin/nrpe2nodexp --alert-rule-hash "b5a92c72542d5f0c031014e7a8db014e" --timeout 10 --check-command "check_mariadb_memory"

• Systemd::Unit[nrpe2nodexp-mysqld.service]

  Parameters differences:

  --- Systemd::Unit[nrpe2nodexp-mysqld.service].orig
  +++ Systemd::Unit[nrpe2nodexp-mysqld.service]
  
  +    unit              => nrpe2nodexp-mysqld.service
  +    require           => ['Class[Systemd]']
  +    restart           => False
  +    ensure            => absent
  +    override          => False
  +    override_filename => puppet-override.conf
  

• Systemd::Timer[nrpe2nodexp-mariadb_disk_space]

  Parameters differences:

  --- Systemd::Timer[nrpe2nodexp-mariadb_disk_space].orig
  +++ Systemd::Timer[nrpe2nodexp-mariadb_disk_space]
  
  +    fixed_random_delay => True
  +    splay              => 60
  +    unit_name          => nrpe2nodexp-mariadb_disk_space.service
  +    accuracy           => 15sec
  +    ensure             => absent
  +    timer_intervals    => [{'start': 'OnUnitInactiveSec', 'interval': '1min'}, {'interval': '1s', 'start': 'OnActiveSec'}]
  

• Exec[systemctl-daemon-reload]

  Parameters differences:

  --- Exec[systemctl-daemon-reload].orig
  +++ Exec[systemctl-daemon-reload]
  
  +    refreshonly => True
  +    command     => /bin/systemctl daemon-reload
  

• Systemd::Unit[nrpe2nodexp-mariadb_disk_space.timer]

  Parameters differences:

  --- Systemd::Unit[nrpe2nodexp-mariadb_disk_space.timer].orig
  +++ Systemd::Unit[nrpe2nodexp-mariadb_disk_space.timer]
  
  +    unit              => nrpe2nodexp-mariadb_disk_space.timer
  +    require           => ['Class[Systemd]']
  +    restart           => False
  +    ensure            => absent
  +    override          => False
  +    override_filename => puppet-override.conf
  

• File[/etc/mysql/ssl/server-cert.pem]

  Parameters differences:

  --- File[/etc/mysql/ssl/server-cert.pem].orig
  +++ File[/etc/mysql/ssl/server-cert.pem]
  
  +    group  => root
  +    owner  => root
  +    ensure => absent
  

• File[/usr/local/bin/mbstream]

  Parameters differences:

  --- File[/usr/local/bin/mbstream].orig
  +++ File[/usr/local/bin/mbstream]
  
  +    group   => root
  +    owner   => root
  +    require => Package[wmf-mariadb1011]
  +    target  => /opt/wmf-mariadb1011/bin/mbstream
  +    ensure  => link
  

• Monitoring::Exported_nagios_service[db2250 mysqld]

  Parameters differences:

  --- Monitoring::Exported_nagios_service[db2250 mysqld].orig
  +++ Monitoring::Exported_nagios_service[db2250 mysqld]
  
  +    check_command          => nrpe_check!check_mysqld!10
  +    notification_period    => 24x7
  +    notification_interval  => 0
  +    check_period           => 24x7
  +    service_description    => mysqld processes
  +    max_check_attempts     => 3
  +    notes_url              => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting
  +    notifications_enabled  => 0
  +    is_volatile            => 0
  +    retry_interval         => 1
  +    ensure                 => present
  +    active_checks_enabled  => 1
  +    notification_options   => c,r,f
  +    contact_groups         => admins
  +    check_interval         => 1
  +    servicegroups          => mysql_codfw
  +    passive_checks_enabled => 1
  +    host_name              => db2250
  +    check_freshness        => 0
  

• Class[Profile::Base::Production]

  Parameters differences:

  --- Class[Profile::Base::Production].orig
  +++ Class[Profile::Base::Production]
  
  @@
  -    role_description => Host being setup by Data Persistence SREs
  +    role_description => MariaDB server containing replicas of mediawiki databases used to generate backups
  

• File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-read-only-s1.conf]

  Parameters differences:

  --- File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-read-only-s1.conf].orig
  +++ File[/etc/rsyslog.d/25-nrpe2nodexp-mariadb-read-only-s1.conf]
  
  +    group  => root
  +    notify => Service[rsyslog]
  +    owner  => root
  +    ensure => absent
  +    mode   => 0444
  

  Content differences:

  --- /etc/rsyslog.d/25-nrpe2nodexp-mariadb-read-only-s1.conf.orig
  +++ /etc/rsyslog.d/25-nrpe2nodexp-mariadb-read-only-s1.conf
  @@ -0,0 +1,10 @@
  +# SPDX-License-Identifier: Apache-2.0
  +if $programname contains "nrpe2nodexp-mariadb_read_only_s1" then {
  +    if ($msg contains "\"ecs.version\": \"1.7.0\"") then {
  +        # Send logs to kafka
  +        set $.log_outputs = "kafka ecs_170 local";
  +    } else {
  +        # Filter out non-relevant nrpe2nodexp messages
  +        stop
  +    }
  +}

• Nrpe::Monitor_service[mariadb_replica_io_state_s1]

  Parameters differences:

  --- Nrpe::Monitor_service[mariadb_replica_io_state_s1].orig
  +++ Nrpe::Monitor_service[mariadb_replica_io_state_s1]
  
  +    enable_nrpe2nodexp          => False
  +    alertmanager_team           => observability
  +    critical                    => False
  +    contact_group               => admins
  +    enable_icinga_check         => True
  +    check_interval              => 1
  +    description                 => MariaDB Replica IO: s1
  +    notes_url                   => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica
  +    nrpe_command                => /usr/local/lib/nagios/plugins/check_mariadb --sock=/run/mysqld/mysqld.s1.sock --user=nagios --pass=  --warn-stopped --check=slave_io_state
  +    nrpe2nodexp_parse_perf_data => False
  +    migration_task              => T315866
  +    ensure                      => present
  +    retry_interval              => 1
  +    retries                     => 3
  +    timeout                     => 10
  

• File[/lib/systemd/system/wmf_auto_restart_prometheus-mysqld-exporter@s1.service]

  Parameters differences:

  --- File[/lib/systemd/system/wmf_auto_restart_prometheus-mysqld-exporter@s1.service].orig
  +++ File[/lib/systemd/system/wmf_auto_restart_prometheus-mysqld-exporter@s1.service]
  
  +    group  => root
  +    notify => Exec[systemd daemon-reload for wmf_auto_restart_prometheus-mysqld-exporter@s1.service (wmf_auto_restart_prometheus-mysqld-exporter@s1.service)]
  +    owner  => root
  +    ensure => present
  +    mode   => 0444
  

  Content differences:

  --- /lib/systemd/system/wmf_auto_restart_prometheus-mysqld-exporter@s1.service.orig
  +++ /lib/systemd/system/wmf_auto_restart_prometheus-mysqld-exporter@s1.service
  @@ -0,0 +1,8 @@
  +[Unit]
  +Description=Auto restart job: prometheus-mysqld-exporter@s1
  +Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
  +
  +[Service]
  +Type=oneshot
  +User=root
  +ExecStart=/usr/local/sbin/wmf-auto-restart -s prometheus-mysqld-exporter@s1

• Systemd::Timer[wmf_auto_restart_prometheus-mysqld-exporter@s1]

  Parameters differences:

  --- Systemd::Timer[wmf_auto_restart_prometheus-mysqld-exporter@s1].orig
  +++ Systemd::Timer[wmf_auto_restart_prometheus-mysqld-exporter@s1]
  
  +    fixed_random_delay => False
  +    splay              => 0
  +    unit_name          => wmf_auto_restart_prometheus-mysqld-exporter@s1.service
  +    accuracy           => 15sec
  +    ensure             => present
  +    timer_intervals    => [{'start': 'OnCalendar', 'interval': 'Mon,Tue,Wed,Thu,Fri *-*-* 0:55:00'}]
  

• File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_io_state_s1.timer]

  Parameters differences:

  --- File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_io_state_s1.timer].orig
  +++ File[/lib/systemd/system/nrpe2nodexp-mariadb_replica_io_state_s1.timer]
  
  +    group  => root
  +    notify => Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_io_state_s1.timer (nrpe2nodexp-mariadb_replica_io_state_s1.timer)]
  +    owner  => root
  +    ensure => absent
  +    mode   => 0444
  

  Content differences:

  --- /lib/systemd/system/nrpe2nodexp-mariadb_replica_io_state_s1.timer.orig
  +++ /lib/systemd/system/nrpe2nodexp-mariadb_replica_io_state_s1.timer
  @@ -0,0 +1,14 @@
  +[Unit]
  +Description=Periodic execution of nrpe2nodexp-mariadb_replica_io_state_s1.service
  +
  +[Timer]
  +Unit=nrpe2nodexp-mariadb_replica_io_state_s1.service
  +# Accuracy sets the maximum time interval around the execution time we want to allow
  +AccuracySec=15sec
  +OnUnitInactiveSec=1min
  +OnActiveSec=1s
  +RandomizedDelaySec=60
  +FixedRandomDelay=true
  +
  +[Install]
  +WantedBy=multi-user.target

• Systemd::Timer::Job[nrpe2nodexp-mariadb_replica_sql_lag_s1]

  Parameters differences:

  --- Systemd::Timer::Job[nrpe2nodexp-mariadb_replica_sql_lag_s1].orig
  +++ Systemd::Timer::Job[nrpe2nodexp-mariadb_replica_sql_lag_s1]
  
  +    group                     => prometheus-node-exporter
  +    command                   => /usr/local/bin/nrpe2nodexp --alert-rule-hash "8ec9d6c1155ae60699aaf9dfd1fa339b" --timeout 10 --check-command "check_mariadb_replica_sql_lag_s1"
  +    private_tmp               => False
  +    logfile_basedir           => /var/log
  +    user                      => nagios
  +    syslog_identifier         => nrpe2nodexp-mariadb_replica_sql_lag_s1
  +    send_mail_to              => root@db2250.codfw.wmnet
  +    success_exit_status       => []
  +    fixed_random_delay        => True
  +    description               => execution of nrpe2nodexp for the check_mariadb_replica_sql_lag_s1 command.
  +    ensure                    => absent
  +    logfile_group             => root
  +    monitoring_enabled        => False
  +    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
  +    syslog_match_startswith   => True
  +    interval                  => [{'start': 'OnUnitInactiveSec', 'interval': '1min'}]
  +    ignore_errors             => True
  +    send_mail_only_on_error   => True
  +    logfile_name              => syslog.log
  +    send_mail                 => False
  +    splay                     => 60
  +    syslog_force_stop         => True
  +    logfile_perms             => all
  +    environment               => {}
  +    logging_enabled           => False
  +    monitoring_contact_groups => admins
  

• Systemd::Unit[wmf_auto_restart_prometheus-mysqld-exporter@s1.service]

  Parameters differences:

  --- Systemd::Unit[wmf_auto_restart_prometheus-mysqld-exporter@s1.service].orig
  +++ Systemd::Unit[wmf_auto_restart_prometheus-mysqld-exporter@s1.service]
  
  +    unit              => wmf_auto_restart_prometheus-mysqld-exporter@s1.service
  +    require           => ['Class[Systemd]']
  +    restart           => False
  +    ensure            => present
  +    override          => False
  +    override_filename => puppet-override.conf
  

• File[/srv/sqldata.s1]

  Parameters differences:

  --- File[/srv/sqldata.s1].orig
  +++ File[/srv/sqldata.s1]
  
  +    group  => mysql
  +    owner  => mysql
  +    mode   => 0755
  +    ensure => directory
  

• Monitoring::Service[mysqld]

  Parameters differences:

  --- Monitoring::Service[mysqld].orig
  +++ Monitoring::Service[mysqld]
  
  +    freshness      => 36000
  +    config_dir     => /etc/nagios
  +    passive        => False
  +    check_command  => nrpe_check!check_mysqld!10
  +    host           => db2250
  +    critical       => False
  +    contact_group  => admins
  +    check_interval => 1
  +    description    => mysqld processes
  +    notes_url      => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting
  +    retry_interval => 1
  +    ensure         => present
  +    migration_task => T315866
  +    retries        => 3
  

• Service[nrpe2nodexp-mariadb_replica_sql_lag_s1.timer]

  Parameters differences:

  --- Service[nrpe2nodexp-mariadb_replica_sql_lag_s1.timer].orig
  +++ Service[nrpe2nodexp-mariadb_replica_sql_lag_s1.timer]
  
  +    before   => ['Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_lag_s1.timer (nrpe2nodexp-mariadb_replica_sql_lag_s1.timer)]']
  +    provider => systemd
  +    enable   => False
  +    ensure   => stopped
  

• File[/etc/mysql/ssl/server.key]

  Parameters differences:

  --- File[/etc/mysql/ssl/server.key].orig
  +++ File[/etc/mysql/ssl/server.key]
  
  +    group     => mysql
  +    owner     => mysql
  +    show_diff => False
  +    ensure    => present
  +    source    => /var/lib/puppet/ssl/private_keys/db2250.codfw.wmnet.pem
  +    mode      => 0400
  

• File[/etc/sudoers.d/nrpe-check_mariadb_replica_sql_lag_s1]

  Parameters differences:

  --- File[/etc/sudoers.d/nrpe-check_mariadb_replica_sql_lag_s1].orig
  +++ File[/etc/sudoers.d/nrpe-check_mariadb_replica_sql_lag_s1]
  
  +    group   => root
  +    owner   => root
  +    require => Package[nagios-nrpe-server]
  +    ensure  => absent
  

• Systemd::Unit[nrpe2nodexp-mysqld.timer]

  Parameters differences:

  --- Systemd::Unit[nrpe2nodexp-mysqld.timer].orig
  +++ Systemd::Unit[nrpe2nodexp-mysqld.timer]
  
  +    unit              => nrpe2nodexp-mysqld.timer
  +    require           => ['Class[Systemd]']
  +    restart           => False
  +    ensure            => absent
  +    override          => False
  +    override_filename => puppet-override.conf
  

• Systemd::Service[wmf_auto_restart_prometheus-mysqld-exporter@s1]

  Parameters differences:

  --- Systemd::Service[wmf_auto_restart_prometheus-mysqld-exporter@s1].orig
  +++ Systemd::Service[wmf_auto_restart_prometheus-mysqld-exporter@s1]
  
  +    monitoring_enabled       => False
  +    require                  => Systemd::Unit[wmf_auto_restart_prometheus-mysqld-exporter@s1.service]
  +    restart                  => False
  +    unit_type                => timer
  +    service_params           => {}
  +    monitoring_contact_group => admins
  +    monitoring_critical      => False
  +    migration_task           => T407130
  +    ensure                   => present
  +    override                 => False
  

• Nrpe::Check[check_mariadb_read_only_s1]

  Parameters differences:

  --- Nrpe::Check[check_mariadb_read_only_s1].orig
  +++ Nrpe::Check[check_mariadb_read_only_s1]
  
  +    before  => Monitoring::Service[mariadb_read_only_s1]
  +    command => db-check-health --port=3311 --icinga --check_read_only=1 --process
  +    ensure  => present
  

• Rsyslog::Conf[nrpe2nodexp-mariadb_disk_space]

  Parameters differences:

  --- Rsyslog::Conf[nrpe2nodexp-mariadb_disk_space].orig
  +++ Rsyslog::Conf[nrpe2nodexp-mariadb_disk_space]
  
  +    ensure   => absent
  +    mode     => 0444
  +    priority => 25
  

• Monitoring::Exported_nagios_service[db2250 mariadb_replica_sql_lag_s1]

  Parameters differences:

  --- Monitoring::Exported_nagios_service[db2250 mariadb_replica_sql_lag_s1].orig
  +++ Monitoring::Exported_nagios_service[db2250 mariadb_replica_sql_lag_s1]
  
  +    check_command          => nrpe_check!check_mariadb_replica_sql_lag_s1!10
  +    notification_period    => 24x7
  +    notification_interval  => 0
  +    check_period           => 24x7
  +    service_description    => MariaDB Replica Lag: s1
  +    max_check_attempts     => 10
  +    notes_url              => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica
  +    notifications_enabled  => 0
  +    is_volatile            => 0
  +    retry_interval         => 1
  +    ensure                 => present
  +    active_checks_enabled  => 1
  +    notification_options   => c,r,f
  +    contact_groups         => admins
  +    check_interval         => 1
  +    servicegroups          => mysql_codfw
  +    passive_checks_enabled => 1
  +    host_name              => db2250
  +    check_freshness        => 0
  

• File[/etc/ferm/conf.d/10_s1_mariadb_dba]

  Parameters differences:

  --- File[/etc/ferm/conf.d/10_s1_mariadb_dba].orig
  +++ File[/etc/ferm/conf.d/10_s1_mariadb_dba]
  
  +    group   => root
  +    notify  => Service[ferm]
  +    owner   => root
  +    require => File[/etc/ferm/conf.d]
  +    ensure  => present
  +    tag     => ferm
  +    mode    => 0400
  

  Content differences:

  --- /etc/ferm/conf.d/10_s1_mariadb_dba.orig
  +++ /etc/ferm/conf.d/10_s1_mariadb_dba
  @@ -0,0 +1,6 @@
  +# Autogenerated by puppet. DO NOT EDIT BY HAND!
  +#
  +# 
  +&R_SERVICE(tcp, 3331, $MYSQL_ROOT_CLIENTS);
  +
  +

• Group[mysql]

  Parameters differences:

  --- Group[mysql].orig
  +++ Group[mysql]
  
  +    system => True
  +    ensure => present
  

• File[/var/lib/prometheus/node.d/check_mysqld.prom]

  Parameters differences:

  --- File[/var/lib/prometheus/node.d/check_mysqld.prom].orig
  +++ File[/var/lib/prometheus/node.d/check_mysqld.prom]
  
  +    group  => root
  +    owner  => root
  +    ensure => absent
  

• File[/etc/sudoers.d/nrpe-check_mariadb_read_only_s1]

  Parameters differences:

  --- File[/etc/sudoers.d/nrpe-check_mariadb_read_only_s1].orig
  +++ File[/etc/sudoers.d/nrpe-check_mariadb_read_only_s1]
  
  +    group   => root
  +    owner   => root
  +    require => Package[nagios-nrpe-server]
  +    ensure  => absent
  

• Prometheus::Alert::Rule[check_mariadb_memory_b5a92c72542d5f0c031014e7a8db014e]

  Parameters differences:

  --- Prometheus::Alert::Rule[check_mariadb_memory_b5a92c72542d5f0c031014e7a8db014e].orig
  +++ Prometheus::Alert::Rule[check_mariadb_memory_b5a92c72542d5f0c031014e7a8db014e]
  
  +    group              => nrpechecks
  +    team               => observability
  +    for                => 3m
  +    site               => codfw
  +    severity           => info
  +    dashboard          => TODO
  +    summary            => NRPE CHECK: MariaDB memory
  +    logs               => https://logstash.wikimedia.org/app/dashboards#/view/2d343ac0-6df8-11f0-8e08-7fab0da52b33?_g=(filters:!((query:(match_phrase:(event.module:check_mariadb_memory))),(query:(match_phrase:(host.name:{{$labels.instance|stripPort}})))))
  +    instance           => ops
  +    alert_name         => nrpe_MariaDB_memory
  +    description        => NRPE CHECK: MariaDB memory
  +    runbook            => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting
  +    ensure             => absent
  +    def_label_whitelst => ['team', 'severity']
  +    expr               => (nagios_nrpe_check_result{alert_rule_hash="b5a92c72542d5f0c031014e7a8db014e",check_name="check_mariadb_memory", status=~"(WARNING|CRITICAL)", severity=~"(warning|critical)"} > 0) * on (instance) group_left (team) role_owner
  

• Ferm::Service[s1_orchestrator]

  Parameters differences:

  --- Ferm::Service[s1_orchestrator].orig
  +++ Ferm::Service[s1_orchestrator]
  
  +    srange  => ['dborch1001.wikimedia.org', 'dborch1002.wikimedia.org', 'dborch1003.eqiad.wmnet']
  +    prio    => 10
  +    proto   => tcp
  +    ensure  => present
  +    port    => [3311]
  +    notrack => True
  +    desc    => 
  

• Prometheus::Alert::Rule[check_mariadb_replica_io_state_s1_0d5979f7abbb49256cfedd70a646784e]

  Parameters differences:

  --- Prometheus::Alert::Rule[check_mariadb_replica_io_state_s1_0d5979f7abbb49256cfedd70a646784e].orig
  +++ Prometheus::Alert::Rule[check_mariadb_replica_io_state_s1_0d5979f7abbb49256cfedd70a646784e]
  
  +    group              => nrpechecks
  +    team               => observability
  +    for                => 3m
  +    site               => codfw
  +    severity           => info
  +    dashboard          => TODO
  +    summary            => NRPE CHECK: MariaDB Replica IO: s1
  +    logs               => https://logstash.wikimedia.org/app/dashboards#/view/2d343ac0-6df8-11f0-8e08-7fab0da52b33?_g=(filters:!((query:(match_phrase:(event.module:check_mariadb_replica_io_state_s1))),(query:(match_phrase:(host.name:{{$labels.instance|stripPort}})))))
  +    instance           => ops
  +    alert_name         => nrpe_MariaDB_Replica_IO__s1
  +    description        => NRPE CHECK: MariaDB Replica IO: s1
  +    runbook            => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica
  +    ensure             => absent
  +    def_label_whitelst => ['team', 'severity']
  +    expr               => (nagios_nrpe_check_result{alert_rule_hash="0d5979f7abbb49256cfedd70a646784e",check_name="check_mariadb_replica_io_state_s1", status=~"(WARNING|CRITICAL)", severity=~"(warning|critical)"} > 0) * on (instance) group_left (team) role_owner
  

• Systemd::Unit[nrpe2nodexp-mariadb_read_only_s1.timer]

  Parameters differences:

  --- Systemd::Unit[nrpe2nodexp-mariadb_read_only_s1.timer].orig
  +++ Systemd::Unit[nrpe2nodexp-mariadb_read_only_s1.timer]
  
  +    unit              => nrpe2nodexp-mariadb_read_only_s1.timer
  +    require           => ['Class[Systemd]']
  +    restart           => False
  +    ensure            => absent
  +    override          => False
  +    override_filename => puppet-override.conf
  

• Systemd::Service[nrpe2nodexp-mariadb_memory]

  Parameters differences:

  --- Systemd::Service[nrpe2nodexp-mariadb_memory].orig
  +++ Systemd::Service[nrpe2nodexp-mariadb_memory]
  
  +    monitoring_enabled       => False
  +    require                  => Systemd::Unit[nrpe2nodexp-mariadb_memory.service]
  +    restart                  => False
  +    unit_type                => timer
  +    service_params           => {}
  +    monitoring_contact_group => admins
  +    monitoring_critical      => False
  +    migration_task           => T407130
  +    ensure                   => absent
  +    override                 => False
  

• File[/etc/mysql/grcat.config]

  Parameters differences:

  --- File[/etc/mysql/grcat.config].orig
  +++ File[/etc/mysql/grcat.config]
  
  +    group  => root
  +    owner  => root
  +    ensure => present
  +    source => puppet:///modules/mariadb/grcat.config
  +    mode   => 0644
  

• Mariadb::Instance[s1]

  Parameters differences:

  --- Mariadb::Instance[s1].orig
  +++ Mariadb::Instance[s1]
  
  +    mysqld_extra_configs    => {}
  +    innodb_buffer_pool_size => 192G
  +    template                => mariadb/instance.cnf.erb
  +    tmpdir                  => undefined
  +    socket                  => undefined
  +    datadir                 => undefined
  +    event_scheduler         => 1
  +    is_critical             => False
  +    port                    => 3311
  +    source_dc               => eqiad
  +    read_only               => 1
  

• Systemd::Unit[nrpe2nodexp-mariadb_read_only_s1.service]

  Parameters differences:

  --- Systemd::Unit[nrpe2nodexp-mariadb_read_only_s1.service].orig
  +++ Systemd::Unit[nrpe2nodexp-mariadb_read_only_s1.service]
  
  +    unit              => nrpe2nodexp-mariadb_read_only_s1.service
  +    require           => ['Class[Systemd]']
  +    restart           => False
  +    ensure            => absent
  +    override          => False
  +    override_filename => puppet-override.conf
  

• Nrpe::Plugin[check_mariadb]

  Parameters differences:

  --- Nrpe::Plugin[check_mariadb].orig
  +++ Nrpe::Plugin[check_mariadb]
  
  +    source => puppet:///modules/icinga/check_mariadb.pl
  +    ensure => present
  

• Systemd::Timer::Job[nrpe2nodexp-mariadb_disk_space]

  Parameters differences:

  --- Systemd::Timer::Job[nrpe2nodexp-mariadb_disk_space].orig
  +++ Systemd::Timer::Job[nrpe2nodexp-mariadb_disk_space]
  
  +    group                     => prometheus-node-exporter
  +    command                   => /usr/local/bin/nrpe2nodexp --alert-rule-hash "d2aae7bfec5549be1a8b9413215bd86c" --timeout 10 --check-command "check_mariadb_disk_space"
  +    private_tmp               => False
  +    logfile_basedir           => /var/log
  +    user                      => nagios
  +    syslog_identifier         => nrpe2nodexp-mariadb_disk_space
  +    send_mail_to              => root@db2250.codfw.wmnet
  +    success_exit_status       => []
  +    fixed_random_delay        => True
  +    description               => execution of nrpe2nodexp for the check_mariadb_disk_space command.
  +    ensure                    => absent
  +    logfile_group             => root
  +    monitoring_enabled        => False
  +    monitoring_notes_url      => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
  +    syslog_match_startswith   => True
  +    interval                  => [{'start': 'OnUnitInactiveSec', 'interval': '1min'}]
  +    ignore_errors             => True
  +    send_mail_only_on_error   => True
  +    logfile_name              => syslog.log
  +    send_mail                 => False
  +    splay                     => 60
  +    syslog_force_stop         => True
  +    logfile_perms             => all
  +    environment               => {}
  +    logging_enabled           => False
  +    monitoring_contact_groups => admins
  

• Monitoring::Service[mariadb_replica_sql_state_s1]

  Parameters differences:

  --- Monitoring::Service[mariadb_replica_sql_state_s1].orig
  +++ Monitoring::Service[mariadb_replica_sql_state_s1]
  
  +    freshness      => 36000
  +    config_dir     => /etc/nagios
  +    passive        => False
  +    check_command  => nrpe_check!check_mariadb_replica_sql_state_s1!10
  +    host           => db2250
  +    critical       => False
  +    contact_group  => admins
  +    check_interval => 1
  +    description    => MariaDB Replica SQL: s1
  +    notes_url      => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica
  +    retry_interval => 1
  +    ensure         => present
  +    migration_task => T315866
  +    retries        => 3
  

• Exec[systemd daemon-reload for nrpe2nodexp-mysqld.service (nrpe2nodexp-mysqld.service)]

  Parameters differences:

  --- Exec[systemd daemon-reload for nrpe2nodexp-mysqld.service (nrpe2nodexp-mysqld.service)].orig
  +++ Exec[systemd daemon-reload for nrpe2nodexp-mysqld.service (nrpe2nodexp-mysqld.service)]
  
  +    refreshonly => True
  +    command     => /bin/systemctl daemon-reload
  

• File[/var/lib/prometheus/node.d/role_owner.prom]

  Content differences:

  --- /var/lib/prometheus/node.d/role_owner.prom.orig
  +++ /var/lib/prometheus/node.d/role_owner.prom
  @@ -1,3 +1,3 @@
   # HELP role_owner The team owner of the server role
   # TYPE role_owner gauge
  -role_owner{team="data-persistence",role="insetup::data_persistence_ferm",cluster="mysql"} 1.0
  +role_owner{team="data-persistence",role="mariadb::backup_source",cluster="mysql"} 1.0

• User[mysql]

  Parameters differences:

  --- User[mysql].orig
  +++ User[mysql]
  
  +    managehome => False
  +    shell      => /bin/false
  +    ensure     => present
  +    system     => True
  +    home       => /nonexistent
  +    gid        => mysql
  

• Rsyslog::Conf[nrpe2nodexp-mariadb_read_only_s1]

  Parameters differences:

  --- Rsyslog::Conf[nrpe2nodexp-mariadb_read_only_s1].orig
  +++ Rsyslog::Conf[nrpe2nodexp-mariadb_read_only_s1]
  
  +    ensure   => absent
  +    mode     => 0444
  +    priority => 25
  

• Monitoring::Exported_nagios_service[db2250 mariadb_read_only_s1]

  Parameters differences:

  --- Monitoring::Exported_nagios_service[db2250 mariadb_read_only_s1].orig
  +++ Monitoring::Exported_nagios_service[db2250 mariadb_read_only_s1]
  
  +    check_command          => nrpe_check!check_mariadb_read_only_s1!10
  +    notification_period    => 24x7
  +    notification_interval  => 0
  +    check_period           => 24x7
  +    service_description    => MariaDB read only s1
  +    max_check_attempts     => 3
  +    notes_url              => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Master_comes_back_in_read_only
  +    notifications_enabled  => 0
  +    is_volatile            => 0
  +    retry_interval         => 1
  +    ensure                 => present
  +    active_checks_enabled  => 1
  +    notification_options   => c,r,f
  +    contact_groups         => admins
  +    check_interval         => 1
  +    servicegroups          => mysql_codfw
  +    passive_checks_enabled => 1
  +    host_name              => db2250
  +    check_freshness        => 0
  

• Rsyslog::Conf[wmf_auto_restart_prometheus-mysqld-exporter_s1]

  Parameters differences:

  --- Rsyslog::Conf[wmf_auto_restart_prometheus-mysqld-exporter_s1].orig
  +++ Rsyslog::Conf[wmf_auto_restart_prometheus-mysqld-exporter_s1]
  
  +    priority => 40
  +    require  => File[/var/log/wmf_auto_restart_prometheus-mysqld-exporter_s1]
  +    mode     => 0444
  +    ensure   => present
  

• File[/etc/mysql]

  Parameters differences:

  --- File[/etc/mysql].orig
  +++ File[/etc/mysql]
  
  +    group  => root
  +    owner  => root
  +    mode   => 0755
  +    ensure => directory
  

• Rsyslog::Conf[nrpe2nodexp-mysqld]

  Parameters differences:

  --- Rsyslog::Conf[nrpe2nodexp-mysqld].orig
  +++ Rsyslog::Conf[nrpe2nodexp-mysqld]
  
  +    ensure   => absent
  +    mode     => 0444
  +    priority => 25
  

• File[/etc/mysql/my.cnf]

  Parameters differences:

  --- File[/etc/mysql/my.cnf].orig
  +++ File[/etc/mysql/my.cnf]
  
  +    group  => root
  +    owner  => root
  +    ensure => absent
  

• Exec[systemctl try-restart prometheus-mysqld-exporter@s1]

  Parameters differences:

  --- Exec[systemctl try-restart prometheus-mysqld-exporter@s1].orig
  +++ Exec[systemctl try-restart prometheus-mysqld-exporter@s1]
  
  +    refreshonly => True
  +    path        => /usr/bin
  

• Systemd::Timer[nrpe2nodexp-mariadb_replica_sql_lag_s1]

  Parameters differences:

  --- Systemd::Timer[nrpe2nodexp-mariadb_replica_sql_lag_s1].orig
  +++ Systemd::Timer[nrpe2nodexp-mariadb_replica_sql_lag_s1]
  
  +    fixed_random_delay => True
  +    splay              => 60
  +    unit_name          => nrpe2nodexp-mariadb_replica_sql_lag_s1.service
  +    accuracy           => 15sec
  +    ensure             => absent
  +    timer_intervals    => [{'start': 'OnUnitInactiveSec', 'interval': '1min'}, {'interval': '1s', 'start': 'OnActiveSec'}]
  

• Prometheus::Alert::Rule[check_mariadb_replica_sql_lag_s1_8ec9d6c1155ae60699aaf9dfd1fa339b]

  Parameters differences:

  --- Prometheus::Alert::Rule[check_mariadb_replica_sql_lag_s1_8ec9d6c1155ae60699aaf9dfd1fa339b].orig
  +++ Prometheus::Alert::Rule[check_mariadb_replica_sql_lag_s1_8ec9d6c1155ae60699aaf9dfd1fa339b]
  
  +    group              => nrpechecks
  +    team               => observability
  +    for                => 10m
  +    site               => codfw
  +    severity           => info
  +    dashboard          => TODO
  +    summary            => NRPE CHECK: MariaDB Replica Lag: s1
  +    logs               => https://logstash.wikimedia.org/app/dashboards#/view/2d343ac0-6df8-11f0-8e08-7fab0da52b33?_g=(filters:!((query:(match_phrase:(event.module:check_mariadb_replica_sql_lag_s1))),(query:(match_phrase:(host.name:{{$labels.instance|stripPort}})))))
  +    instance           => ops
  +    alert_name         => nrpe_MariaDB_Replica_Lag__s1
  +    description        => NRPE CHECK: MariaDB Replica Lag: s1
  +    runbook            => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica
  +    ensure             => absent
  +    def_label_whitelst => ['team', 'severity']
  +    expr               => (nagios_nrpe_check_result{alert_rule_hash="8ec9d6c1155ae60699aaf9dfd1fa339b",check_name="check_mariadb_replica_sql_lag_s1", status=~"(WARNING|CRITICAL)", severity=~"(warning|critical)"} > 0) * on (instance) group_left (team) role_owner
  

• Systemd::Timer[nrpe2nodexp-mariadb_replica_sql_state_s1]

  Parameters differences:

  --- Systemd::Timer[nrpe2nodexp-mariadb_replica_sql_state_s1].orig
  +++ Systemd::Timer[nrpe2nodexp-mariadb_replica_sql_state_s1]
  
  +    fixed_random_delay => True
  +    splay              => 60
  +    unit_name          => nrpe2nodexp-mariadb_replica_sql_state_s1.service
  +    accuracy           => 15sec
  +    ensure             => absent
  +    timer_intervals    => [{'start': 'OnUnitInactiveSec', 'interval': '1min'}, {'interval': '1s', 'start': 'OnActiveSec'}]
  

• Service[nrpe2nodexp-mysqld.timer]

  Parameters differences:

  --- Service[nrpe2nodexp-mysqld.timer].orig
  +++ Service[nrpe2nodexp-mysqld.timer]
  
  +    before   => ['Exec[systemd daemon-reload for nrpe2nodexp-mysqld.timer (nrpe2nodexp-mysqld.timer)]']
  +    provider => systemd
  +    enable   => False
  +    ensure   => stopped
  

• File[/usr/lib/nagios/plugins/check_mariadb.pl]

  Parameters differences:

  --- File[/usr/lib/nagios/plugins/check_mariadb.pl].orig
  +++ File[/usr/lib/nagios/plugins/check_mariadb.pl]
  
  +    group  => root
  +    owner  => root
  +    ensure => absent
  

• File[/var/log/wmf_auto_restart_prometheus-mysqld-exporter_s1]

  Parameters differences:

  --- File[/var/log/wmf_auto_restart_prometheus-mysqld-exporter_s1].orig
  +++ File[/var/log/wmf_auto_restart_prometheus-mysqld-exporter_s1]
  
  +    group  => root
  +    owner  => root
  +    force  => True
  +    ensure => directory
  +    backup => False
  +    mode   => 0755
  

• Sudo::User[nrpe-check_mariadb_replica_sql_lag_s1]

  Parameters differences:

  --- Sudo::User[nrpe-check_mariadb_replica_sql_lag_s1].orig
  +++ Sudo::User[nrpe-check_mariadb_replica_sql_lag_s1]
  
  +    privileges => []
  +    require    => ['Class[Sudo]']
  +    ensure     => absent
  +    tag        => nrpe::check
  +    user       => nagios
  

• Prometheus::Mysqld_exporter::Instance[s1]

  Parameters differences:

  --- Prometheus::Mysqld_exporter::Instance[s1].orig
  +++ Prometheus::Mysqld_exporter::Instance[s1]
  
  +    client_password => This is a fake passsword, but cannot be empty due to Debian #953040
  +    listen_address  => :13311
  +    client_socket   => /run/mysqld/mysqld.s1.sock
  +    client_user     => prometheus
  

• File[/usr/local/etc/my.cnf]

  Parameters differences:

  --- File[/usr/local/etc/my.cnf].orig
  +++ File[/usr/local/etc/my.cnf]
  
  +    group  => root
  +    owner  => root
  +    ensure => absent
  

• File[/etc/logrotate.d/wmf_auto_restart_prometheus-mysqld-exporter_s1]

  Parameters differences:

  --- File[/etc/logrotate.d/wmf_auto_restart_prometheus-mysqld-exporter_s1].orig
  +++ File[/etc/logrotate.d/wmf_auto_restart_prometheus-mysqld-exporter_s1]
  
  +    group  => root
  +    owner  => root
  +    mode   => 0444
  +    ensure => present
  

  Content differences:

  --- /etc/logrotate.d/wmf_auto_restart_prometheus-mysqld-exporter_s1.orig
  +++ /etc/logrotate.d/wmf_auto_restart_prometheus-mysqld-exporter_s1
  @@ -0,0 +1,12 @@
  +# logrotate(8) config for wmf_auto_restart_prometheus-mysqld-exporter_s1
  +
  +/var/log/wmf_auto_restart_prometheus-mysqld-exporter_s1/*.log {
  +    daily
  +    copytruncate
  +    missingok
  +    compress
  +    delaycompress
  +    notifempty
  +    rotate 15
  +    size 256M
  +}

• Motd::Script[db-section-s1]

  Parameters differences:

  --- Motd::Script[db-section-s1].orig
  +++ Motd::Script[db-section-s1]
  
  +    priority => 6
  +    ensure   => present
  

• File[/lib/systemd/system/nrpe2nodexp-mariadb_disk_space.service]

  Parameters differences:

  --- File[/lib/systemd/system/nrpe2nodexp-mariadb_disk_space.service].orig
  +++ File[/lib/systemd/system/nrpe2nodexp-mariadb_disk_space.service]
  
  +    group  => root
  +    notify => Exec[systemd daemon-reload for nrpe2nodexp-mariadb_disk_space.service (nrpe2nodexp-mariadb_disk_space.service)]
  +    owner  => root
  +    ensure => absent
  +    mode   => 0444
  

  Content differences:

  --- /lib/systemd/system/nrpe2nodexp-mariadb_disk_space.service.orig
  +++ /lib/systemd/system/nrpe2nodexp-mariadb_disk_space.service
  @@ -0,0 +1,11 @@
  +[Unit]
  +Description=execution of nrpe2nodexp for the check_mariadb_disk_space command.
  +Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state
  +
  +[Service]
  +Type=oneshot
  +User=nagios
  +
  +Group=prometheus-node-exporter
  +SyslogIdentifier=nrpe2nodexp-mariadb_disk_space
  +ExecStart=-/usr/local/bin/nrpe2nodexp --alert-rule-hash "d2aae7bfec5549be1a8b9413215bd86c" --timeout 10 --check-command "check_mariadb_disk_space"

• Class[Mariadb::Monitor_memory]

  Parameters differences:

  --- Class[Mariadb::Monitor_memory].orig
  +++ Class[Mariadb::Monitor_memory]
  
  +    warning       => 90
  +    is_critical   => False
  +    critical      => 95
  +    contact_group => admins
  

• File[/etc/mysql/ssl/server-keypair.pem]

  Parameters differences:

  --- File[/etc/mysql/ssl/server-keypair.pem].orig
  +++ File[/etc/mysql/ssl/server-keypair.pem]
  
  +    group  => mysql
  +    owner  => mysql
  +    mode   => 0400
  +    ensure => absent
  

• Sslcert::X509_to_pkcs12[puppet::expose_agent_cert: /etc/mysql]

  Parameters differences:

  --- Sslcert::X509_to_pkcs12[puppet::expose_agent_cert: /etc/mysql].orig
  +++ Sslcert::X509_to_pkcs12[puppet::expose_agent_cert: /etc/mysql]
  
  +    group       => mysql
  +    private_key => /var/lib/puppet/ssl/private_keys/db2250.codfw.wmnet.pem
  +    owner       => mysql
  +    certfile    => /var/lib/puppet/ssl/certs/ca.pem
  +    public_key  => /var/lib/puppet/ssl/certs/db2250.codfw.wmnet.pem
  +    ensure      => absent
  +    outfile     => /etc/mysql/ssl/server.p12
  

• File[/var/lib/prometheus/node.d/check_mariadb_replica_sql_lag_s1.prom]

  Parameters differences:

  --- File[/var/lib/prometheus/node.d/check_mariadb_replica_sql_lag_s1.prom].orig
  +++ File[/var/lib/prometheus/node.d/check_mariadb_replica_sql_lag_s1.prom]
  
  +    group  => root
  +    owner  => root
  +    ensure => absent
  

• Package[monitoring-plugins-contrib]

  Parameters differences:

  --- Package[monitoring-plugins-contrib].orig
  +++ Package[monitoring-plugins-contrib]
  
  +    provider => apt
  +    ensure   => installed
  

• File[/etc/default/prometheus]

  Parameters differences:

  --- File[/etc/default/prometheus].orig
  +++ File[/etc/default/prometheus]
  
  +    group  => prometheus
  +    owner  => prometheus
  +    mode   => 0550
  +    ensure => directory
  

• Motd::Script[insetup::data_persistence_ferm]

  Parameters differences:

  --- Motd::Script[insetup::data_persistence_ferm].orig
  +++ Motd::Script[insetup::data_persistence_ferm]
  
  -    ensure   => present
  -    priority => 5
  

• File[/lib/systemd/system/wmf_auto_restart_prometheus-mysqld-exporter@s1.timer]

  Parameters differences:

  --- File[/lib/systemd/system/wmf_auto_restart_prometheus-mysqld-exporter@s1.timer].orig
  +++ File[/lib/systemd/system/wmf_auto_restart_prometheus-mysqld-exporter@s1.timer]
  
  +    group  => root
  +    notify => Exec[systemd daemon-reload for wmf_auto_restart_prometheus-mysqld-exporter@s1.timer (wmf_auto_restart_prometheus-mysqld-exporter@s1.timer)]
  +    owner  => root
  +    ensure => present
  +    mode   => 0444
  

  Content differences:

  --- /lib/systemd/system/wmf_auto_restart_prometheus-mysqld-exporter@s1.timer.orig
  +++ /lib/systemd/system/wmf_auto_restart_prometheus-mysqld-exporter@s1.timer
  @@ -0,0 +1,12 @@
  +[Unit]
  +Description=Periodic execution of wmf_auto_restart_prometheus-mysqld-exporter@s1.service
  +
  +[Timer]
  +Unit=wmf_auto_restart_prometheus-mysqld-exporter@s1.service
  +# Accuracy sets the maximum time interval around the execution time we want to allow
  +AccuracySec=15sec
  +OnCalendar=Mon,Tue,Wed,Thu,Fri *-*-* 0:55:00
  +RandomizedDelaySec=0
  +
  +[Install]
  +WantedBy=multi-user.target

• Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_state_s1.service (nrpe2nodexp-mariadb_replica_sql_state_s1.service)]

  Parameters differences:

  --- Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_state_s1.service (nrpe2nodexp-mariadb_replica_sql_state_s1.service)].orig
  +++ Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_state_s1.service (nrpe2nodexp-mariadb_replica_sql_state_s1.service)]
  
  +    refreshonly => True
  +    command     => /bin/systemctl daemon-reload
  

• Prometheus::Alert::Rule[check_mariadb_read_only_s1_42baf529e0f3bfca391aca0a24c880e9]

  Parameters differences:

  --- Prometheus::Alert::Rule[check_mariadb_read_only_s1_42baf529e0f3bfca391aca0a24c880e9].orig
  +++ Prometheus::Alert::Rule[check_mariadb_read_only_s1_42baf529e0f3bfca391aca0a24c880e9]
  
  +    group              => nrpechecks
  +    team               => observability
  +    for                => 3m
  +    site               => codfw
  +    severity           => info
  +    dashboard          => TODO
  +    summary            => NRPE CHECK: MariaDB read only s1
  +    logs               => https://logstash.wikimedia.org/app/dashboards#/view/2d343ac0-6df8-11f0-8e08-7fab0da52b33?_g=(filters:!((query:(match_phrase:(event.module:check_mariadb_read_only_s1))),(query:(match_phrase:(host.name:{{$labels.instance|stripPort}})))))
  +    instance           => ops
  +    alert_name         => nrpe_MariaDB_read_only_s1
  +    description        => NRPE CHECK: MariaDB read only s1
  +    runbook            => https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Master_comes_back_in_read_only
  +    ensure             => absent
  +    def_label_whitelst => ['team', 'severity']
  +    expr               => (nagios_nrpe_check_result{alert_rule_hash="42baf529e0f3bfca391aca0a24c880e9",check_name="check_mariadb_read_only_s1", status=~"(WARNING|CRITICAL)", severity=~"(warning|critical)"} > 0) * on (instance) group_left (team) role_owner
  

• Systemd::Service[nrpe2nodexp-mariadb_replica_io_state_s1]

  Parameters differences:

  --- Systemd::Service[nrpe2nodexp-mariadb_replica_io_state_s1].orig
  +++ Systemd::Service[nrpe2nodexp-mariadb_replica_io_state_s1]
  
  +    monitoring_enabled       => False
  +    require                  => Systemd::Unit[nrpe2nodexp-mariadb_replica_io_state_s1.service]
  +    restart                  => False
  +    unit_type                => timer
  +    service_params           => {}
  +    monitoring_contact_group => admins
  +    monitoring_critical      => False
  +    migration_task           => T407130
  +    ensure                   => absent
  +    override                 => False
  

• File[/etc/wmfmariadbpy]

  Parameters differences:

  --- File[/etc/wmfmariadbpy].orig
  +++ File[/etc/wmfmariadbpy]
  
  +    group  => root
  +    owner  => root
  +    mode   => 0755
  +    ensure => directory
  

• Systemd::Timer[nrpe2nodexp-mariadb_read_only_s1]

  Parameters differences:

  --- Systemd::Timer[nrpe2nodexp-mariadb_read_only_s1].orig
  +++ Systemd::Timer[nrpe2nodexp-mariadb_read_only_s1]
  
  +    fixed_random_delay => True
  +    splay              => 60
  +    unit_name          => nrpe2nodexp-mariadb_read_only_s1.service
  +    accuracy           => 15sec
  +    ensure             => absent
  +    timer_intervals    => [{'start': 'OnUnitInactiveSec', 'interval': '1min'}, {'interval': '1s', 'start': 'OnActiveSec'}]
  

• Concat[/etc/mysql/ssl/cert.pem]

  Parameters differences:

  --- Concat[/etc/mysql/ssl/cert.pem].orig
  +++ Concat[/etc/mysql/ssl/cert.pem]
  
  +    replace        => True
  +    show_diff      => True
  +    order          => alpha
  +    path           => /etc/mysql/ssl/cert.pem
  +    warn           => False
  +    force          => False
  +    ensure_newline => False
  +    ensure         => present
  +    backup         => puppet
  +    format         => plain
  +    mode           => 0644
  

• Systemd::Service[nrpe2nodexp-mariadb_replica_sql_lag_s1]

  Parameters differences:

  --- Systemd::Service[nrpe2nodexp-mariadb_replica_sql_lag_s1].orig
  +++ Systemd::Service[nrpe2nodexp-mariadb_replica_sql_lag_s1]
  
  +    monitoring_enabled       => False
  +    require                  => Systemd::Unit[nrpe2nodexp-mariadb_replica_sql_lag_s1.service]
  +    restart                  => False
  +    unit_type                => timer
  +    service_params           => {}
  +    monitoring_contact_group => admins
  +    monitoring_critical      => False
  +    migration_task           => T407130
  +    ensure                   => absent
  +    override                 => False
  

• Service[nrpe2nodexp-mariadb_replica_sql_state_s1.timer]

  Parameters differences:

  --- Service[nrpe2nodexp-mariadb_replica_sql_state_s1.timer].orig
  +++ Service[nrpe2nodexp-mariadb_replica_sql_state_s1.timer]
  
  +    before   => ['Exec[systemd daemon-reload for nrpe2nodexp-mariadb_replica_sql_state_s1.timer (nrpe2nodexp-mariadb_replica_sql_state_s1.timer)]']
  +    provider => systemd
  +    enable   => False
  +    ensure   => stopped
  

• Nrpe::Check[check_mariadb_disk_space]

  Parameters differences:

  --- Nrpe::Check[check_mariadb_disk_space].orig
  +++ Nrpe::Check[check_mariadb_disk_space]
  
  +    before  => Monitoring::Service[mariadb_disk_space]
  +    command => /usr/lib/nagios/plugins/check_disk -w 10% -c 5% -l -e --exclude-type=tracefs --exclude-type=tmpfs --exclude-type=ramfs --exclude-type=devtmpfs
  +    ensure  => present
  

• Service[nrpe2nodexp-mariadb_memory.timer]

  Parameters differences:

  --- Service[nrpe2nodexp-mariadb_memory.timer].orig
  +++ Service[nrpe2nodexp-mariadb_memory.timer]
  
  +    before   => ['Exec[systemd daemon-reload for nrpe2nodexp-mariadb_memory.timer (nrpe2nodexp-mariadb_memory.timer)]']
  +    provider => systemd
  +    enable   => False
  +    ensure   => stopped
  

• Profile::Mariadb::Ferm[s1]

  Parameters differences:

  --- Profile::Mariadb::Ferm[s1].orig
  +++ Profile::Mariadb::Ferm[s1]
  
  +    port => 3311
  

• Package[wmfmariadbpy-common]

  Parameters differences:

  --- Package[wmfmariadbpy-common].orig
  +++ Package[wmfmariadbpy-common]
  
  +    provider => apt
  +    ensure   => installed
  

• Service[wmf_auto_restart_prometheus-mysqld-exporter@s1.timer]

  Parameters differences:

  --- Service[wmf_auto_restart_prometheus-mysqld-exporter@s1.timer].orig
  +++ Service[wmf_auto_restart_prometheus-mysqld-exporter@s1.timer]
  
  +    enable   => True
  +    provider => systemd
  +    ensure   => running
  

Relevant files

• Production catalog
• Change catalog
• Production errors/warnings
• Change errors/warnings
• Core Diff
• Standard Diff