Compilation results for an-launcher1003.eqiad.wmnet: System changes detected
You can retrieve this result from host.json.Catalog differences
Summary
| Total Resources: | 4104 |
|---|---|
| Resources added: | 36 |
| Resources removed: | 0 |
| Resources modified: | 49 |
| Change percentage: | 2.07% |
Resources only in the new catalog
- Kerberos::Systemd_timer[refinery-sqoop-mediawiki-production]
- File[/lib/systemd/system/refinery-sqoop-mediawiki-production.service]
- Systemd::Unit[refinery-sqoop-mediawiki-clouddb.service]
- File[/usr/local/bin/refinery-sqoop-mediawiki-production]
- Systemd::Service[refinery-sqoop-mediawiki-centralauth-production]
- Systemd::Timer::Job[refinery-sqoop-mediawiki-clouddb]
- File[/usr/local/bin/refinery-sqoop-mediawiki-clouddb]
- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-production.timer (refinery-sqoop-mediawiki-production.timer)]
- Systemd::Timer::Job[refinery-sqoop-mediawiki-production]
- Service[refinery-sqoop-mediawiki-clouddb.timer]
- Service[refinery-sqoop-mediawiki-centralauth-production.timer]
- File[/lib/systemd/system/refinery-sqoop-mediawiki-production.timer]
- Systemd::Unit[refinery-sqoop-mediawiki-centralauth-production.service]
- Systemd::Timer[refinery-sqoop-mediawiki-centralauth-production]
- File[/usr/local/bin/refinery-sqoop-mediawiki-centralauth-production]
- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-production.service (refinery-sqoop-mediawiki-production.service)]
- Systemd::Timer::Job[refinery-sqoop-mediawiki-centralauth-production]
- Kerberos::Systemd_timer[refinery-sqoop-mediawiki-centralauth-production]
- Kerberos::Systemd_timer[refinery-sqoop-mediawiki-clouddb]
- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-centralauth-production.service (refinery-sqoop-mediawiki-centralauth-production.service)]
- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-centralauth-production.timer (refinery-sqoop-mediawiki-centralauth-production.timer)]
- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-clouddb.service (refinery-sqoop-mediawiki-clouddb.service)]
- Systemd::Unit[refinery-sqoop-mediawiki-centralauth-production.timer]
- File[/lib/systemd/system/refinery-sqoop-mediawiki-centralauth-production.timer]
- Systemd::Unit[refinery-sqoop-mediawiki-clouddb.timer]
- Systemd::Service[refinery-sqoop-mediawiki-clouddb]
- Systemd::Timer[refinery-sqoop-mediawiki-clouddb]
- Systemd::Service[refinery-sqoop-mediawiki-production]
- Systemd::Timer[refinery-sqoop-mediawiki-production]
- File[/lib/systemd/system/refinery-sqoop-mediawiki-clouddb.timer]
- File[/lib/systemd/system/refinery-sqoop-mediawiki-centralauth-production.service]
- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-clouddb.timer (refinery-sqoop-mediawiki-clouddb.timer)]
- Service[refinery-sqoop-mediawiki-production.timer]
- Systemd::Unit[refinery-sqoop-mediawiki-production.timer]
- File[/lib/systemd/system/refinery-sqoop-mediawiki-clouddb.service]
- Systemd::Unit[refinery-sqoop-mediawiki-production.service]
Resources modified
- File[/lib/systemd/system/refinery-sqoop-whole-mediawiki.service]
- Parameters differences:
--- File[/lib/systemd/system/refinery-sqoop-whole-mediawiki.service].orig +++ File[/lib/systemd/system/refinery-sqoop-whole-mediawiki.service] @@ - ensure => present + ensure => absent
- Systemd::Unit[refinery-sqoop-mediawiki-production.timer]
- Parameters differences:
--- Systemd::Unit[refinery-sqoop-mediawiki-production.timer].orig +++ Systemd::Unit[refinery-sqoop-mediawiki-production.timer] + ensure => present + override_filename => puppet-override.conf + unit => refinery-sqoop-mediawiki-production.timer + restart => False + require => ['Class[Systemd]'] + override => False
- Systemd::Unit[refinery-sqoop-mediawiki-production.service]
- Parameters differences:
--- Systemd::Unit[refinery-sqoop-mediawiki-production.service].orig +++ Systemd::Unit[refinery-sqoop-mediawiki-production.service] + ensure => present + override_filename => puppet-override.conf + unit => refinery-sqoop-mediawiki-production.service + restart => False + require => ['Class[Systemd]'] + override => False
- Systemd::Unit[refinery-sqoop-mediawiki-clouddb.service]
- Parameters differences:
--- Systemd::Unit[refinery-sqoop-mediawiki-clouddb.service].orig +++ Systemd::Unit[refinery-sqoop-mediawiki-clouddb.service] + ensure => present + override_filename => puppet-override.conf + unit => refinery-sqoop-mediawiki-clouddb.service + restart => False + require => ['Class[Systemd]'] + override => False
- File[/lib/systemd/system/refinery-sqoop-whole-mediawiki.timer]
- Parameters differences:
--- File[/lib/systemd/system/refinery-sqoop-whole-mediawiki.timer].orig +++ File[/lib/systemd/system/refinery-sqoop-whole-mediawiki.timer] @@ - ensure => present + ensure => absent
- Kerberos::Systemd_timer[refinery-sqoop-mediawiki-centralauth-production]
- Parameters differences:
--- Kerberos::Systemd_timer[refinery-sqoop-mediawiki-centralauth-production].orig +++ Kerberos::Systemd_timer[refinery-sqoop-mediawiki-centralauth-production] + send_mail => True + require => ['File[/usr/local/bin/refinery-sqoop-mediawiki-centralauth-production]', 'File[/tmp/sqoop-jars]', 'Class[Kerberos::Wrapper]'] + syslog_force_stop => True + description => Schedules sqoop to import centralauth MediaWiki databases into Hadoop monthly. + logfile_basedir => /var/log/ + syslog_match_startswith => True + ensure => present + logfile_group => analytics + logfile_perms => all + command => /usr/local/bin/refinery-sqoop-mediawiki-centralauth-production + user => analytics + logfile_owner => analytics + interval => *-*-01 00:00:00 + environment => {} + send_mail_to => data-engineering-alerts@wikimedia.org- File[/lib/systemd/system/refinery-sqoop-mediawiki-centralauth-production.service]
- Parameters differences:
--- File[/lib/systemd/system/refinery-sqoop-mediawiki-centralauth-production.service].orig +++ File[/lib/systemd/system/refinery-sqoop-mediawiki-centralauth-production.service] + notify => Exec[systemd daemon-reload for refinery-sqoop-mediawiki-centralauth-production.service (refinery-sqoop-mediawiki-centralauth-production.service)] + ensure => present + group => root + mode => 0444 + owner => root
- Content differences:
--- /lib/systemd/system/refinery-sqoop-mediawiki-centralauth-production.service.orig +++ /lib/systemd/system/refinery-sqoop-mediawiki-centralauth-production.service @@ -0,0 +1,9 @@ +[Unit] +Description=Schedules sqoop to import centralauth MediaWiki databases into Hadoop monthly. +Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state + +[Service] +Type=oneshot +User=analytics +SyslogIdentifier=kerberos-run-command +ExecStart=/usr/local/bin/systemd-timer-mail-wrapper --subject refinery-sqoop-mediawiki-centralauth-production --mail-to data-engineering-alerts@wikimedia.org --only-on-error /usr/local/bin/kerberos-run-command analytics /usr/local/bin/refinery-sqoop-mediawiki-centralauth-production
- Service[refinery-sqoop-mediawiki-production.timer]
- Parameters differences:
--- Service[refinery-sqoop-mediawiki-production.timer].orig +++ Service[refinery-sqoop-mediawiki-production.timer] + ensure => running + provider => systemd + enable => True
- Service[refinery-sqoop-mediawiki-clouddb.timer]
- Parameters differences:
--- Service[refinery-sqoop-mediawiki-clouddb.timer].orig +++ Service[refinery-sqoop-mediawiki-clouddb.timer] + ensure => running + provider => systemd + enable => True
- File[/usr/local/bin/refinery-sqoop-mediawiki-production-history]
- Content differences:
--- /usr/local/bin/refinery-sqoop-mediawiki-production-history.orig +++ /usr/local/bin/refinery-sqoop-mediawiki-production-history @@ -8,7 +8,7 @@ --job-name sqoop-mediawiki-monthly-production-history-$(/bin/date --date="$(/bin/date +%Y-%m-15) -1 month" +'%Y-%m') \ --output-dir /wmf/data/raw/mediawiki_private/tables \ --wiki-file /mnt/hdfs/wmf/data/wmf/mediawiki/database/grouped_wikis.csv \ - --tables actor,comment \ + --tables actor,comment,content \ --user getup \ --password-file /user/analytics/mysql-analytics-research-client-pw.txt \ --partition-name snapshot \
- Systemd::Unit[refinery-sqoop-whole-mediawiki.timer]
- Parameters differences:
--- Systemd::Unit[refinery-sqoop-whole-mediawiki.timer].orig +++ Systemd::Unit[refinery-sqoop-whole-mediawiki.timer] @@ - ensure => present + ensure => absent
- File[/usr/local/bin/refinery-sqoop-mediawiki-centralauth-production]
- Parameters differences:
--- File[/usr/local/bin/refinery-sqoop-mediawiki-centralauth-production].orig +++ File[/usr/local/bin/refinery-sqoop-mediawiki-centralauth-production] + ensure => present + group => analytics + require => File[/usr/local/bin/refinery-sqoop-centralauth-production] + mode => 0550 + owner => analytics
- Content differences:
--- /usr/local/bin/refinery-sqoop-mediawiki-centralauth-production.orig +++ /usr/local/bin/refinery-sqoop-mediawiki-centralauth-production @@ -0,0 +1,10 @@ +#!/bin/bash +# NOTE: This file is managed by puppet +# +# This file is a wrapper running sqoop for whole mediawiki tables. +# It runs one sqoops: +# * sqoops the necessary tables we care about from centralauth production replicas +# +# See each script referenced below for the list of centralauth production replica tables + +/usr/local/bin/refinery-sqoop-centralauth-production
- Kerberos::Systemd_timer[refinery-sqoop-whole-mediawiki]
- Parameters differences:
--- Kerberos::Systemd_timer[refinery-sqoop-whole-mediawiki].orig +++ Kerberos::Systemd_timer[refinery-sqoop-whole-mediawiki] @@ - ensure => present + ensure => absent
- File[/usr/local/bin/refinery-sqoop-mediawiki-history]
- Content differences:
--- /usr/local/bin/refinery-sqoop-mediawiki-history.orig +++ /usr/local/bin/refinery-sqoop-mediawiki-history @@ -9,7 +9,7 @@ --clouddb \ --output-dir /wmf/data/raw/mediawiki/tables \ --wiki-file /mnt/hdfs/wmf/data/wmf/mediawiki/database/grouped_wikis.csv \ - --tables archive,change_tag,change_tag_def,content,logging,page,revision,slots,slot_roles,user,user_groups \ + --tables archive,change_tag,change_tag_def,logging,page,revision,slots,slot_roles,user,user_groups \ --user ann \ --password-file /user/analytics/mysql-analytics-labsdb-client-pw.txt \ --partition-name snapshot \
- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-clouddb.service (refinery-sqoop-mediawiki-clouddb.service)]
- Parameters differences:
--- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-clouddb.service (refinery-sqoop-mediawiki-clouddb.service)].orig +++ Exec[systemd daemon-reload for refinery-sqoop-mediawiki-clouddb.service (refinery-sqoop-mediawiki-clouddb.service)] + refreshonly => True + command => /bin/systemctl daemon-reload
- Systemd::Service[refinery-sqoop-mediawiki-clouddb]
- Parameters differences:
--- Systemd::Service[refinery-sqoop-mediawiki-clouddb].orig +++ Systemd::Service[refinery-sqoop-mediawiki-clouddb] + service_params => {} + unit_type => timer + require => Systemd::Unit[refinery-sqoop-mediawiki-clouddb.service] + restart => False + monitoring_critical => False + override => False + monitoring_contact_group => admins + ensure => present + migration_task => T407130 + monitoring_enabled => False- Systemd::Timer[refinery-sqoop-mediawiki-clouddb]
- Parameters differences:
--- Systemd::Timer[refinery-sqoop-mediawiki-clouddb].orig +++ Systemd::Timer[refinery-sqoop-mediawiki-clouddb] + ensure => present + fixed_random_delay => False + splay => 0 + timer_intervals => [{'start': 'OnCalendar', 'interval': '*-*-01 00:00:00'}] + unit_name => refinery-sqoop-mediawiki-clouddb.service + accuracy => 15sec- File[/lib/systemd/system/refinery-sqoop-mediawiki-clouddb.timer]
- Parameters differences:
--- File[/lib/systemd/system/refinery-sqoop-mediawiki-clouddb.timer].orig +++ File[/lib/systemd/system/refinery-sqoop-mediawiki-clouddb.timer] + notify => Exec[systemd daemon-reload for refinery-sqoop-mediawiki-clouddb.timer (refinery-sqoop-mediawiki-clouddb.timer)] + ensure => present + group => root + mode => 0444 + owner => root
- Content differences:
--- /lib/systemd/system/refinery-sqoop-mediawiki-clouddb.timer.orig +++ /lib/systemd/system/refinery-sqoop-mediawiki-clouddb.timer @@ -0,0 +1,12 @@ +[Unit] +Description=Periodic execution of refinery-sqoop-mediawiki-clouddb.service + +[Timer] +Unit=refinery-sqoop-mediawiki-clouddb.service +# Accuracy sets the maximum time interval around the execution time we want to allow +AccuracySec=15sec +OnCalendar=*-*-01 00:00:00 +RandomizedDelaySec=0 + +[Install] +WantedBy=multi-user.target
- Systemd::Unit[refinery-sqoop-mediawiki-centralauth-production.service]
- Parameters differences:
--- Systemd::Unit[refinery-sqoop-mediawiki-centralauth-production.service].orig +++ Systemd::Unit[refinery-sqoop-mediawiki-centralauth-production.service] + ensure => present + override_filename => puppet-override.conf + unit => refinery-sqoop-mediawiki-centralauth-production.service + restart => False + require => ['Class[Systemd]'] + override => False
- Systemd::Unit[refinery-sqoop-whole-mediawiki.service]
- Parameters differences:
--- Systemd::Unit[refinery-sqoop-whole-mediawiki.service].orig +++ Systemd::Unit[refinery-sqoop-whole-mediawiki.service] @@ - ensure => present + ensure => absent
- Systemd::Service[refinery-sqoop-mediawiki-production]
- Parameters differences:
--- Systemd::Service[refinery-sqoop-mediawiki-production].orig +++ Systemd::Service[refinery-sqoop-mediawiki-production] + service_params => {} + unit_type => timer + require => Systemd::Unit[refinery-sqoop-mediawiki-production.service] + restart => False + monitoring_critical => False + override => False + monitoring_contact_group => admins + ensure => present + migration_task => T407130 + monitoring_enabled => False- File[/usr/local/bin/refinery-sqoop-whole-mediawiki]
- Parameters differences:
--- File[/usr/local/bin/refinery-sqoop-whole-mediawiki].orig +++ File[/usr/local/bin/refinery-sqoop-whole-mediawiki] @@ - ensure => present + ensure => absent
- Systemd::Unit[refinery-sqoop-mediawiki-clouddb.timer]
- Parameters differences:
--- Systemd::Unit[refinery-sqoop-mediawiki-clouddb.timer].orig +++ Systemd::Unit[refinery-sqoop-mediawiki-clouddb.timer] + ensure => present + override_filename => puppet-override.conf + unit => refinery-sqoop-mediawiki-clouddb.timer + restart => False + require => ['Class[Systemd]'] + override => False
- File[/lib/systemd/system/refinery-sqoop-mediawiki-production.service]
- Parameters differences:
--- File[/lib/systemd/system/refinery-sqoop-mediawiki-production.service].orig +++ File[/lib/systemd/system/refinery-sqoop-mediawiki-production.service] + notify => Exec[systemd daemon-reload for refinery-sqoop-mediawiki-production.service (refinery-sqoop-mediawiki-production.service)] + ensure => present + group => root + mode => 0444 + owner => root
- Content differences:
--- /lib/systemd/system/refinery-sqoop-mediawiki-production.service.orig +++ /lib/systemd/system/refinery-sqoop-mediawiki-production.service @@ -0,0 +1,9 @@ +[Unit] +Description=Schedules sqoop to import production MediaWiki databases into Hadoop monthly. +Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state + +[Service] +Type=oneshot +User=analytics +SyslogIdentifier=kerberos-run-command +ExecStart=/usr/local/bin/systemd-timer-mail-wrapper --subject refinery-sqoop-mediawiki-production --mail-to data-engineering-alerts@wikimedia.org --only-on-error /usr/local/bin/kerberos-run-command analytics /usr/local/bin/refinery-sqoop-mediawiki-production
- Systemd::Timer::Job[refinery-sqoop-mediawiki-production]
- Parameters differences:
--- Systemd::Timer::Job[refinery-sqoop-mediawiki-production].orig +++ Systemd::Timer::Job[refinery-sqoop-mediawiki-production] + send_mail => True + syslog_force_stop => True + ignore_errors => False + logfile_group => analytics + monitoring_notes_url => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state + logfile_perms => all + send_mail_only_on_error => True + command => /usr/local/bin/kerberos-run-command analytics /usr/local/bin/refinery-sqoop-mediawiki-production + monitoring_enabled => False + user => analytics + logfile_name => syslog.log + syslog_match_startswith => True + logging_enabled => False + interval => {'start': 'OnCalendar', 'interval': '*-*-01 00:00:00'} + fixed_random_delay => False + description => Schedules sqoop to import production MediaWiki databases into Hadoop monthly. + logfile_basedir => /var/log/ + private_tmp => False + ensure => present + monitoring_contact_groups => admins + success_exit_status => [] + logfile_owner => analytics + environment => {} + send_mail_to => data-engineering-alerts@wikimedia.org- File[/usr/local/bin/refinery-sqoop-mediawiki-production]
- Parameters differences:
--- File[/usr/local/bin/refinery-sqoop-mediawiki-production].orig +++ File[/usr/local/bin/refinery-sqoop-mediawiki-production] + ensure => present + group => analytics + require => ['File[/usr/local/bin/refinery-sqoop-mediawiki-production-history]', 'File[/usr/local/bin/refinery-sqoop-mediawiki-production-not-history]'] + mode => 0550 + owner => analytics
- Content differences:
--- /usr/local/bin/refinery-sqoop-mediawiki-production.orig +++ /usr/local/bin/refinery-sqoop-mediawiki-production @@ -0,0 +1,13 @@ +#!/bin/bash +# NOTE: This file is managed by puppet +# +# This file is a wrapper running sqoop for whole mediawiki tables. +# It runs two sqoops in series: +# * first sqoops tables necessary for the mediawiki history dataset from production replicas +# * and finally, sqoops the remaining tables we care about from production replicas +# +# See each script referenced below for the list of prod replica tables + +/usr/local/bin/refinery-sqoop-mediawiki-production-history \ + && /usr/local/bin/refinery-sqoop-mediawiki-production-not-history +
- Systemd::Service[refinery-sqoop-mediawiki-centralauth-production]
- Parameters differences:
--- Systemd::Service[refinery-sqoop-mediawiki-centralauth-production].orig +++ Systemd::Service[refinery-sqoop-mediawiki-centralauth-production] + service_params => {} + unit_type => timer + require => Systemd::Unit[refinery-sqoop-mediawiki-centralauth-production.service] + restart => False + monitoring_critical => False + override => False + monitoring_contact_group => admins + ensure => present + migration_task => T407130 + monitoring_enabled => False- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-production.timer (refinery-sqoop-mediawiki-production.timer)]
- Parameters differences:
--- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-production.timer (refinery-sqoop-mediawiki-production.timer)].orig +++ Exec[systemd daemon-reload for refinery-sqoop-mediawiki-production.timer (refinery-sqoop-mediawiki-production.timer)] + refreshonly => True + before => ['Service[refinery-sqoop-mediawiki-production.timer]'] + command => /bin/systemctl daemon-reload
- Systemd::Timer[refinery-sqoop-mediawiki-production]
- Parameters differences:
--- Systemd::Timer[refinery-sqoop-mediawiki-production].orig +++ Systemd::Timer[refinery-sqoop-mediawiki-production] + ensure => present + fixed_random_delay => False + splay => 0 + timer_intervals => [{'start': 'OnCalendar', 'interval': '*-*-01 00:00:00'}] + unit_name => refinery-sqoop-mediawiki-production.service + accuracy => 15sec- Systemd::Service[refinery-sqoop-whole-mediawiki]
- Parameters differences:
--- Systemd::Service[refinery-sqoop-whole-mediawiki].orig +++ Systemd::Service[refinery-sqoop-whole-mediawiki] @@ - ensure => present + ensure => absent
- Systemd::Timer::Job[refinery-sqoop-mediawiki-centralauth-production]
- Parameters differences:
--- Systemd::Timer::Job[refinery-sqoop-mediawiki-centralauth-production].orig +++ Systemd::Timer::Job[refinery-sqoop-mediawiki-centralauth-production] + send_mail => True + syslog_force_stop => True + ignore_errors => False + logfile_group => analytics + monitoring_notes_url => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state + logfile_perms => all + send_mail_only_on_error => True + command => /usr/local/bin/kerberos-run-command analytics /usr/local/bin/refinery-sqoop-mediawiki-centralauth-production + monitoring_enabled => False + user => analytics + logfile_name => syslog.log + syslog_match_startswith => True + logging_enabled => False + interval => {'start': 'OnCalendar', 'interval': '*-*-01 00:00:00'} + fixed_random_delay => False + description => Schedules sqoop to import centralauth MediaWiki databases into Hadoop monthly. + logfile_basedir => /var/log/ + private_tmp => False + ensure => present + monitoring_contact_groups => admins + success_exit_status => [] + logfile_owner => analytics + environment => {} + send_mail_to => data-engineering-alerts@wikimedia.org- File[/lib/systemd/system/refinery-sqoop-mediawiki-centralauth-production.timer]
- Parameters differences:
--- File[/lib/systemd/system/refinery-sqoop-mediawiki-centralauth-production.timer].orig +++ File[/lib/systemd/system/refinery-sqoop-mediawiki-centralauth-production.timer] + notify => Exec[systemd daemon-reload for refinery-sqoop-mediawiki-centralauth-production.timer (refinery-sqoop-mediawiki-centralauth-production.timer)] + ensure => present + group => root + mode => 0444 + owner => root
- Content differences:
--- /lib/systemd/system/refinery-sqoop-mediawiki-centralauth-production.timer.orig +++ /lib/systemd/system/refinery-sqoop-mediawiki-centralauth-production.timer @@ -0,0 +1,12 @@ +[Unit] +Description=Periodic execution of refinery-sqoop-mediawiki-centralauth-production.service + +[Timer] +Unit=refinery-sqoop-mediawiki-centralauth-production.service +# Accuracy sets the maximum time interval around the execution time we want to allow +AccuracySec=15sec +OnCalendar=*-*-01 00:00:00 +RandomizedDelaySec=0 + +[Install] +WantedBy=multi-user.target
- Systemd::Timer::Job[refinery-sqoop-mediawiki-clouddb]
- Parameters differences:
--- Systemd::Timer::Job[refinery-sqoop-mediawiki-clouddb].orig +++ Systemd::Timer::Job[refinery-sqoop-mediawiki-clouddb] + send_mail => True + syslog_force_stop => True + ignore_errors => False + logfile_group => analytics + monitoring_notes_url => https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state + logfile_perms => all + send_mail_only_on_error => True + command => /usr/local/bin/kerberos-run-command analytics /usr/local/bin/refinery-sqoop-mediawiki-clouddb + monitoring_enabled => False + user => analytics + logfile_name => syslog.log + syslog_match_startswith => True + logging_enabled => False + interval => {'start': 'OnCalendar', 'interval': '*-*-01 00:00:00'} + fixed_random_delay => False + description => Schedules sqoop to import clouddb MediaWiki databases into Hadoop monthly. + logfile_basedir => /var/log/ + private_tmp => False + ensure => present + monitoring_contact_groups => admins + success_exit_status => [] + logfile_owner => analytics + environment => {} + send_mail_to => data-engineering-alerts@wikimedia.org- Systemd::Timer[refinery-sqoop-whole-mediawiki]
- Parameters differences:
--- Systemd::Timer[refinery-sqoop-whole-mediawiki].orig +++ Systemd::Timer[refinery-sqoop-whole-mediawiki] @@ - ensure => present + ensure => absent
- Kerberos::Systemd_timer[refinery-sqoop-mediawiki-clouddb]
- Parameters differences:
--- Kerberos::Systemd_timer[refinery-sqoop-mediawiki-clouddb].orig +++ Kerberos::Systemd_timer[refinery-sqoop-mediawiki-clouddb] + send_mail => True + require => ['File[/usr/local/bin/refinery-sqoop-mediawiki-clouddb]', 'File[/tmp/sqoop-jars]', 'Class[Kerberos::Wrapper]'] + syslog_force_stop => True + description => Schedules sqoop to import clouddb MediaWiki databases into Hadoop monthly. + logfile_basedir => /var/log/ + syslog_match_startswith => True + ensure => present + logfile_group => analytics + logfile_perms => all + command => /usr/local/bin/refinery-sqoop-mediawiki-clouddb + user => analytics + logfile_owner => analytics + interval => *-*-01 00:00:00 + environment => {} + send_mail_to => data-engineering-alerts@wikimedia.org- File[/lib/systemd/system/refinery-sqoop-mediawiki-clouddb.service]
- Parameters differences:
--- File[/lib/systemd/system/refinery-sqoop-mediawiki-clouddb.service].orig +++ File[/lib/systemd/system/refinery-sqoop-mediawiki-clouddb.service] + notify => Exec[systemd daemon-reload for refinery-sqoop-mediawiki-clouddb.service (refinery-sqoop-mediawiki-clouddb.service)] + ensure => present + group => root + mode => 0444 + owner => root
- Content differences:
--- /lib/systemd/system/refinery-sqoop-mediawiki-clouddb.service.orig +++ /lib/systemd/system/refinery-sqoop-mediawiki-clouddb.service @@ -0,0 +1,9 @@ +[Unit] +Description=Schedules sqoop to import clouddb MediaWiki databases into Hadoop monthly. +Documentation=https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state + +[Service] +Type=oneshot +User=analytics +SyslogIdentifier=kerberos-run-command +ExecStart=/usr/local/bin/systemd-timer-mail-wrapper --subject refinery-sqoop-mediawiki-clouddb --mail-to data-engineering-alerts@wikimedia.org --only-on-error /usr/local/bin/kerberos-run-command analytics /usr/local/bin/refinery-sqoop-mediawiki-clouddb
- Service[refinery-sqoop-mediawiki-centralauth-production.timer]
- Parameters differences:
--- Service[refinery-sqoop-mediawiki-centralauth-production.timer].orig +++ Service[refinery-sqoop-mediawiki-centralauth-production.timer] + ensure => running + provider => systemd + enable => True
- Systemd::Timer::Job[refinery-sqoop-whole-mediawiki]
- Parameters differences:
--- Systemd::Timer::Job[refinery-sqoop-whole-mediawiki].orig +++ Systemd::Timer::Job[refinery-sqoop-whole-mediawiki] @@ - ensure => present + ensure => absent
- File[/lib/systemd/system/refinery-sqoop-mediawiki-production.timer]
- Parameters differences:
--- File[/lib/systemd/system/refinery-sqoop-mediawiki-production.timer].orig +++ File[/lib/systemd/system/refinery-sqoop-mediawiki-production.timer] + notify => Exec[systemd daemon-reload for refinery-sqoop-mediawiki-production.timer (refinery-sqoop-mediawiki-production.timer)] + ensure => present + group => root + mode => 0444 + owner => root
- Content differences:
--- /lib/systemd/system/refinery-sqoop-mediawiki-production.timer.orig +++ /lib/systemd/system/refinery-sqoop-mediawiki-production.timer @@ -0,0 +1,12 @@ +[Unit] +Description=Periodic execution of refinery-sqoop-mediawiki-production.service + +[Timer] +Unit=refinery-sqoop-mediawiki-production.service +# Accuracy sets the maximum time interval around the execution time we want to allow +AccuracySec=15sec +OnCalendar=*-*-01 00:00:00 +RandomizedDelaySec=0 + +[Install] +WantedBy=multi-user.target
- Kerberos::Systemd_timer[refinery-sqoop-mediawiki-production]
- Parameters differences:
--- Kerberos::Systemd_timer[refinery-sqoop-mediawiki-production].orig +++ Kerberos::Systemd_timer[refinery-sqoop-mediawiki-production] + send_mail => True + require => ['File[/usr/local/bin/refinery-sqoop-mediawiki-production]', 'File[/tmp/sqoop-jars]', 'Class[Kerberos::Wrapper]'] + syslog_force_stop => True + description => Schedules sqoop to import production MediaWiki databases into Hadoop monthly. + logfile_basedir => /var/log/ + syslog_match_startswith => True + ensure => present + logfile_group => analytics + logfile_perms => all + command => /usr/local/bin/refinery-sqoop-mediawiki-production + user => analytics + logfile_owner => analytics + interval => *-*-01 00:00:00 + environment => {} + send_mail_to => data-engineering-alerts@wikimedia.org- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-centralauth-production.service (refinery-sqoop-mediawiki-centralauth-production.service)]
- Parameters differences:
--- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-centralauth-production.service (refinery-sqoop-mediawiki-centralauth-production.service)].orig +++ Exec[systemd daemon-reload for refinery-sqoop-mediawiki-centralauth-production.service (refinery-sqoop-mediawiki-centralauth-production.service)] + refreshonly => True + command => /bin/systemctl daemon-reload
- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-centralauth-production.timer (refinery-sqoop-mediawiki-centralauth-production.timer)]
- Parameters differences:
--- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-centralauth-production.timer (refinery-sqoop-mediawiki-centralauth-production.timer)].orig +++ Exec[systemd daemon-reload for refinery-sqoop-mediawiki-centralauth-production.timer (refinery-sqoop-mediawiki-centralauth-production.timer)] + refreshonly => True + before => ['Service[refinery-sqoop-mediawiki-centralauth-production.timer]'] + command => /bin/systemctl daemon-reload
- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-production.service (refinery-sqoop-mediawiki-production.service)]
- Parameters differences:
--- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-production.service (refinery-sqoop-mediawiki-production.service)].orig +++ Exec[systemd daemon-reload for refinery-sqoop-mediawiki-production.service (refinery-sqoop-mediawiki-production.service)] + refreshonly => True + command => /bin/systemctl daemon-reload
- Systemd::Unit[refinery-sqoop-mediawiki-centralauth-production.timer]
- Parameters differences:
--- Systemd::Unit[refinery-sqoop-mediawiki-centralauth-production.timer].orig +++ Systemd::Unit[refinery-sqoop-mediawiki-centralauth-production.timer] + ensure => present + override_filename => puppet-override.conf + unit => refinery-sqoop-mediawiki-centralauth-production.timer + restart => False + require => ['Class[Systemd]'] + override => False
- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-clouddb.timer (refinery-sqoop-mediawiki-clouddb.timer)]
- Parameters differences:
--- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-clouddb.timer (refinery-sqoop-mediawiki-clouddb.timer)].orig +++ Exec[systemd daemon-reload for refinery-sqoop-mediawiki-clouddb.timer (refinery-sqoop-mediawiki-clouddb.timer)] + refreshonly => True + before => ['Service[refinery-sqoop-mediawiki-clouddb.timer]'] + command => /bin/systemctl daemon-reload
- Exec[systemd daemon-reload for refinery-sqoop-whole-mediawiki.timer (refinery-sqoop-whole-mediawiki.timer)]
- Parameters differences:
--- Exec[systemd daemon-reload for refinery-sqoop-whole-mediawiki.timer (refinery-sqoop-whole-mediawiki.timer)].orig +++ Exec[systemd daemon-reload for refinery-sqoop-whole-mediawiki.timer (refinery-sqoop-whole-mediawiki.timer)] - before => ['Service[refinery-sqoop-whole-mediawiki.timer]']
- Service[refinery-sqoop-whole-mediawiki.timer]
- Parameters differences:
--- Service[refinery-sqoop-whole-mediawiki.timer].orig +++ Service[refinery-sqoop-whole-mediawiki.timer] + before => ['Exec[systemd daemon-reload for refinery-sqoop-whole-mediawiki.timer (refinery-sqoop-whole-mediawiki.timer)]'] @@ - ensure => running + ensure => stopped @@ - enable => True + enable => False
- File[/usr/local/bin/refinery-sqoop-mediawiki-clouddb]
- Parameters differences:
--- File[/usr/local/bin/refinery-sqoop-mediawiki-clouddb].orig +++ File[/usr/local/bin/refinery-sqoop-mediawiki-clouddb] + ensure => present + group => analytics + require => ['File[/usr/local/bin/refinery-sqoop-mediawiki-history]', 'File[/usr/local/bin/refinery-sqoop-mediawiki-not-history]'] + mode => 0550 + owner => analytics
- Content differences:
--- /usr/local/bin/refinery-sqoop-mediawiki-clouddb.orig +++ /usr/local/bin/refinery-sqoop-mediawiki-clouddb @@ -0,0 +1,12 @@ +#!/bin/bash +# NOTE: This file is managed by puppet +# +# This file is a wrapper running sqoop for whole mediawiki tables. +# It runs two sqoops in series: +# * first sqoops tables necessary for the mediawiki history dataset from clouddb +# * sqoops the remaining tables we care about from clouddb +# +# See each script referenced below for the list of clouddb tables + +/usr/local/bin/refinery-sqoop-mediawiki-history \ + && /usr/local/bin/refinery-sqoop-mediawiki-not-history
- Systemd::Timer[refinery-sqoop-mediawiki-centralauth-production]
- Parameters differences:
--- Systemd::Timer[refinery-sqoop-mediawiki-centralauth-production].orig +++ Systemd::Timer[refinery-sqoop-mediawiki-centralauth-production] + ensure => present + fixed_random_delay => False + splay => 0 + timer_intervals => [{'start': 'OnCalendar', 'interval': '*-*-01 00:00:00'}] + unit_name => refinery-sqoop-mediawiki-centralauth-production.service + accuracy => 15sec
- Content differences:
- File[/usr/local/bin/refinery-sqoop-mediawiki-clouddb]
- Service[refinery-sqoop-whole-mediawiki.timer]
- Exec[systemd daemon-reload for refinery-sqoop-whole-mediawiki.timer (refinery-sqoop-whole-mediawiki.timer)]
- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-clouddb.timer (refinery-sqoop-mediawiki-clouddb.timer)]
- Systemd::Unit[refinery-sqoop-mediawiki-centralauth-production.timer]
- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-production.service (refinery-sqoop-mediawiki-production.service)]
- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-centralauth-production.timer (refinery-sqoop-mediawiki-centralauth-production.timer)]
- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-centralauth-production.service (refinery-sqoop-mediawiki-centralauth-production.service)]
- Content differences:
- File[/lib/systemd/system/refinery-sqoop-mediawiki-production.timer]
- Systemd::Timer::Job[refinery-sqoop-whole-mediawiki]
- Content differences:
- File[/lib/systemd/system/refinery-sqoop-mediawiki-clouddb.service]
- Kerberos::Systemd_timer[refinery-sqoop-mediawiki-clouddb]
- Systemd::Timer[refinery-sqoop-whole-mediawiki]
- Content differences:
- File[/lib/systemd/system/refinery-sqoop-mediawiki-centralauth-production.timer]
- Systemd::Timer::Job[refinery-sqoop-mediawiki-centralauth-production]
- Systemd::Service[refinery-sqoop-whole-mediawiki]
- Systemd::Timer[refinery-sqoop-mediawiki-production]
- Exec[systemd daemon-reload for refinery-sqoop-mediawiki-production.timer (refinery-sqoop-mediawiki-production.timer)]
- Content differences:
- File[/usr/local/bin/refinery-sqoop-mediawiki-production]
- Content differences:
- File[/lib/systemd/system/refinery-sqoop-mediawiki-production.service]
- Systemd::Unit[refinery-sqoop-mediawiki-clouddb.timer]
- File[/usr/local/bin/refinery-sqoop-whole-mediawiki]
- Systemd::Service[refinery-sqoop-mediawiki-production]
- Systemd::Unit[refinery-sqoop-whole-mediawiki.service]
- Content differences:
- File[/lib/systemd/system/refinery-sqoop-mediawiki-clouddb.timer]
- Systemd::Timer[refinery-sqoop-mediawiki-clouddb]
- Systemd::Service[refinery-sqoop-mediawiki-clouddb]
- File[/usr/local/bin/refinery-sqoop-mediawiki-history]
- Content differences:
- File[/usr/local/bin/refinery-sqoop-mediawiki-centralauth-production]
- File[/usr/local/bin/refinery-sqoop-mediawiki-production-history]
- Service[refinery-sqoop-mediawiki-clouddb.timer]
- Content differences:
- File[/lib/systemd/system/refinery-sqoop-mediawiki-centralauth-production.service]
- Kerberos::Systemd_timer[refinery-sqoop-mediawiki-centralauth-production]
- File[/lib/systemd/system/refinery-sqoop-whole-mediawiki.timer]
- Systemd::Unit[refinery-sqoop-mediawiki-clouddb.service]
- Systemd::Unit[refinery-sqoop-mediawiki-production.service]
- Systemd::Unit[refinery-sqoop-mediawiki-production.timer]
- Parameters differences: