Compilation results for an-test-presto1001.eqiad.wmnet: System changes detected

Catalog differences

Summary

Resources only in the new catalog

• File[/srv/presto/var/spill]
• File[/etc/presto/selectors.json]
• File[/etc/presto/resource-groups.json]

Resources modified

• Class[Presto::Server]

  Parameters differences:

  --- Class[Presto::Server].orig
  +++ Class[Presto::Server]
  
  +    spill_enabled           => True
  +    resource_config         => {'glob_mem_limit': '100%', 'glob_concur_limit': 10, 'glob_max_queued': 50, 'hp_mem_limit': '35%', 'hp_concur_limit': 3, 'hp_max_queued': 10, 'hp_sched_weight': 6, 'std_mem_limit': '45%', 'std_concur_limit': 5, 'std_max_queued': 25, 'std_sched_weight': 3, 'hv_mem_limit': '20%', 'hv_concur_limit': 1, 'hv_max_queued': 5, 'hv_sched_weight': 1}
  +    resource_groups_enabled => True
  @@
  -    config_properties       => {'jmx.rmiregistry.port': 8279, 'discovery.uri': 'https://analytics-test-presto.eqiad.wmnet:8281', 'node-scheduler.network-topology': 'legacy', 'http-server.http.port': 8280, 'http-server.http.enabled': True, 'query.max-memory': '2GB', 'query.max-memory-per-node': '2GB', 'query.max-total-memory-per-node': '2GB', 'join-distribution-type': 'AUTOMATIC', 'http-server.https.keystore.path': '/etc/presto/ssl/server.p12', 'http-server.https.keystore.key': 'batman', 'internal-communication.https.required': True, 'internal-communication.https.keystore.path': '/etc/presto/ssl/server.p12', 'internal-communication.https.keystore.key': 'batman', 'internal-communication.https.trust-store-path': '/etc/ssl/certs/java/cacerts', 'internal-communication.https.trust-store-password': 'changeit', 'http-server.https.port': 8281, 'http-server.https.enabled': True, 'internal-communication.kerberos.enabled': True, 'http-server.authentication.type': 'KERBEROS', 'http.authentication.krb5.config': '/etc/krb5.conf', 'http.server.authentication.krb5.keytab': '/etc/security/keytabs/presto/presto.keytab', 'http.server.authentication.krb5.service-name': 'presto'}
  +    config_properties       => {'jmx.rmiregistry.port': 8279, 'discovery.uri': 'https://analytics-test-presto.eqiad.wmnet:8281', 'node-scheduler.network-topology': 'legacy', 'http-server.http.port': 8280, 'http-server.http.enabled': True, 'memory.heap-headroom-per-node': '30GB', 'query.max-execution-time': '30m', 'query.max-memory': '2GB', 'query.max-memory-per-node': '2GB', 'query.max-total-memory-per-node': '2GB', 'query.max-spill-per-node': '100GB', 'spill-enabled': True, 'spiller-spill-path': '/srv/presto/var/spill', 'task.concurrency': 4, 'task.max-worker-threads': 16, 'join-distribution-type': 'AUTOMATIC', 'http-server.https.keystore.path': '/etc/presto/ssl/server.p12', 'http-server.https.keystore.key': 'batman', 'internal-communication.https.required': True, 'internal-communication.https.keystore.path': '/etc/presto/ssl/server.p12', 'internal-communication.https.keystore.key': 'batman', 'internal-communication.https.trust-store-path': '/etc/ssl/certs/java/cacerts', 'internal-communication.https.trust-store-password': 'changeit', 'http-server.https.port': 8281, 'http-server.https.enabled': True, 'internal-communication.kerberos.enabled': True, 'http-server.authentication.type': 'KERBEROS', 'http.authentication.krb5.config': '/etc/krb5.conf', 'http.server.authentication.krb5.keytab': '/etc/security/keytabs/presto/presto.keytab', 'http.server.authentication.krb5.service-name': 'presto'}
  

• Presto::Properties[config]

  Parameters differences:

  --- Presto::Properties[config].orig
  +++ Presto::Properties[config]
  
  @@
  -    properties => {'coordinator': False, 'node-scheduler.include-coordinator': False, 'http-server.http.port': 8280, 'jmx.rmiregistry.port': 8279, 'discovery.uri': 'https://analytics-test-presto.eqiad.wmnet:8281', 'node-scheduler.network-topology': 'legacy', 'http-server.http.enabled': True, 'query.max-memory': '2GB', 'query.max-memory-per-node': '2GB', 'query.max-total-memory-per-node': '2GB', 'join-distribution-type': 'AUTOMATIC', 'http-server.https.keystore.path': '/etc/presto/ssl/server.p12', 'http-server.https.keystore.key': 'batman', 'internal-communication.https.required': True, 'internal-communication.https.keystore.path': '/etc/presto/ssl/server.p12', 'internal-communication.https.keystore.key': 'batman', 'internal-communication.https.trust-store-path': '/etc/ssl/certs/java/cacerts', 'internal-communication.https.trust-store-password': 'changeit', 'http-server.https.port': 8281, 'http-server.https.enabled': True, 'internal-communication.kerberos.enabled': True, 'http-server.authentication.type': 'KERBEROS', 'http.authentication.krb5.config': '/etc/krb5.conf', 'http.server.authentication.krb5.keytab': '/etc/security/keytabs/presto/presto.keytab', 'http.server.authentication.krb5.service-name': 'presto'}
  +    properties => {'coordinator': False, 'node-scheduler.include-coordinator': False, 'http-server.http.port': 8280, 'jmx.rmiregistry.port': 8279, 'discovery.uri': 'https://analytics-test-presto.eqiad.wmnet:8281', 'resource-groups.configuration-manager': 'file', 'resource-groups.config-file': '/etc/presto/resource-groups.json', 'resource-groups.selector-file': '/etc/presto/selectors.json', 'node-scheduler.network-topology': 'legacy', 'http-server.http.enabled': True, 'memory.heap-headroom-per-node': '30GB', 'query.max-execution-time': '30m', 'query.max-memory': '2GB', 'query.max-memory-per-node': '2GB', 'query.max-total-memory-per-node': '2GB', 'query.max-spill-per-node': '100GB', 'spill-enabled': True, 'spiller-spill-path': '/srv/presto/var/spill', 'task.concurrency': 4, 'task.max-worker-threads': 16, 'join-distribution-type': 'AUTOMATIC', 'http-server.https.keystore.path': '/etc/presto/ssl/server.p12', 'http-server.https.keystore.key': 'batman', 'internal-communication.https.required': True, 'internal-communication.https.keystore.path': '/etc/presto/ssl/server.p12', 'internal-communication.https.keystore.key': 'batman', 'internal-communication.https.trust-store-path': '/etc/ssl/certs/java/cacerts', 'internal-communication.https.trust-store-password': 'changeit', 'http-server.https.port': 8281, 'http-server.https.enabled': True, 'internal-communication.kerberos.enabled': True, 'http-server.authentication.type': 'KERBEROS', 'http.authentication.krb5.config': '/etc/krb5.conf', 'http.server.authentication.krb5.keytab': '/etc/security/keytabs/presto/presto.keytab', 'http.server.authentication.krb5.service-name': 'presto'}
  

• File[/etc/presto/selectors.json]

  Parameters differences:

  --- File[/etc/presto/selectors.json].orig
  +++ File[/etc/presto/selectors.json]
  
  +    owner => root
  +    group => root
  

  Content differences:

  --- /etc/presto/selectors.json.orig
  +++ /etc/presto/selectors.json
  @@ -0,0 +1,22 @@
  +{
  +  "selectors": [
  +    {
  +      "source": "superset_user",
  +      "group": "global.high_priority"
  +    },
  +    {
  +      "user": "growthbook_user",
  +      "group": "global.high_priority"
  +    },
  +    {
  +      "clientTags": [
  +        "heavy"
  +      ],
  +      "group": "global.heavy"
  +    },
  +    {
  +      "queryType": "SELECT",
  +      "group": "global.standard"
  +    }
  +  ]
  +}

• File[/etc/presto/resource-groups.json]

  Parameters differences:

  --- File[/etc/presto/resource-groups.json].orig
  +++ File[/etc/presto/resource-groups.json]
  
  +    owner => root
  +    group => root
  

  Content differences:

  --- /etc/presto/resource-groups.json.orig
  +++ /etc/presto/resource-groups.json
  @@ -0,0 +1,34 @@
  +{
  +  "rootGroups": [
  +    {
  +      "name": "global",
  +      "softMemoryLimit": "100%",
  +      "hardConcurrencyLimit": 10,
  +      "maxQueued": 50,
  +      "schedulingPolicy": "weighted",
  +      "subGroups": [
  +        {
  +          "name": "high_priority",
  +          "softMemoryLimit": "35%",
  +          "hardConcurrencyLimit": 3,
  +          "maxQueued": 10,
  +          "schedulingWeight": 6
  +        },
  +        {
  +          "name": "standard",
  +          "softMemoryLimit": "45%",
  +          "hardConcurrencyLimit": 5,
  +          "maxQueued": 25,
  +          "schedulingWeight": 3
  +        },
  +        {
  +          "name": "heavy",
  +          "softMemoryLimit": "20%",
  +          "hardConcurrencyLimit": 1,
  +          "maxQueued": 5,
  +          "schedulingWeight": 1
  +        }
  +      ]
  +    }
  +  ]
  +}

• File[/etc/presto/config.properties]

  Content differences:

  --- /etc/presto/config.properties.orig
  +++ /etc/presto/config.properties
  @@ -20,8 +20,18 @@
   internal-communication.kerberos.enabled=true
   jmx.rmiregistry.port=8279
   join-distribution-type=AUTOMATIC
  +memory.heap-headroom-per-node=30GB
   node-scheduler.include-coordinator=false
   node-scheduler.network-topology=legacy
  +query.max-execution-time=30m
   query.max-memory=2GB
   query.max-memory-per-node=2GB
  +query.max-spill-per-node=100GB
   query.max-total-memory-per-node=2GB
  +resource-groups.config-file=/etc/presto/resource-groups.json
  +resource-groups.configuration-manager=file
  +resource-groups.selector-file=/etc/presto/selectors.json
  +spill-enabled=true
  +spiller-spill-path=/srv/presto/var/spill
  +task.concurrency=4
  +task.max-worker-threads=16

• File[/srv/presto/var/spill]

  Parameters differences:

  --- File[/srv/presto/var/spill].orig
  +++ File[/srv/presto/var/spill]
  
  +    require => Package[presto-server]
  +    ensure  => directory
  +    owner   => presto
  +    mode    => 0755
  +    before  => Service[presto-server]
  +    group   => presto
  

• Class[Profile::Presto::Server]

  Parameters differences:

  --- Class[Profile::Presto::Server].orig
  +++ Class[Profile::Presto::Server]
  
  +    spill_enabled           => True
  +    resource_config         => {'glob_mem_limit': '100%', 'glob_concur_limit': 10, 'glob_max_queued': 50, 'hp_mem_limit': '35%', 'hp_concur_limit': 3, 'hp_max_queued': 10, 'hp_sched_weight': 6, 'std_mem_limit': '45%', 'std_concur_limit': 5, 'std_max_queued': 25, 'std_sched_weight': 3, 'hv_mem_limit': '20%', 'hv_concur_limit': 1, 'hv_max_queued': 5, 'hv_sched_weight': 1}
  +    resource_groups_enabled => True
  @@
  -    config_properties       => {'node-scheduler.network-topology': 'legacy', 'query.max-memory': '2GB', 'query.max-memory-per-node': '2GB', 'query.max-total-memory-per-node': '2GB', 'join-distribution-type': 'AUTOMATIC'}
  +    config_properties       => {'node-scheduler.network-topology': 'legacy', 'memory.heap-headroom-per-node': '30GB', 'query.max-execution-time': '30m', 'query.max-memory': '2GB', 'query.max-memory-per-node': '2GB', 'query.max-total-memory-per-node': '2GB', 'query.max-spill-per-node': '100GB', 'spill-enabled': True, 'spiller-spill-path': '/srv/presto/var/spill', 'task.concurrency': 4, 'task.max-worker-threads': 16, 'join-distribution-type': 'AUTOMATIC'}
  

• Service[presto-server]

  Parameters differences:

  --- Service[presto-server].orig
  +++ Service[presto-server]
  
  @@
  -    require => ['Presto::Properties[config]', 'Presto::Properties[node]', 'Presto::Properties[log]', 'File[/etc/presto/jvm.config]', 'File[/var/log/presto]', 'Rsyslog::Conf[presto-server]', 'Systemd::Override[presto-umask]']
  +    require => ['Presto::Properties[config]', 'Presto::Properties[node]', 'Presto::Properties[log]', 'File[/etc/presto/jvm.config]', 'File[/var/log/presto]', 'Rsyslog::Conf[presto-server]', 'Systemd::Override[presto-umask]', 'File[/etc/presto/resource-groups.json]', 'File[/etc/presto/selectors.json]']
  

Relevant files

• Production catalog
• Change catalog
• Production errors/warnings
• Change errors/warnings
• Core Diff
• Standard Diff