Fixed issues in 7.1.9 SP1 CHF 7

CDPD-73375: Publishing hadoop metrics immediately in Prometheus sink fills up SinkQueue quickly

Prometheus sink already have a mechanism to publish metrics every 10 seconds by default using call back with timer event. So, we removed the code to publish immediately and this issue is fixed.

CDPD-80823: Snapshot creation is removing extra keys from Active Object storage's DB

Wrong keys were trapped in the DeletedTable of the snapshot if the OBS bucket name is a prefix for another OBS bucket, resulting in orphaned blocks.

This issue is fixed and the extra keys will not be removed from the DeletedTable for Active Object storage.

CDPD-78781: Tarball creation is interfering with snapshot purge

If Tarball creation is in process while the snapshot is getting purged, it fails the snapshot db dir delete command causing the snapshot db dir to linger around even though the snapshot is purged from the snapshotInfoTable and needs to manually delete the dir.

This issue is fixed and the Synchronized SnapshotDeletingService on BootstrapStateHandler.Lock makes sure that no background service is running when Tarball creation is in progress.

CDPD-57559: New Ozone Manager leader cannot verify the Ozone delegation token signed by old Ozone Manager leader

If an Ozone cluster is upgraded and then later downgraded, then the new Ozone Manager after downgrade cannot verify the new Ozone delegation tokens issued before downgrade causing the clients which are still running during this upgrade download period to fail. If there is no downgrade, then everything is fine.

This issue is fixed and this fix changes the Ozone delegation token sign from using asymmetric key to symmetric key.

CDPD-81557: Improve Ranger Admin Diagnostic Collection command from Cloudera Manager scripts

The Ranger Admin Diagnostic Collection command is now enhanced and a new configuration option called ranger.admin.diag.metrics.collection.type is now introduced. This option allows you to specify the type of metrics data to be collected. This option appears while you are upgrading from 7.1.9 SP1 CHF6 to 7.1.9 SP1 CHF7, and upgrade Cloudera Manager.

CDPD-80112: Knox service might fail due to JARs picked up from the /usr/share/java folder

Knox service might fail due to Java Archive (JAR) files picked up from the /usr/share/java folder.

This issue is now resolved.

Apache JIRA: KNOX-3108

CDPD-80018: Concurrent policy label update leads to an infinite loop

Concurrent updates to policy labels for a policy lead to an infinite loop causing the Ranger database to crash. Concurrent policy label update and underlying policy deletion also result in the same behavior.

Fixed concurrent updates to policy labels. Policy label updates are skipped if the underlying policy gets deleted.

CDPD-80628: Unable to query Iceberg tables from Impala

After upgrading to CDP Private Cloud Base version 7.1.9 SP1 CHF 6 or lower versions, you may notice issues while querying Iceberg tables from Impala. An error is reported indicating that the migrated file has unexpected schema or partitioning.

In migrated Iceberg tables, there can be data files with missing field IDs. It is assumed that their schema corresponds to the table schema at the point when the table migration happened, which means field IDs can be generated during runtime. The logic becomes complicated when there are complex types in the table and the table is partitioned. In such cases, some adjustments are required during field ID generation and we verify that the file schema corresponds to the table schema (during migration).

This fix ensures that these adjustments are not needed when the table does not have complex types and therefore schema verification is skipped. As a result, Impala can still read the table if there were some trivial schema changes before migration.

Apache JIRA: IMPALA-13853

CDPD-79911: Netty upgrade to 4.1.118.Final

Upgraded Netty to version 4.1.118.Final due to CVE-2025-24970, CVE-2025-25193.

CDPD-77107: Knox UI session timeout is not working with SAML authentication

This issue is resolved by the pac4j.cookie.max.age parameter introduced for the pac4j provider, which Knox uses for SAML authentication. This parameter enforces cookie timeout for the cookies created by the pac4j provider.

To set the pac4j.cookie.max.age parameter, go to Cloudera Manager > Knox > Configuration, and add the following value to the

Knox Simplified Topology
            Management - SSO Authentication Provider

field: federation.param.pac4j.cookie.max.age={value}

Apache JIRA: KNOX-3077

CDPD-80208: When the ignore pattern property is added on a hook, Iceberg tables are also ignored

When using the atlas.hook.hive.hive_table.ignore.pattern=.*test_.* property as an ignore pattern by Hive Server 2 or Hive Metastore hooks, it no longer ignores iceberg_table entities next to Hive entities. hive_table ignore patterns ignore only hive_table related entities.

CDPD-72496: File extension restrictions for Hue file uploads

Earlier, Hue permitted uploading all file types to the configured filesystems, including unsupported extensions which posed a security risk.

To enhance security, Hue now allows restricting specific file extensions across all configured filesystems. For example, you can allow .csv file uploads while blocking .exe files. By default, no file extensions are restricted during file uploads.

For more information, see Managing file extensions for Hue uploads.

CDPD-77746: Metrics Collection Failure with "DPI-1010: not connected" on Oracle DB backend

Metrics collection fails with the error

"DPI-1010: not
            connected"

when using an Oracle database as the backend.

This issue is now fixed.

CDPD-79237: Hive Metastore schema upgrade fails due to NULL values

Upgrading from CDP Private Cloud Base 7.1.7.2052 to 7.1.9.1010 fails during the Hive Metastore schema upgrade. The upgrade script issues the following command:

ALTER TABLE "DBS" ALTER COLUMN "TYPE" SET DEFAULT 'NATIVE', ALTER COLUMN "TYPE" SET NOT NULL;

This fails because the DBS.TYPE column contains NULL values. These NULLs are introduced by canary databases created by Cloudera Manager, which insert entries in the HMS database without setting the TYPE.

The issue was addressed by ensuring that canary databases created by Cloudera Manager correctly populate the TYPE column in the DBS table, preventing NULL values and allowing the schema upgrade to proceed.

CDPD-81079: Metastore did not enforce maximum Thrift message size

The Metastore server always used the default 100 MB Thrift message size, even if a higher limit was set on the client. Large client requests caused silent connection drops and unclear exceptions.

The issue is addressed by applying the configured maximum Thrift message size on the server, ensuring consistent behavior and avoiding unexpected disconnections.

Apache Jira: HIVE-28824

CDPD-80942: Aborted transactions due to timeout are not logged in notification log

Transactions aborted due to timeout are not logged as ABORTED in the NOTIFICATION_LOG.

To address the issue, the fix includes logging an ABORT event for transactions aborted due to timeout. This allows such transactions to be replicated and cleaned up from the target cluster without delay, reducing unnecessary overhead and improving query performance.

Apache Jira: HIVE-27797

CDPD-79990: Incorrect error shown for successful LOAD DATA command in Beeline

When running a LOAD DATA statement using EXECUTE IMMEDIATE in Beeline with HiveServer2, the data loads successfully into the target table, but an error is incorrectly displayed in the console. HiveServer2 logs show a NullPointerException when generating result set metadata, even though the command completes and the table is correctly updated.

This issue is fixed.

Apache Jira: HIVE-28766

CDPD-77626: Improving performance of ALTER PARTITION operations using direct SQL

Running ALTER PARTITION operations using direct SQL failed for some databases. The failures occurred due to missing data type conversions for CLOB and Boolean fields, causing the system to fall back to slower ORM (Object Relational Mapping) paths.

The issue was addressed by adding proper handling for CLOB and Boolean type conversions. With this fix, ALTER PARTITION operations now run successfully using direct SQL.

Apache Jira: HIVE-28271, HIVE-27530

CDPD-49323: INSERT statement does not respect Ranger policies for HDFS

In a cluster with Ranger auth (and with legacy catalog mode), even if you provide RWX to cm_hdfs -> all-path for the user impala, inserting into a table whose HDFS POSIX permissions happen to exclude impala access will result in

AnalysisException: Unable to INSERT into target table (default.t1) because Impala does not have WRITE access to HDFS location: hdfs://XXXXXXXXXXXX

To address the issue, the fix includes skipping the HDFS permission check and assuming the Impala service user has read and write access to all HDFS paths associated with the target table during query analysis when Ranger is enabled. This ensures that Ranger policies are respected during insert operations.

Apache Jira: IMPALA-11871

COMPX-77765: Hadoop - Upgraded Kafka-clients to version 3.7.2/3.8.1+ due to CVE-2024-56128

Upgraded the Kafka client to verion 3.9.0 to fix CVE-2024-31141.

COMPX-78928: Backport HADOOP-16299 to avoid module access violation in LdapGroupsMapping

When LdapGroupsMapping was the selected group mapping method, running any commands that relied on this group mapping lead to an exception due to a Java module access violation when the JVM version was above Java11. This illegal access is now avoided in the group mapping implementation without functional changes.

Apache Jira: HADOOP-16299

CDPD-79763: Fix clobbering of files across epochs in Spark Structured streaming with Iceberg

A bug in structured streaming was found that resulted in clobbering of files in Iceberg tables.

The issue was addressed by backporting an upstream fix.

CDPD-77461: Livy: Upgrade mina-core to 2.0.27/2.1.10/2.2.4+ due to CVE-2024-52046

Upgrades mina-core to patch a vulnerability issue.