Fixed issues in 7.1.9 SP1 CHF 12

Know more about the list of fixes that are shipped for CDP Private Cloud Base version 7.1.9 SP1 CHF 12.

CDPD-82979: Added Cloudera-specific configuration for MapPartitionsRDD default determinacy

To avoid failures during error handling, a Cloudera-specific configuration was added to change the default determinacy level for any MapPartitionsRDD.

CDPD-91213: Non-deterministic SQL expressions failed to set indeterminate map stage output level

Previously, a bug in Spark related to handling non-deterministic data types could cause data corruption or loss during task failures and retries. This issue is now fixed.

Apache Jira: SPARK-51272

CDPD-89515: Fixed assertion error in Spark submit application

A missing commons-configuration dependency in the spark-atlas-connector assembly jar failed caused an assertion error, as the spark-submit application did not return an application ID. This issue is now fixed.

CDPD-87670: [Atlas UI] Apache Atlas Glossary becomes unresponsive when the page size is set to 50

Previously, when the page limit was set to 50 in Glossary, the user interface was unresponsive due to an incorrect API call with an error message, when trying to show more than 25 associated entities for a glossary item. This issue is now fixed by updating the API function logic, and now more than 50 associated glossary items can be shown.

Apache Jira: ATLAS-5067

CDPD-88447: Prevent unchecked scheduling of flush operations under memory pressure

Previously, in some clusters, tablet servers (tservers) maintained high memory usage up to 60%–80%, causing the maintenance manager to limit operations to WAL garbage collection and MRS/DRS flushes. This led to degraded performance and potential OOM failures.

The issue is now resolved with a configurable parameter that allows the maintenance manager to continue additional operations under moderate memory pressure. It dynamically adjusts MRS/DRS flush probability based on memory usage thresholds, improving resource management and preventing performance issues.

Apache Jira: KUDU-3407

CDPD-75829: Memory leak when using Ozone FS filesystem through Hadoop FileContext API

Previously, memory used to leak when the Hadoop FileContext API was used. The Hadoop FileContext API internally uses the AbstractFileSystem API that is implemented by the org.apache.hadoop.fs.ozone.RootedOzFs class when using the Ozone FS filesystem. The org.apache.hadoop.fs.ozone.RootedOzFs class internally creates a new RootedOzoneFileSystem instance that never closes and might cause a memory leak. This issue is now fixed.

Apache Jira: HDDS-11635

CDPD-72076: The OMDoubleBuffer error is displayed when handling OMRequest: cmdType: SnapshotMoveDeletedKeys

Previously, the Ozone Manager crashed when the follower was lagging and executed the purgeKeys or snapshotMoveDeletedKeys task for the Snapshot in a single transaction. This issue is now fixed.

Apache Jira: HDDS-11152

CDPD-73318: Datanode cannot report for a long time

Previously, some datanodes could not report for a long time and their status was DEAD or STALE. This issue is now fixed.

Apache Jira: HDDS-11331

CDPD-89911: High hash collision could slow down table renames

This issue is addressed by optimizing the hashing function for PartColNameInfo, which significantly improved performance when renaming tables with a large number of columns.

Apache Jira: HIVE-29165

CDPD-90263: Performance degradation with LIMIT clause in vectorized queries

Some vectorized queries that included a LIMIT clause would process all rows instead of stopping after the limit was reached. This issue is now resolved by modifying the vectorized code to correctly check if child operators are done, allowing the query to terminate early and significantly improving performance.

Apache Jira: HIVE-24930

CDPD-90821: Bucket map join query hangs due to auto-parallelism changes

Queries using a bucket map join could hang when the auto reducer parallelism feature changed the number of tasks for a source vertex. This issue is now resolved by temporarily disabling the auto reducer parallelism feature for any vertices that are the source of a bucket map join.

Apache Jira: HIVE-27078

CDPD-91157: Incorrect results from bucket map join on decimal bucketed columns

Queries using a bucket map join on tables with decimal bucketed columns could return incorrect results.

The issue is now resolved by ensuring that the bucket map join correctly gathers bucket expression indices, allowing for a complete hash table and accurate results.

Apache Jira: HIVE-27267

CDPD-89389: Major compaction could lead to data loss

A major compaction could cause data loss when a table was overwritten and then new data was inserted.

This issue is now resolved by ensuring that the compactor correctly handles these bucket mismatches, so all base files are included in the major compaction, preventing data loss.

Apache Jira: HIVE-28700

CDPD-89791: Incorrect COUNT(DISTINCT) results in subqueries

Queries using the COUNT(DISTINCT) SQL function within a subquery were returning incorrect results.

This issue is now resolved by ensuring that the optimizer uses the most current plan, fixing the incorrect count.

Apache Jira: HIVE-27084

CDPD-89852/ CDPD-89853: Crash when casting timestamp strings with timezone offsets to DATE

Attempting to cast a timestamp string that included a timezone offset, such as "+08:00" in "2025-08-31 06:23:24.9392129 +08:00", to the DATE data type would cause a crash.

This issue is now resolved by adding a check to ensure that the timestamp string length does not exceed the maximum length of the default date-time format. Longer strings will now use a lazily-created format, which prevents the crash.

Apache Jira: IMPALA-14383

CDPD-89964/CDPD-88861: SHA2 function fails in FIPS mode

When using the SHA2 function with a bit length of 384 in a FIPS-compliant environment, Impala crashed.

This issue is now resolved by updating the function to ensure compatibility with FIPS mode, preventing the crash.

Apache Jira: IMPALA-14385

CDPD-90828: Revert HDDS-8923 since it is using a lot of memory

HDDS-8923 added CacheMetrics to XceiverClientManager and created a unique name per instance. This process could use a lot of memory. This issue is now fixed, as HDDS-8923 is now reverted.

COMPX-22905: Null Pointer Exception in FifoCandidatesSelector during preemption check in Scheduling Monitor

A Null Pointer Exception occurred during preemption because childless parent queues were being misidentified as leaf queues. This issue is now resolved by adding an explicit check to ensure that only a true instance of the Abstract Leaf Queues are processed.

CDPD-90931: Emit offset syncs more often than offset.lag.max for low-throughput or finite partitions

Previously, Streams Replication Manager (SRM) only emitted offset syncs based on message count. This could cause steady offset translation lag (up to the value of offset.lag.max) for finite-length or low-throughput topics.

This issue is now resolved by adding time-based offset sync emission. This ensures offset syncs are emitted regularly, even for slow or stopped topics

Apache Jira: KAFKA-15906

COMPX-22434: Queue Manager failed to recreate DQS rule after exiting mixed mode

Previously, Queue Manager failed to recreate the Dynamic Queue Schedules (DQS) rule after exiting YARN mixed mode. This issue is now resolved.