Fixed Issues in Apache Atlas

Review the list of Atlas issues that are resolved in Cloudera Runtime 7.1.9.

OPSAPS-67782: ZDU | During rolling upgrade one ATLAS server failed to start while other server went fine

Exclusion of jackson databind from parent dependency

OPSAPS-67878: [CKP-2,3,4] Apply config injectors for all Atlas csd

Changes made in 7.1.8 CSD, for 7.1.9 CSD CM team has already taken care.

CDPD-56309: [Atlas] Download Search with Basic Search gives java.io.FileNotFoundException

Default value for the download directory will be read from system property "user.dir" instead of property "atlas.home"

CDPD-49053: Atlas - Upgrade Tinkerpop to 3.5.4

Upgrade Tinkerpop to 3.5.4

CDPD-48090: Atlas - Upgrade icu4j to 66.1+ due to CVE-2020-21913

Upgrade icu4j to 66.1+ due to CVE-2020-21913

CDPD-50022: Test failure - Atlas 'Service Unavailable' error

Circular dependency issue caused by repository.src.main.java.org.apache.atlas.tasks.TaskRegistry class is resolved by adding @Lazy annotation on @Component definition of this class.

CDPD-29307: Kafka producer entity stays in incomplete state in Atlas

The Kafka-Atlas plugin now fully creates Producer and Consumer entities and won't generate incomplete ones.

CDPD-57305: Atlas - Upgrade moment.js to 2.29.4 due to CVE-2022-24785, CVE-2022-31129

Updated moment.js version to 2.29.4

CDPD-45073: Implement aging for audits stored by Atlas.

Feature to reduce Atlas audit storage by aging out audit data, using different criteria like TTL and audit count

CDPD-58592: [CKP4 (same value)] Atlas not null filter on classification returns null values

Cause: It is because of Solr version upgrade, untill 8.4.1, Solr supported non empty string. Fix: For IndexQuery : ["" TO *] works to get nonEmpty field entities. For Inmemory Predicates: Used NonEmptyPredicate

CDPD-55098: 7.1.9 - Dynamic Index Recovery issues and improvements

Introduced AtlasClient API to perform index recovery

CDPD-50762: Regression : admin/audits , admin/purge fail with "[__AtlasAuditEntry.startTime] is not indexed in the targeted index [vertex_index]" 7.1.9

Cause: The attributes of AtlasAuditEntry type where not getting indexed in the Solr, because the entity of AtlasAuditEntry gets created before the attributes gets indexed. FIx: Ordering the typeDefChangeListeners helped, to create the AtlasAuditEntry typeDef first before auditing.

CDPD-59409: Tags are not getting synced from one node of rangertagsync

Log4j version incompatibility between Atlas and Ranger led to this issue. A temporary fix has been merged to avoid the exception.

CDPD-54645: Ranger tag sync for Iceberg table type

Added iceberg support for Ranger Tagsync

CDPD-59713: [backport] Indexed string field (solr.StrField) which is too large ERROR

Imapala process entities created by ImpalaHook saves query-string in name field. Since query-string can be large, you are getting the longer than the max error.

To store qualifiedName in name field instead of query-string

CDPD-49495: OOM issue with DSL search caching added by CDPD-27872

Fixed OOM seen after large number DSL search requests. This was caused by the fix for ATLAS-4347

CDPD-57277: Atlas - Upgrade Spring Framework to 5.3.27/6.0.8 due to CVE-2023-20861, CVE-2023-20860 and CVE-2023-20863

Upgrade Spring Framework to 5.3.27 from 5.3.21

CDPD-57433: Atlas - Upgrade gremlin shaded to 3.5.5+ due to jackson-databind CVEs

Upgrade gremlin shaded to 3.5.5 from 3.5.4

CDPD-49450: Atlas - Upgrade jettison to 1.5.4 due to CVE-2022-45685 and CVE-2022-45693

Upgrade jettison to 1.5.4 from 1.3.7.

CDPD-53745: commits in CDH-7.1.8.x but NOT IN CDH-7.1.9.x

This issue is fix in CDPD-52776.

CDPD-54846: Atlas: CVE-2023-24998-upgrade commons-fileupload library to version 1.5

Upgrade commons-fileupload library to version 1.5 from 1.3.3.

CDPD-54852: Backward compatibility for check provided for AttributeName in Parent and Child TypeDef

This patch provides backward compatibility for two changes mentioned: https://issues.apache.org/jira/browse/ATLAS-3872 Restrict typedef creation when a child type attribute conflicts with parent type attribute of same name https://issues.apache.org/jira/browse/ATLAS-4522. Updating typedef with new supertype should be allowed only if attributes are unique compared to other existing supertypes.

CDPD-49451: Atlas - Upgrade snakeyaml due to CVE-2022-1471

Upgrade snakeyaml to 2.0 from 1.33.

CDPD-50740: [7.1.9]Atlas - Upgrade azure-storage libraries due to CVE-2022-30187

Upgrade azure-storage-blob version to 12.9.0 from 12.20.2 Upgrade azure-storage-queue version to 12.7.0 from 12.15.2.

CDPD-55440: Atlas - Upgrade snakeyaml to 2.0 due to CVE-2022-1471

Upgrade snakeyaml to 2.0 from 1.33.

CDPD-58492: Atlas - Upgrade Netty Project to 4.1.94.Final due CVE-2023-34462

Upgrade Netty Project to 4.1.94.Final from 4.1.86.Final.

CDPD-49452: Atlas - Upgrade Netty to 4.1.86.Final due to CVE-2022-41881, CVE-2022-41915

Upgrade Netty to 4.1.86.Final from 4.1.77.Final.

CDPD-57685: Atlas docs failures in CDH builds

Added the library "fix-esm" to handle the dependency update issue which was causing the build to fail.

CDPD-55617: Atlas - Upgrade Nimbus-JOSE-JWT to 9.24 due to CVEs coming from json-smart

Upgrade Nimbus-JOSE-JWT to 9.24 from 9.8.1.

CDPD-49978: Atlas - Upgrade icu4j to 66.1+ due to CVE-2020-21913

Upgrade icu4j to 66.1.

CDPD-58596: Large number of warn messages logged in tagsync log indicating process entity notification dropped

Added code to send version in the messages sent from Atlas to Ranger, so that the warning messages are not seen.

CDPD-53808: Atlas - Upgrade Spring Framework to 5.3.27/6.0.8 due to CVE-2023-20861, CVE-2023-20860 and CVE-2023-20863

Upgrade Spring Framework to 5.3.27 from 5.3.20.

CDPD-51181: Atlas - Upgrade Woodstox to 5.4.0/6.4.0 due to multiple CVEs

Upgrade Woodstox to 5.4.0 from 5.0.3.

CDPD-49980: Atlas - Upgrade Tinkerpop to 3.5.4

Upgrade Tinkerpop to 3.5.4 from 3.5.2.

CDPD-55876: Atlas - Upgrade Spring Security to 5.7.8+/5.8.3+/6.0.3+ due to CVE-2023-20862

Upgrade Spring Security to 5.8.3 to 5.7.5.

CDPD-55252: Atlas - Upgrade jackson-databind to 2.12.7.1/2.13.4.1+ due to CVE-2022-42003, CVE-2022-42004

Upgrade jackson-databind to 2.12.7.1.

CDPD-50741: [7.1.9]Atlas - Upgrade reactor-netty to 1.0.24+ due to CVE-2022-31684

Upgrade reactor-netty to 1.0.24.

CDPD-54864: Every hive insert generates an Atlas audit event

Introduced "DML audit filters" feature to skip unnecessary DML audit events using configuration property atlas.hook.hive.skip.dml.messages=true (Enabled by default).

CDPD-48641: Atlas - Support for JDK17 in all sub-components

Added JDK17 support for Atlas.

CDPD-56497: Regression : DSL queries redirected to passive server fails

DSL search request sent to Passive server having HTML encoded characters are now properly redirected to Active server.

CDPD-59758: [ST][Atlas] test_export_import_api_sanity test fails

Upgraded tinkerpop version from 3.5.5 to 3.5.6 : This resolved unsupported class major version error faced while running export API in JDK17 Runtime.

CDPD-56594: Lineage (spark_process) is not created for views created on Iceberg tables.

Added lineage support for creating views to Spark Atlas Connector for Spark 3 which is required for Iceberg tables.

CDPD-54990: The in-place migration of Hive table to Iceberg table with ALTER TABLE storage_handler using Beeline creates new iceberg_table entity but retains the old hive_table entity as is

Running the query no longer results with Atlas having two entities with same name but different types. One with hive_table and another with iceberg_table.