Amazon EMR releases 5.30.1, 5.30.2, 5.31.1, 5.32.1, 6.0.1, 6.1.1, 6.2.1, 5.33.0, 6.3.0 and later now include a permanent fix with a higher "Max open files" setting.įor a list of component versions, see About Amazon EMR Releases in this guide. Newer Amazon EMR releases fix the issue with a lower "Max open files" limit on older AL2 in Amazon EMR. This was because the Amazon EMR on-cluster daemon did not renew the Kerberos ticket, which is required to securely communicate with HDFS/YARN running on the primary node. Fixed an issue where job failures occurred due to a race condition in YARN decommissioning when cluster tried to scale up or down.įixed issue with step or job failures during cluster scaling by ensuring that the node states are always consistent between the Amazon EMR on-cluster daemons and YARN/HDFS.įixed an issue where cluster operations such as scale down and step submission failed for Amazon EMR clusters enabled with Kerberos authentication. Fixed an issue where job failures occurred during cluster scale-down as Spark was assuming all available nodes were deny-listed. Improved EMR on-cluster daemons to correctly track the node states when IP addresses are reused to improve reliability during scaling operations. This was happening because on-cluster daemons were not able to communicate the health status data of a node to internal Amazon EMR components. This is a release to fix issues with Amazon EMR Scaling when it fails to scale up/scale down a cluster successfully or causes application failures.įixed an issue where scaling requests failed for a large, highly utilized cluster when Amazon EMR on-cluster daemons were running health checking activities, such as gathering YARN node state and HDFS node state. ChangesĬhanges, enhancements, and resolved issues The following release notes include information for Amazon EMR release 5.32.0.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |